Data engineering

Stage 1 · Lesson 8 of 8

A local practice setup

What data engineers do

Set up a workspace on your own computer that you can break freely. Install Python with a tool, plus Git, VS Code and . Docker runs software inside a container, a sealed box with everything it needs.

Run PostgreSQL in a container to act as a pretend source system, like an app's database. Also install . It is a small that runs inside your program, reads CSV and Parquet files directly and behaves much like a .

Keep one Git repository for everything you build in this roadmap, with a README that says how to run it. Pick one realistic dataset to reuse in every stage. Good options are the New York taxi trip records, a public e-commerce dataset or your country's open data portal.

Common trapstarting in the cloud on day one and paying for resources while you are still learning basic SQL.

A question to check yourself, an AI practice session and 2 cheat sheet entries come with the full roadmap. Unlock

Resources for this stage

Start with· follow one of these

Keep handy

Also worth your time

Mark this stage as not done?