Start Data Engineering

What it is

Start Data Engineering is a long-running blog, started in 2020 by Joseph Machado, a working data engineer. It publishes hands-on articles and small end-to-end projects, many with a companion GitHub repository you can clone and run. Topics include SQL for data engineers, Python, Apache Spark, dbt, Airflow, Apache Iceberg, data warehousing, pipeline testing, data quality checks and career questions. There is a newsletter, and a Courses and Books section with some paid material.

Why it is useful

Most courses show you how to use a tool. This blog spends more time on how to build a pipeline well:

• Designing pipelines that can be rerun safely (idempotency) and backfilled.
• Testing data pipelines: unit tests for transformation code and data quality checks on outputs.
• Structuring a data project repository, with Docker, a Makefile and CI.
• Explaining concepts such as slowly changing dimensions, incremental loads and partitioning with concrete code.

Those are the things that separate a portfolio project that impresses a reviewer from one that only runs once on the author's laptop.

How to use it well

Do not read it like a feed. When you reach a stage in your roadmap (orchestration, dbt, Spark, testing), search the site for that topic and work through one project article completely, including running the code. Then apply the same pattern to your own dataset. Recent posts discuss how AI tools change day-to-day data engineering work, which is useful context without hype.

Prerequisites

Basic Python, SQL and Docker. Many articles assume you can clone a repository and run docker compose up.

Free versus paid

The blog articles are free to read. Some courses or books listed on the site are paid; they are optional, and the free articles alone cover a great deal.

Is it worth your time?

Yes, as a companion to a structured course. It is written by a practitioner, focuses on patterns that last longer than any single tool version, and the project repositories are a good template for your own portfolio.

Common questions

Is it beginner friendly? Mostly intermediate. Start with a course such as the Data Engineering Zoomcamp, then use this blog to deepen each topic.

Are the older posts out of date? Some tool versions have moved on (Airflow 3 and dbt v2 changed details), but the design ideas hold. Check commands against current official documentation.