You already have the hardest part: SQL, business context and knowing what good data looks like. Add Python for pipelines, data modelling, an orchestrator, a cloud warehouse, Git and testing. Build one end-to-end pipeline, then move internally by taking on engineering work in your current team before applying outside.
What people tell me
I work as a data analyst. I write SQL every day and build dashboards, but I spend most of my time fixing broken data and waiting for someone else to change the pipelines. I want to become a data engineer because I enjoy the technical side more than presenting reports. I am not sure which skills I am missing, whether I need a new degree, or how to make the move without starting again as a junior.
A composite of the messages behind this question, with personal details left out.
Key takeaways
- Strong SQL and knowing how the business uses data are real advantages; do not undersell them.
- The main gaps are usually Python, data modelling, orchestration, cloud warehouses, Git and testing.
- An internal move is often the easiest route: take on pipeline work before changing your title.
- One end-to-end pipeline project with tests and documentation is worth more than several certificates.
You are closer than you think
Data analyst to data engineer is one of the most natural moves in tech. You already write SQL daily, you know which tables matter, and you have felt the pain of bad data first-hand. Data engineers who have never been on the receiving end of a broken pipeline often build systems that are technically clean but useless to analysts. You will not make that mistake.
The difference between the roles is mostly about what you build:
| Data analyst | Data engineer | |
|---|---|---|
| Main output | Reports, dashboards, answers to questions | Reliable pipelines and well-modelled tables |
| Main tools | SQL, spreadsheets, BI tools | SQL, Python, orchestrators, warehouses, Git |
| Cares most about | Is the insight correct and useful? | Is the data correct, on time and cheap to produce, every day? |
| Failure looks like | A wrong number in a meeting | A pipeline that silently stops at 3 a.m. |
The skills you probably need to add
- Python for data work. Not data science: reading files and APIs, transforming data, handling errors, writing functions and small modules. Automate the Boring Stuff with Python is a gentle start if you are new.
- Git and code review. Pipelines are software. Everything goes in version control.
- Data modelling. Facts and dimensions, slowly changing dimensions, and why wide tables are sometimes fine. Kimball Dimensional Modeling Techniques is the classic reference.
- Transformation in SQL with tests. dbt is widely used for this. dbt Learn is free and fits analysts well because it builds on SQL you already know.
- Orchestration. Scheduling and monitoring jobs with a tool such as Airflow. See the Apache Airflow documentation.
- A cloud data warehouse and storage. Loading data, partitioning, and understanding what makes queries expensive.
- Data quality. Tests for nulls, duplicates, freshness and row counts, plus alerts when they fail.
Later, depending on the roles you target: streaming, Spark and larger-scale processing.
Free structured courses
Data Engineering Zoomcamp is free, project-based and covers most of the list above in one course. Start Data Engineering has practical articles, and Fundamentals of Data Engineering is the book that ties the concepts together.
Build one pipeline end to end
Pick a public data source that changes regularly: a public API, government open data or anything with daily updates. Then:
- Extract it on a schedule with Python.
- Load the raw data into a warehouse or a local database.
- Transform it into clean, modelled tables with dbt, with tests.
- Orchestrate it with Airflow or a similar tool, with retries and alerts.
- Put a small dashboard on top, which you already know how to do.
- Write a README with an architecture diagram, what breaks and how you would scale it.
This shows the full cycle and plays to your analyst strength at the end.
Move internally first if you can
The easiest switch often happens inside your current company:
- Tell your manager you want to grow towards data engineering.
- Volunteer to fix the pipelines that break your reports, or to move a manual spreadsheet process into a scheduled job.
- Offer to write data quality tests for the tables you rely on.
- Sit with the data engineering team and review their pull requests to learn their conventions.
After six months of this, you have real data engineering work on your CV, and sometimes the title follows without a job search.
When you apply outside
Titles such as analytics engineer are a common bridge: heavy SQL and modelling, with software practices. Rewrite your CV around the engineering parts of your analyst work: automation you built, data quality problems you fixed, time you saved. You are not starting again as a junior if you present it properly.
If you are still stuck
If you are unsure whether to stay closer to analytics, the Data analytics roadmap and cheat sheet show the depth available on that side too. If you want help planning the move from your current role, join Sefism and, as a member, book a 1:1 session.
Was this answer helpful?
Read next
Your situation is not quite this one?
Members get written answers to their own questions, a roadmap built for them and feedback on their projects. Early access is open: sign up and your account is active shortly after. Prefer to talk? A free call works too.