What Actually Happens Between Your Pull Request and Production
Most engineers learn CI/CD as a sequence of buttons rather than a system, which works right up until the pipeline goes red and nobody knows which stage failed or why it matters. The five stages walked through one at a time, what each is protecting you from, why builds pin their versions, the difference between rolling, blue green and canary releases, and the ten second skill of reading a failed stage.

Tauseef Fayyaz

The gap nobody explains
You open a pull request. Some checks appear. A colleague leaves comments. Eventually it merges, and at some point after that your change is live.
Most people learn this as a sequence of buttons rather than as a system. That is fine until something breaks in the middle of it, and then you are staring at a red pipeline with no idea which stage failed or why it matters.
Here is the whole thing, stage by stage, and what each one is actually protecting you from.
Stage one: the change itself
You branch, you commit, you push, you open a pull request.
The parts worth caring about at this stage are unglamorous. Keep the branch short lived, because a branch that lives for three weeks turns into a merge conflict you will spend a day on. Write a commit message that says why rather than what, because the diff already says what. Keep the pull request small enough that a human can actually hold it in their head, which in practice means a few hundred lines rather than a few thousand.
That last one has more effect on your review quality than anything else on this list. A reviewer given 60 lines finds real problems. A reviewer given 2,000 lines approves it.
Stage two: build
The moment you push, something notices and starts working.
The build turns your source into an artifact: usually a container image, sometimes a package. The important property is that this artifact is the thing that goes to production. Not a rebuild later, not the same code compiled again on a different machine. The exact bytes that were tested are the exact bytes that ship.
That is why builds pin their dependency versions and their base images. If your build pulls latest of anything, then two builds of the same commit can produce different software, and every test result you have is about a thing that no longer exists.
The artifact goes into a registry so that staging and production can both pull the same one.
Stage three: test
This stage exists to answer one question: is there a reason not to let this go further?
It usually runs in layers, fastest first, because feedback speed matters:
Linting and formatting. Trivial, instant, and it removes an entire category of pointless review comments.
Unit tests. Does each piece behave as expected in isolation. Fast, and they should stay fast, because a slow unit test suite is one people stop running locally.
Integration tests. Does it still work when the pieces are connected and a real database is involved. Slower, fewer of them, and they catch the things unit tests structurally cannot.
Security scanning. Known vulnerable dependencies, secrets accidentally committed, obvious injection patterns. Cheap to run and occasionally saves you from something serious.
Human review. Another engineer reads the change. This is not redundant with the automated checks, because it catches a different class of problem entirely: wrong approach, missing case, something that works but that nobody will be able to maintain.
If any layer fails, the change stops here. That is the point. A quality gate that can be waved through when you are in a hurry is not a gate.
Stage four: deploy
It passed. Now it has to reach production without breaking anything currently running.
The naive version is to replace everything at once, which works right up until the new version is broken and now everything is broken. So real deployments move gradually:
Rolling. Replace instances a few at a time. Simple, and for a while you are running two versions at once, which your code has to tolerate.
Blue green. Two full environments. Deploy to the idle one, test it, then switch traffic. Rollback is a switch back, which is as fast as it gets. It costs twice the infrastructure.
Canary. Send a small slice of real traffic, maybe two per cent, to the new version. Watch the error rate. If it holds, widen it. If it does not, pull it back having affected very few people.
Canary is the one worth understanding properly, because it is the only strategy where production traffic tells you the truth before most of your users are exposed to it.
Stage five: monitor
Deployment is not the end. It is the moment you start paying attention.
Watch error rate, latency, traffic and resource saturation. Compare them against what they were an hour ago, before your change. Most bad deployments are visible in those four numbers within minutes.
Alerts need an owner. An alert that fires into a channel where everyone assumes somebody else is handling it is decoration.
The words, since they get used loosely
Continuous integration means everyone merges into the shared branch frequently, and every merge is built and tested automatically. The point is that integration problems surface hourly instead of at the end of a quarter.
Continuous delivery means every change that passes the pipeline is deployable. It might wait for a human to press a button, but nothing technical is stopping it.
Continuous deployment means it goes out automatically, with no button.
Most teams say the third and practise the second, which is fine. The distinction only matters when someone is describing what they do.
What this is actually for
The usual pitch is speed. That is the smaller half.
The real function is risk reduction. Small changes, tested automatically, released gradually, watched closely, and reversible in minutes. Every one of those stages exists because at some point somebody shipped something that broke, and the stage is the scar.
Shipping faster is a side effect of the risk being lower. Teams that deploy ten times a day are not braver than teams that deploy monthly. They have made each deployment small enough not to require bravery.
If you are early in your career
You do not need to build a pipeline to benefit from understanding one. What you need is to stop treating the red X as an obstacle between you and merging.
When a stage fails, read which stage it was. Build failure means your change does not compile somewhere, or a dependency moved. Test failure means behaviour changed, and possibly correctly, in which case the test is what needs updating. Security scan failure usually means a dependency needs bumping.
Knowing which of those you are looking at, in the first ten seconds, is a genuinely useful skill and almost nobody teaches it explicitly.
Comments (0)
Comments are closed for now.
No comments yet.
Stuck on something specific?
Writing only gets you so far. If you want an answer to your situation rather than the general case, book a session and we will work through it together. Every session is free; a few slots open each week.
Follow along
New writing, resources and project ideas land here first.