Sefism is free for learners. We are here to help you grow, one step at a time.Join for free
Sefism.
Cloud & DevOps

GitOps Agent for Docker Compose Servers with Drift Detection

A pull-based GitOps agent for teams that run Docker Compose on ordinary VPS instead of Kubernetes: it reconciles each server with a Git repository, detects and reports manual drift, and rolls back a failed deploy using health checks. An intermediate cloud and DevOps final year project.

Topics
Difficulty

Intermediate

A small team, or one strong student willing to learn something new.
Effort

2 semesters · 2 to 3 students

Deliverables

6 to ship

3 optional extras

Suggested stack

GoDocker Engine APIDocker ComposeGit (go-git)SOPS or age for secretsPostgreSQLReact fleet view
A suggestion, not a requirement. Swap anything for what you already know.

What you should ship

  • An agent that polls or receives webhooks from a Git repository and brings each Compose stack on the server to the committed state.
  • Drift detection that compares running containers, images, environment, ports and volumes with the repository and reports each difference with a clear explanation.
  • A health-gated deploy: after updating a stack the agent waits for container health checks and an HTTP probe, and reverts to the previous commit if they fail.
  • Encrypted secrets stored in Git with SOPS or age and decrypted only on the target server.
  • A small control service and fleet view showing every server, the commit it runs, drift status and deploy history.
  • A test suite that injects drift and failed deploys on at least three servers and records how the agent responds.

If you have time left

  • A "pause reconciliation" switch with an expiry, so an engineer can debug live without the agent fighting them.
  • Offline tolerance for servers on unreliable links: queue reports and apply the latest commit when the connection returns.
  • A pull request comment that previews exactly which containers a change will recreate on which servers.

The problem: servers that no longer match the repository

Many small companies do not run Kubernetes, and for their size they are right not to. They run a handful of VPS, each with a docker-compose.yml, and deploy by SSHing in, running git pull and docker compose up. It works, and then it drifts. Someone edits an environment variable on the server during an incident and forgets to commit it. Someone pins an image by hand. Six months later the repository no longer describes what is running, and nobody can rebuild the server from it with confidence.

GitOps fixes this on Kubernetes: the repository is the source of truth and an agent in the cluster keeps reality in line with it. The same idea is valuable, and simpler, for Compose-based teams, but most of the tooling assumes a Kubernetes control plane.

What already exists and the gap

  • Argo CD and Flux are the standard GitOps tools, built for Kubernetes clusters.
  • Komodo manages Docker servers and stacks with a Git-backed sync model and a web interface.
  • doco-cd is a small Compose-native agent that watches a repository and redeploys on change.
  • Portainer offers Git-based stack deployment as part of its container management platform.

The gap is the part after "deploy on change": explaining drift in human terms, gating a deploy on health and rolling back automatically, handling secrets in Git properly, and giving a small team one view of which commit every server is running.

What you build

  • Agent: a single Go binary on each server. It clones the repository, reads the stacks assigned to the server, computes the desired state and talks to the Docker Engine API to reconcile.
  • Drift engine: compares desired and actual state field by field and produces a report such as "container api: env LOG_LEVEL is debug on server, info in Git (changed 3 days ago)".
  • Deploy controller: applies changes stack by stack, watches health, and rolls back to the last good commit on failure, recording why.
  • Secrets: encrypted files in the repository, decrypted with a key held only on the server.
  • Control service: agents report status; a simple web page shows the fleet, drift and history.

The technically interesting part

Computing a meaningful diff between a Compose file and running containers is harder than it looks. Compose adds labels, default networks and generated names; images are referenced by tag in Git but run by digest; environment variables can come from several files. You must normalise both sides so that real drift shows up and noise does not. The second hard part is the reconciliation loop itself: it must be idempotent, safe when two commits arrive quickly, and never leave a stack half-updated. Show your evaluators the state machine for a deploy and the normalisation rules for drift, with tests for each.

How to evaluate it

  • Drift detection: script at least 15 kinds of manual change (env edits, port changes, image swaps, stopped containers, extra containers) and report how many are caught with a correct explanation, and how many false positives appear on an untouched server over a week.
  • Deploy safety: push deliberately broken commits (bad image, failing health check, missing secret) and measure rollback time and whether the service stayed up.
  • Convergence time: seconds from push to all servers running the new commit, with webhooks and with polling.
  • Resource use: agent memory and CPU on a small VPS.
  • Pilot: run it for a month on a real side project or a university society's servers.

Suggested plan

  1. Semester 1, weeks 1 to 5: agent skeleton, Git sync and basic reconcile for one stack.
  2. Weeks 6 to 10: desired and actual state model, normalisation and drift reports.
  3. Weeks 11 to 14: health-gated deploys and rollback; mid-project demo.
  4. Semester 2, weeks 1 to 5: secrets, multi-server assignment and the control service.
  5. Weeks 6 to 10: fault injection suite, pilot deployment.
  6. Weeks 11 to 14: measurements and write-up.

Path to a product

The first customer is a software house hosting twenty client apps on Compose servers, where drift regularly causes "works on staging" surprises. Release the agent as open source and sell a hosted control plane per server per month, with drift reports and audit history as the paid features. Agencies in Pakistan and elsewhere that host client work on low-cost VPS are the core market.

Scope warning

Do not reimplement Docker Compose; call the Docker API and reuse the Compose file parser. Do not add Kubernetes support. Do not build a general server management panel with terminals and file browsers. Keep the web view read-only in the first version.

Learn what you need on Sefism

Get help with this idea

Most of the grade sits in the drift normalisation rules and the deploy state machine, so bring those to a mentor early. A mentor can help you set the scope, pick a stack you can finish with, and design an evaluation your examiners will respect. The build is yours; the guidance is on scope, stack and evaluation. New here? Create a free account first.

Ideas and guidance, not finished projects

These are project ideas and scoping guidance, published free for students to use as a starting point. I do not build, write, or sell final-year projects, and I do not complete coursework for anyone. Take an idea, make it yours, and build it.

Not sure this one fits you?

I help students pick an idea that matches their skills and their deadline, then scope it down until it is finishable. Guidance only, never done for you.

Discuss this idea
Work with me

Stuck on something specific?

Writing only gets you so far. If you want an answer to your situation rather than the general case, book a session and we will work through it together. Sessions are free for approved Sefism members, and a few slots open each week.

Book a free sessionSee all services

Follow along

New writing, resources and project ideas land here first.