Roadmap · Specialise
MLOps and LLMOps (LLM apps)
Opens with creditsFor developers and ML learners who can train a model or call an LLM but have never shipped one and kept it running. By the end you can package, version, track, serve, test, monitor and retrain a model or an LLM feature, and prove it with a monitored service in production.
Curated by Tauseef Fayyaz10 stages, 72 topics~12 months at a few hours a weekBy the end, you will be able to
- Turn notebook code into a tested, locked and containerised Python project that trains or runs with one command
- Version data, models and prompts so any prediction can be traced back to the exact code, data and configuration that produced it
- Build an orchestrated pipeline that validates data, trains or evaluates, compares with the live version and promotes only on a quality gate
- Serve a model or LLM feature behind an API with sensible latency, batching and cost, and release it with shadow, canary and rollback
- Monitor service health, drift and output quality, trace LLM calls, and decide with evidence when to retrain, re-prompt or fine-tune
The path
10 stages, in the order to learn them. Open one to start.
1From notebook to packageSee the whole lifecycle once, then turn prototype LLM code into a Python project with one module for model calls, locked dependencies, config and secrets outside the code, tests and a container image.8 topics · ~4 weeks
- 1The lifecycle and the tool mapMLOps is the set of practices that keep a model useful after it leaves a notebook.
- 2Why notebook code breaks in productionNotebooks are good for exploring and bad for running things repeatedly.
- 3Project layout and pyproject.tomlA Python project keeps importable code under
src/<package>/, tests undertests/, and declares its name, dependencies, entry points and tool settings in… - 4Dependencies and lockfiles with uvA lockfile records the exact version of every package your project uses, including the packages your packages depend on.
- 5Config and secrets outside the codeConfig is anything that changes between runs or environments: data paths, hyperparameters (settings chosen before training), model names, prompt versions,…
- 6One module for every model callIn an LLM app, the equivalent of a training script is a single module that every feature uses to call a model.
- 7Tests for ML codeMost production ML failures are ordinary bugs in data loading and feature code, not in the maths, so test that code like any other with pytest.
- 8Containerise the projectA container image packages your code, a Python runtime and the locked dependencies into one file system that runs the same way on a laptop, in CI and in the…
8 in this stageOpen this stage2Data and model versioningVersion large data and model files with DVC next to Git, describe the pipeline as reproducible stages, validate data before it is used and freeze an evaluation set every model is judged on.7 topics · ~3 weeks
- 1Why Git alone is not enoughGit is built for text files that change line by line.
- 2DVC: versioning data next to GitDVC (Data Version Control) is an open source command-line tool that versions large files alongside Git.
- 3Reproducible stages with dvc.yamlDVC can also describe how outputs are made.
- 4Validating data before you use itData validation means automatic checks that data looks the way your code expects before anything is trained or scored: the right columns and types, values in…
- 5A frozen evaluation setA frozen evaluation set is a fixed, versioned collection of examples with known correct answers that every candidate model or prompt is scored on.
- 6Versioning documents, chunks and indexesIn a retrieval-augmented generation (RAG) app, the model answers from documents fetched at request time, so the documents are part of the model.
- 7Lineage: from prediction back to dataLineage is the recorded chain from a single prediction back to everything that produced it: the model version that answered, the training run that created it,…
7 in this stageOpen this stage- Experiment tracking and the model registryRecord every run with MLflow so results can be compared and reproduced, run a shared tracking server, and use the model registry with aliases to say which version is live.6 topics · ~3 weeksOpens with credits
- Pipelines and orchestrationBreak training or indexing into steps, run them as a graph with an orchestrator on a schedule or trigger, and make every step safe to retry, rerun and backfill.6 topics · ~5 weeksOpens with credits
- Serving: batch, real time and GPUsChoose between batch and online for each LLM feature, put it behind a validated API with streaming and timeouts, deploy and load test it, and learn the latency, batching and scaling ideas that decide cost.6 topics · ~6 weeksOpens with credits
- CI/CD for MLRun tests and eval sets on every change to code, prompts or models, promote only through an eval gate, and release with shadow, canary and rollback so a bad prompt or model is cheap to catch and undo.7 topics · ~5 weeksOpens with credits
- Monitoring, drift and retrainingWatch service health and model inputs and outputs, log every request with its model and prompt version, detect drift in what users ask, cope with delayed feedback and decide when to re-prompt, re-index or retrain.8 topics · ~5 weeksOpens with credits
- LLMOps: prompts, evals, tracing, caching, costRun LLM features like production software: prompts as code, eval sets and metrics, judges you can trust, traces of every call, caching, token budgets, gateways and defences against prompt injection.9 topics · ~7 weeksOpens with credits
- Fine-tuning and serving open modelsDecide when fine-tuning beats prompting or RAG, fine-tune an open model with LoRA, quantise it, serve it with vLLM and do the GPU memory and cost arithmetic that tells you whether self-hosting pays.8 topics · ~7 weeksOpens with credits
- Capstone: a monitored model in productionPut everything together: one model or LLM feature, live, with versioned data, tracked runs, an orchestrated pipeline, CI/CD with gates, monitoring, a runbook, a cost report and an honest write-up.7 topics · ~2 monthsOpens with credits
8 more stages, 57 more topics, free with an account.
You have seen the open stages of the MLOps and LLMOps roadmap. Every topic it covers is listed above, and an account opens the rest: what each topic means, a question to check yourself, the courses, the practice and Tauseef's advice for every stage. Early access is open: sign up and your account is active shortly after.
Everything here is free to learn from. A few things, like adding a roadmap, keeping a cheat sheet or asking Tauseef a question, use credits. You get 15 when you join, your first roadmap is free, and if you are learning seriously and run out, you can ask for more. Credits only make sure the platform goes to people who are actually learning.
Build a project
A finished project proves more than a certificate.
- Start with this oneadvanced2 semesters · 2 to 3 studentsFine-Tuned Small Open Model with Eval-Gated Releases and Self-Hosted Serving
Fine-tune, gate, serve with vLLM, trace and prove the cost saving against an API baseline
- Python
- Unsloth or Hugging Face TRL and PEFT
- vLLM
- promptfoo
- Langfuse or Arize Phoenix
Related on Sefism
Picked from the other libraries by shared topics.
Best practices
See allChecklists
See allProject structures
See all- Machine learning project project structureFor students and analysts whose model works in a notebook and now needs to be reproducible, reviewable, served and watched. It starts from the Cookiecutter Data Science v2 layout (immutable raw data, numbered notebooks, a real Python package) and grows into versioned data with DVC, typed parameters, data validation with pandera, experiment tracking and a model registry with MLflow, a FastAPI prediction service, drift monitoring and, for a team, scheduled retraining, model cards, fairness checks and shadow deployment.
- MLOps pipeline project structureFor engineers who can train a model and now need the training to be a pipeline anyone can rerun, review and release. A uv workspace holds the pipeline package (prepare, validate, featurize, train, evaluate, register steps run by DVC), a small shared feature package, and a separate FastAPI prediction service that loads models from the MLflow registry by alias. It grows into CI that reproduces the pipeline on pull requests, scheduled retraining, approved promotion and monitoring.
Career questions
See all- My machine learning projects never leave a Jupyter notebook. How do I learn to deploy a model?Move the notebook code into a small Python package, save the whole preprocessing and model pipeline, put it behind a FastAPI endpoint in a Docker container and deploy it. Then add what makes it production: tracked runs, a registry, a test in CI, prediction logging and a drift check. Start with batch predictions if you can.
- How do I become an MLOps engineer?MLOps engineers keep models working in production: packaging, versioning, pipelines, serving, CI/CD and monitoring. Come in from either software or DevOps with ML basics, or from ML with engineering habits. Learn the lifecycle in order, then prove it with one model you deploy, monitor and roll back yourself.
- Should I learn AI and machine learning, or become a strong software engineer first?For most people, become a solid software engineer first, then add AI on top. Most AI jobs today are engineering jobs: building products around models, data pipelines and APIs. Research-heavy machine learning is a different path that needs strong maths and usually a postgraduate degree.
Cheat sheets
See all- Machine learning cheat sheetData, models, deep learning.
- PyTorch cheat sheetTensors, autograd, nn.Module, training loops, data loading, GPUs, mixed precision, torch.compile, DDP and export in PyTorch.
- Hugging Face cheat sheetThe Hub, transformers, tokenisers, datasets, fine-tuning, LoRA, quantisation, inference and embeddings with the Hugging Face libraries.
Work with me
Stuck on something specific?
Writing only gets you so far. If you want an answer to your situation rather than the general case, book a session and we will work through it together. Sessions are free for approved Sefism members, and a few slots open each week.
- 1:1 Mentorship
- Fresh Graduate Mentorship
- Career Guidance
Brand deals, promos and partnerships: tauseeffayyaz@gmail.com