Sefism MLOps and LLMOps Cheat Sheet

What this cheat sheet is

The MLOps and LLMOps cheat sheet is the revision companion to Sefism's MLOps and LLMOps roadmap. It is for ML engineers and developers who want to take a model from a notebook to production and keep it working, and it covers packaging, data and model versioning, experiment tracking, pipelines, serving, CI/CD, monitoring and drift, LLMOps and open models.

Use it to revise a chapter before a project, pick the right tool for each step and spot the classic mistake early. Each concept explains what it is and why it matters, and many include a snippet or a pitfall.

Two editions, one per focus

The sheet follows the roadmap's focus:

• Classic ML (the default): for shipping tabular and deep learning models. Snippets use Python, scikit-learn, Prefect, FastAPI and GitHub Actions, and the chapters cover training entry points, model artefact formats, model signatures, feature stores, transfer learning, ONNX and GPU serving. 70 concepts.
• LLM apps: for running LLM features in production, with prompts and configs as code, versioned eval sets and indexes, prompt registries, RAG ingestion, token streaming, eval gates, LoRA and QLoRA, the KV cache and serving with vLLM. It keeps the classic chapters too, because the same discipline applies. 75 concepts.

Both editions have the same ten chapters.

Chapter by chapter

1. From notebook to package

The MLOps lifecycle and a tool map per stage, project layout and pyproject.toml, dependencies and lockfiles, config and secrets, tests for ML code and the container image, plus a training entry point and artefact formats (classic) or an LLM client module (LLM apps).

2. Data and model versioning

DVC, DVC pipelines, data validation, the frozen evaluation set and lineage, with versioning documents and indexes added on the LLM edition.

3. Experiment tracking and the registry

Experiment runs, tracking servers and stores, autologging, the model registry and model aliases, with model signatures (classic) or prompt registries and tracking LLM experiments (LLM apps). The standalone MLflow cheat sheet goes deeper into the tool itself.

4. Pipelines and orchestration

Training pipeline steps, the DAG, orchestrators, a Prefect flow snippet, idempotent tasks, and schedules, triggers and backfills, with a feature store (classic) or a RAG ingestion pipeline (LLM apps).

5. Serving: batch, real time and GPUs

Serving options compared, batch and online inference, serving frameworks, latency percentiles, dynamic batching, and autoscaling and cold starts, with token streaming and time to first token on the LLM edition.

6. CI/CD for ML

MLOps maturity levels, CI for ML, the model quality gate, champion and challenger, shadow deployment, canary releases and A/B tests, and rollback, with eval gates in CI for prompts on the LLM edition.

7. Monitoring, drift and retraining

Service metrics, prediction logging, types of drift, concept drift, the population stability index, drift reports with Evidently, label delay and proxies, and retraining strategy. Several entries carry snippets.

8. LLMOps: prompts, evals, tracing, cost (10 concepts in both editions)

Prompts as code, LLM eval sets and metrics, LLM as a judge, tracing, caching LLM calls, token cost maths, model gateways and fallbacks, prompt injection and guardrails, and online evals and feedback.

9. Fine-tuning and serving open models

Quantisation, GPU memory and cost maths and evaluating a tuned model in both editions. Classic adds transfer learning, ONNX and optimised runtimes and serving deep learning models on GPUs; LLM apps adds prompting versus RAG versus fine-tuning, LoRA and QLoRA, training data and chat templates, the KV cache and serving with vLLM.

10. Capstone: a monitored model in production (5 concepts)

The documents and habits that make a deployed model ownable: the service level objective, the model card, the runbook, the blameless postmortem and cost per prediction.

Each chapter lists up to three library resources to go deeper with, including Made With ML, DVC Documentation, MLOps Zoomcamp, Evidently Documentation, Langfuse Documentation and vLLM Documentation.

Rules of thumb and glossary

Six rules are shared by both editions, with two more for classic ML and three more for LLM apps. A few of them: if a model cannot be rebuilt from a commit, a data version and one command, it is not ready for production; every promotion is a comparison with the current model on the same frozen evaluation set; monitor the service and the model separately, because a healthy endpoint can serve bad predictions for months; move an alias to deploy and move it back to roll back; and on the LLM edition, check an LLM judge against human labels before trusting its scores.

A glossary of 80 short terms sits alongside, and every concept joins it automatically.

How to use it with the roadmap

• After each stage, read its chapter and try to explain each concept in your own project's terms. Roadmap topics link straight to the concepts that cover them.
• While building, keep the serving, CI/CD and monitoring chapters open: latency percentiles, quality gates and drift checks are where projects most often cut corners.
• Before a technical conversation, reread the capstone chapter. Being able to talk through an SLO, a runbook and the cost per prediction of something you ran shows you have operated a model, not just trained one.

How it works on Sefism

Everything on Sefism is free to learn from. The public page shows all ten chapters with their introductions and opens the first two in full. Keeping the sheet uses credits and opens every chapter in the member area; new members get starting credits when they join. Adding the MLOps and LLMOps roadmap to your dashboard (your first roadmap is free) brings this sheet with it.

Related on Sefism

• How to become an MLOps engineer
• How to deploy a machine learning model from a notebook
• Evaluation Harness for Language Model Applications, a project that exercises chapter 8.
• The Machine learning cheat sheet and AI engineering cheat sheet for the modelling and LLM ground underneath.

FAQ

What does an MLOps cheat sheet need to cover? Packaging, versioning, tracking, orchestration, serving, CI/CD, monitoring and the production habits around them. This one adds LLMOps and open models.

Which edition should I pick? Classic ML if you ship tabular or deep learning models, LLM apps if you run LLM features. Both keep the shared chapters.

Is it tied to particular tools? The snippets use common open source tools such as DVC, MLflow, Prefect, FastAPI, Evidently and vLLM, but the concepts apply whichever tools your team uses.

Is this enough to learn MLOps? No. Learn through the roadmap's stages and the resources each chapter lists, and use the sheet to revise.