MLflow cheat sheet

1. 1Tracking experimentsExperiments, runs and what you log to them: parameters, metrics, tags, artefacts and datasets.8 concepts

   1. 1What MLflow isAn open-source platform for the machine learning life cycle: tracking experiments, packaging models, a model registry, serving, evaluation, and tracing for…
   2. 2Experiments and runsA run is one execution of training or evaluation code.
   3. 3ParametersInputs to a run, such as hyperparameters and settings, stored as strings.
   4. 4MetricsNumeric results logged over time.
   5. 5TagsFree-form key-value labels on runs, experiments and models.
   6. 6ArtefactsOutput files attached to a run: plots, reports, feature lists, model files, sample predictions.
   7. 7Dataset trackingmlflow.data wraps a DataFrame or table with its source, name, digest and schema; log_input records it on the run with a context such as training or…
   8. 8System metricsWith log_system_metrics=True on a run (the psutil package is needed, plus nvidia-ml-py for GPUs), MLflow samples CPU, memory, disk and GPU use during…

   8 in this chapterOpen this chapter
2. 2Autologging, search and the UICapturing runs with one line, organising hyperparameter searches, and finding the best run in code or in the UI.7 concepts

   1. 1Autologgingmlflow.autolog() hooks into supported libraries such as scikit-learn, XGBoost, LightGBM, PyTorch Lightning and Keras, and logs parameters, metrics and the…
   2. 2Library-specific autologEach integration also has its own autolog, such as mlflow.sklearn.autolog(), with options like log_models, log_datasets and disable.
   3. 3Nested runsRuns started with nested=True inside another run become its children, and the UI groups them under the parent.
   4. 4Searching runsmlflow.search_runs returns a pandas DataFrame of runs matching a SQL-like filter on metrics, params, tags and attributes, in the order you ask.
   5. 5MlflowClientThe lower-level API for reading and editing runs, experiments, metric histories and registered models by id.
   6. 6The MLflow UIA web app listing experiments and runs, with metric charts, side-by-side run comparison, parallel coordinates plots, artefact browsing, models and traces.
   7. 7Organising runsClear experiment names, meaningful run names and consistent tags, plus descriptions on experiments explaining the question being explored.

   7 in this chapterOpen this chapter
3. Tracking server and storesWhere runs and files live, and how to run a shared tracking server for a team.7 conceptsOpens with credits
4. Models, flavours and signaturesThe MLflow Model format: how models are logged with their environment and schema, and loaded anywhere through one interface.9 conceptsOpens with credits
5. Model RegistryNamed, versioned models with aliases that decide which version is live, and a clear promotion path.7 conceptsOpens with credits
6. Serving, deployment and projectsTurning a logged model into a REST endpoint, a container or a batch job, and packaging training code with MLflow Projects.7 conceptsOpens with credits
7. EvaluationScoring classical models and GenAI apps in a repeatable way, and using the results to decide what gets promoted.6 conceptsOpens with credits
8. GenAI tracing and promptsObserving LLM applications with traces, and versioning prompts in the registry alongside models.7 conceptsOpens with credits
9. MLOps practiceThe habits and pipelines around MLflow that make models reproducible, reviewable and safe to change.7 conceptsOpens with credits
10. 10Glossary, A to ZEvery concept in the sheet and the smaller terms it uses, in one list.83 terms

Experiment tracking, the tracking server, models and signatures, the Model Registry with aliases, serving, evaluation, GenAI tracing and MLOps.