MLOps pipeline project structure

MLOps pipeline project structure

Explorer

  • .venv…, not committed
  • .git…
  • .gitignore
  • .python-version
  • README.md
  • pyproject.toml
  • uv.lock
  • .env, not committed
  • .env.example
  • mlflow.db, not committed
  • mlruns…, not committed
  • notebooks
  • 01-explore-deliveries.ipynb
  • 02-error-analysis.ipynbnew
  • data
  • raw.dvcnew
  • .gitignorenew
  • prepared…new, not committed
  • features…new, not committed
  • src
  • packagesnew
  • params.yamlnew
  • dvc.yamlnew
  • dvc.locknew
  • .dvcnew
  • confignew
  • config.localnew, not committed
  • .gitignorenew
  • cache…new, not committed
  • .dvcignorenew
  • models…new, not committed
  • metricsnew
  • metrics.jsonnew
  • slices.jsonnew
  • tests
  • test_smoke.py
  • conftest.pynew
  • Makefilenew
  • .pre-commit-config.yamlnew
  • docker-compose.ymlnew
  • AGENTS.mdnew

  1. # Create the project (uv makes a src/ package and a git repository)

    uv init eta-mlops && cd eta-mlops

  2. # Add the training stack and track runs locally (MLflow defaults to a SQLite mlflow.db)

    uv add pandas pyarrow scikit-learn lightgbm mlflow pydantic-settings

  3. # Open the MLflow UI to compare runs

    uv run mlflow ui --port 5000

  4. # Growing: version data and define the pipeline

    uv add "dvc[s3]" pandera typer pyyaml && uv run dvc init && uv run dvc add data/raw

  5. # Growing: run the pipeline; DVC skips steps whose inputs did not change

    uv run dvc repro

  6. # Growing: try a parameter without editing params.yaml

    uv run dvc exp run -S train.num_leaves=63

  7. # Growing: dev tools as a dependency group

    uv add --dev pytest pytest-cov ruff mypy pre-commit

  8. # Production: add the prediction service as a workspace member

    uv init --package services/predictor && uv add --package predictor "fastapi[standard]" mlflow-skinny