Prefect Documentation

What Prefect is

Prefect is a workflow orchestration framework for building data and ML pipelines in Python. Its own description is the simplest way to turn a script into a production workflow, and that matches the experience: you add @task and @flow decorators to ordinary functions, and Prefect adds retries, caching, logging, scheduling and a UI that records every run. It needs Python 3.10 or newer.

Why it suits MLOps learners

Many learners stall between a training script that works and a pipeline that runs every week without them. Prefect closes that gap with very little new infrastructure. Run prefect server start to get the UI locally, then call .serve() on a flow with a name and a cron schedule, and you have a deployment that runs on schedule, retries failed tasks and shows its history. The same code later runs on workers in the cloud or on Kubernetes when you need it.

The concepts to learn first

• Flows and tasks: the units of work, with parameters, return values and retries.
• Deployments: a flow plus when and where it runs, on a schedule, by hand or in response to events.
• Results and caching: skipping tasks whose inputs have not changed.
• Work pools and workers: where runs actually execute once you move beyond your laptop.
• Automations and events: reacting to something that happened, such as notifying a channel when a run fails.

How long it takes

The Get started pages and a first scheduled flow take an afternoon. Deployments, work pools and automations are a weekend of reading and experimenting.

Free or paid?

Prefect is open source under the Apache 2.0 licence, and you can self-host the Prefect server for free. Prefect Cloud is the company's managed service with extra features for teams; check its official pricing page for current plans.

How it compares

Apache Airflow is the long-standing standard in data teams and worth knowing if your employer uses it. Dagster is built around the data assets pipelines produce. Prefect's strength is how little code changes between a script and a scheduled flow, which makes it a good first orchestrator.

Common questions

Do I need Kubernetes? No. A local server and a process running .serve() are enough to learn and for small projects.

What can I do after it? Schedule retraining or re-indexing with retries, failure notifications and a history of every run, which is the core of the orchestration stage in the MLOps roadmap.

Best for: Python developers and ML engineers who want their first reliable, scheduled pipeline without a heavy platform.