Fairlearn
An open source, community-driven Python toolkit to assess and improve the fairness of machine learning models: group fairness metrics with MetricFrame, mitigation algorithms and a user guide that explains the social side of fairness.
Tool
AI & Machine Learning
Fairlearn contributors
An afternoon for the user guide and a first assessment
Intermediate
Free
What it is
Fairlearn is an open source Python project that helps data scientists assess and improve the fairness of AI systems. It is community-driven and maintained by contributors from many backgrounds. You install it with pip install fairlearn and use it alongside scikit-learn style models.
What you can do with it
- Assess.
MetricFramecomputes any metric (accuracy, false positive rate, selection rate) broken down by groups such as gender or region, and shows the gap between the best and worst group. This is the core of a fairness assessment. - Mitigate. Algorithms that reduce measured disparities, either by adjusting the training process or by post-processing the model's decisions with group-specific thresholds.
- Learn. The user guide is one of the best free explanations of what fairness means in practice. It is honest that fairness is a socio-technical problem: no metric can decide on its own whether a system is fair, and different fairness definitions can conflict with each other.
Why it matters for governance
Fairness testing is one of the main controls in model assurance, and regulators and frameworks (the EU AI Act's data governance requirements, the NIST AI RMF, ISO/IEC 42001) expect organisations to look for harmful bias. A governance professional must be able to read a disaggregated metrics table and ask the right questions. An engineer must be able to produce one.
How to use it
Take a small public tabular dataset, train a simple classifier and compute selection rate and false negative rate by group with MetricFrame. Write one paragraph on what the gaps mean for the people affected, and which fairness definition matters most for this use case. That paragraph is the governance work; the code is the easy part.
Limitations
Fairlearn focuses on group fairness for classification and regression. It does not test large language model outputs directly, and it cannot tell you which groups matter or whether you have the legal right to process sensitive attributes. Those are decisions for people, documented in the impact assessment.
Free or paid?
Free and open source.
Ready to start?
Opens on Fairlearn contributors in a new tab.
Stuck on something specific?
Writing only gets you so far. If you want an answer to your situation rather than the general case, book a session and we will work through it together. Sessions are free for approved Sefism members, and a few slots open each week.
Follow along
New writing, resources and project ideas land here first.