Thin-File Credit Scoring with Alternative Data and a Fairness Audit for Digital Lenders

The problem: millions are scored by models nobody audits

Most adults in Pakistan and much of South Asia have no formal credit history. Mobile wallets and digital lenders now offer small loans to these thin-file borrowers, scoring them with alternative data: wallet transactions, bill payments, phone usage. Done well, this extends credit to people banks ignored. Done badly, it encodes bias against women, younger people or certain regions, and borrowers are refused with no explanation.

Regulators are paying attention to digital lending, and lenders increasingly need to show that their models are explainable and do not discriminate. Students rarely get to work on this side of credit scoring, yet it is where much of the real work in fintech data science lies.

What already exists and the gap

• Credit bureaus score people with formal borrowing histories.
• Alternative credit scoring companies and digital lenders build proprietary models on phone and transaction data, with little public detail about fairness testing.
• Open source fairness tools such as Fairlearn and explanation tools such as SHAP exist, but are usually demonstrated on toy datasets rather than a full credit workflow.

The gap is an end-to-end, documented example of thin-file scoring that treats fairness, calibration and explanation as first-class deliverables, with a model card a regulator or risk committee could read.

Inspiration

Björkegren and Grissen's paper "Behavior Revealed in Mobile Phone Usage Predicts Credit Repayment" (World Bank Economic Review) showed, using call records matched to loan outcomes from a South American telecom, that phone usage patterns can predict default with accuracy approaching traditional credit scores. It is the evidence behind much alternative lending. Your project takes the next step that the paper's setting invites: asking who the model is accurate for, and who it fails.

What you build

• A data scientist trains a model on alternative and traditional features, tracks experiments and compares against a traditional-only baseline.
• A risk reviewer opens the audit dashboard: accuracy, calibration and fairness metrics by group, with the trade-offs of each mitigation.
• An applicant who is declined receives the top reasons in plain language, generated from the model's explanations.
• The model card records everything a regulator would ask about.

The technically interesting part

• What alternative data adds. Measure the lift over the traditional baseline, and specifically for applicants with little bureau history, where it should matter most.
• Fairness metrics conflict. Demographic parity, equal opportunity and calibration within groups cannot all be satisfied at once when base rates differ. Show this on your data, and explain which metric suits which lending decision.
• Proxies. Removing gender from the features does not remove gender from the model; other features can act as proxies. Test how well the sensitive attribute can be predicted from the remaining features.
• Explanations people understand. SHAP values are not reasons. Turn them into a short list such as "irregular income in the last three months" and check that explanations are stable for similar applicants.
• Privacy. Alternative data is personal. Document which features you would refuse to use, such as contact lists or location traces, even if they improved accuracy.

How to evaluate it

• Accuracy: area under the ROC curve and expected loss at chosen approval thresholds, against the baseline.
• Calibration: reliability curves overall and per group.
• Fairness: at least three metrics per group, before and after mitigation, with the accuracy cost of each.
• Explanations: stability across similar applicants and a small reading test with non-technical users.
• Thin-file lift: the accuracy gain from alternative data for applicants with the least history, reported separately from the overall gain.

Suggested plan

• Weeks 1 to 4: data exploration, feature engineering, baseline model.
• Weeks 5 to 9: full model, calibration, experiment tracking.
• Weeks 10 to 14: fairness audit, proxy analysis, first mitigation.
• Second semester: explanations and adverse action reasons, synthetic wallet data, dashboard, model card and write-up.

Path to a product

Every digital lender needs model governance, and smaller lenders cannot build it in house. A fairness and explanation audit service, or an open source audit toolkit with paid support, is a realistic direction. The skills map directly to credit risk, model validation and responsible AI roles at banks and fintech companies.

Scope warning

Do not collect real borrowers' data; use public datasets and synthetic data. Do not build a lending app. Do not claim a model is fair because one metric looks good; show several and explain the trade-offs. The audit, not the score, is the core of this project. If time is short, drop the dashboard polish and keep the proxy analysis and the model card, because those are what a risk committee would actually read.

Learn what you need on Sefism

• Machine learning roadmap: gradient boosting, validation and calibration.
• MLOps and LLMOps roadmap: experiment tracking and model documentation.
• MLflow cheat sheet: logging runs, metrics and model versions.
• Designing Machine Learning Systems: monitoring, drift and responsible deployment.
• Hands-On Machine Learning with Scikit-Learn and PyTorch: the modelling foundations.

Get help with this idea

The risk is a strong model with a shallow audit. Tauseef can help you choose fairness metrics that fit the lending decision, plan the proxy analysis and frame the model card for a real reviewer. If you are new here, create a free account first.

• Discuss this idea with a mentor
• Send a query to Tauseef
• Book a 1:1 session