Early Warning System for Student Disengagement

The problem instructors face with struggling students

Students who fail a course usually show signs weeks earlier: submissions arriving later and later, attendance thinning, no activity on the learning platform. In a class of 150, an instructor cannot watch every pattern, and by the time the final result is recorded, the moment to help has passed. Universities with large first-year cohorts see this every term, and students who fall behind early are the ones most likely to drop out.

The data to spot this already sits in the attendance system and the learning platform. It is rarely combined, and almost never turned into something an instructor can act on in week five.

What already exists and the gap

• Moodle Learning Analytics includes a built-in model for predicting students at risk of dropping out, based on activity in a course.
• EAB Navigate and Civitas Learning are commercial student success platforms used by universities, mainly in the United States.
• Purdue's Course Signals was an early and widely studied early warning system that showed traffic-light risk indicators to students and instructors.
• The Open University Learning Analytics Dataset (OULAD) is a public, anonymised dataset of student demographics, assessments and virtual learning environment activity.

Commercial platforms are priced for large institutions, and built-in models often give a label without reasons or any check for bias. The gap is a transparent system that names its reasons, audits fairness openly and is designed from the start to support human judgement.

What you build

• Feature pipeline: weekly engagement features, such as days since last login, share of assessments submitted on time, change in activity compared with the student's own earlier weeks, attendance rate.
• Risk model: start with logistic regression, then try a tree-based model; choose based on validation and explainability.
• Explanations: per-student reasons, from model coefficients or SHAP values, written in plain language ("three consecutive late submissions, no platform activity for 12 days").
• Fairness audit: error rates by group, using attributes that are not model inputs.
• Dashboard: a weekly at-risk list for instructors, with reasons and a note field for follow-up.
• Policy document.

The technically interesting part

The constraint that defines the project is use behaviour, not identity. Gender, ethnicity, nationality and socioeconomic proxies must not be inputs, and you must still audit outcomes across those groups, because a model trained on engagement can encode them indirectly. A student working two jobs shows the same signals as a disengaged one, and a system that cannot tell the difference will flag the wrong people. Show your supervisor the audit: false positive and false negative rates by group, what you changed when you found a gap, and what you could not fix.

The second is early prediction. A model that is accurate in week 12 is useless. Evaluate at weeks 3, 5 and 8 and show the trade-off between earliness and accuracy.

How to evaluate it

• Temporal validation: train on earlier cohorts, test on a later one; report precision and recall at the operating threshold for weeks 3, 5 and 8.
• Lead time: median number of weeks between first flag and final result for students who failed.
• Fairness: error rates by group with confidence intervals.
• Instructor test: 3 to 5 instructors review a sample list and rate whether the reasons are clear and actionable.

Suggested plan

1. Weeks 1 to 2: ethics approval or public dataset (OULAD), data understanding.
2. Weeks 3 to 5: weekly feature pipeline.
3. Weeks 6 to 8: models, temporal validation, explanations.
4. Weeks 9 to 10: fairness audit and fixes.
5. Weeks 11 to 13: dashboard and policy document.
6. Weeks 14 to 15: instructor test and report.

Path to a product

The first customer is your own department or a college with a large first-year intake and a Moodle installation. Offer a one-term pilot with one course, then an annual licence per institution, ideally delivered as a plugin for the learning platform they already use. Student success offices, not individual instructors, usually hold the budget.

Scope warning

Prediction and presentation only. Do not build automated messages to students, automated penalties or anything that feeds a decision without a person. Use anonymised historical data with institutional ethics approval, or a public dataset. Say plainly in the report that the tool supports human judgement.

Learn what you need on Sefism

• Machine learning roadmap : classification, validation and explainability.
• Machine learning cheat sheet : precision, recall and threshold choice.
• Full-stack roadmap : building the dashboard and API.
• EU Data Protection Rules for Business and Organisations : principles for handling personal data responsibly.
• OpenIntro Statistics : confidence intervals for the fairness audit.

Get help with this idea

This idea touches real students, so design and ethics matter as much as accuracy. Tauseef can help you scope the data, plan the fairness audit and frame an evaluation examiners will respect. If you are new here, create a free account first.

• Discuss this idea with a mentor
• Send a query to Tauseef
• Book a 1:1 session