Clustered Short-Answer Grading for Large University Classes with Lecturer Control

The problem: too many answers to read

A lecturer teaching a first-year course in a Pakistani university may have three hundred students across sections. A short written question, "explain why this loop never ends", tests understanding far better than multiple choice. Marking three hundred answers by hand takes a whole weekend, so the written question gets dropped, or marked in a rush and inconsistently.

Most answers to a short question fall into a handful of patterns: the right idea in different words, a common misconception, a partly right answer. If the lecturer could mark each pattern once, the weekend becomes an hour.

What already exists and the gap

• Gradescope offers AI-assisted answer groups, so an instructor can grade a group at once. Its own guide says answer groups are available on fixed-template PDF assignments, not online assignments, and require an institutional licence.
• Learning management systems such as Moodle collect typed answers but leave grading one by one.
• Fully automatic graders using language models assign marks directly, which many lecturers do not trust for assessed work.

The gap is a free, lightweight tool for typed answers from the quizzes lecturers already run, which groups answers but leaves every mark to the lecturer, and works with the mix of English and Roman Urdu students actually write.

Inspiration

"Powergrading: a Clustering Approach to Amplify Human Effort for Short Answer Grading" by Sumit Basu, Chuck Jacobs and Lucy Vanderwende of Microsoft Research, published in Transactions of the Association for Computational Linguistics in 2013, learned a similarity measure between student answers, grouped them into clusters and subclusters, and showed how far a small budget of human grading actions could go. They released their answer corpus. Your project updates the idea with modern sentence embeddings and builds the tool around it.

What you build

• Lecturers import a quiz, see answers grouped, grade a group with one click and write one comment for all of them, then check the outliers.
• Teaching assistants share the grading, each taking some groups.
• Students get specific feedback instead of a bare mark.

The grading screen shows each group with its most typical answer at the top and the rest below, sorted from most to least similar, so the lecturer's eye naturally checks the edge cases. A grade applied to a group is applied to every answer in it, but each answer still shows its own mark and can be changed. A log records every grade and who gave it, which helps when a student asks for a re-check.

The technically interesting part

• Choosing the embedding. Compare a few sentence embedding models on the Powergrading corpus and on your own answers. Short technical answers with code words behave differently from general text.
• How many groups. Too few and different ideas get merged; too many and there is no time saved. Let the lecturer split and merge, and measure what they actually do.
• Mixed language. Students write "loop khatam nahi hota kyunke i increment nahi ho raha". A multilingual model should place this near the English equivalent; check that it does.
• Keeping the lecturer in charge. Every answer must be visible, and moving an answer between groups must be easy, or the tool will mark someone unfairly.

How to evaluate it

• Grading actions: number of clicks to grade all answers with grouping versus one by one.
• Agreement: grade a set of answers one by one, then with the tool a week later, and compare marks.
• Cluster purity on the Powergrading corpus, where human grades are available.
• Time: minutes per hundred answers in a real course, with lecturer permission.

Suggested plan

• Weeks 1 to 3: read the paper, get the corpus, compare embedding models.
• Weeks 4 to 7: clustering, outliers and the import from quiz exports.
• Weeks 8 to 10: the grading screen, feedback and export.
• Weeks 11 to 13: a trial with one lecturer on a real quiz, with student data anonymised.
• Weeks 14 to 15: report and demo.

Path to a product

Universities and schools everywhere want written assessment without the marking load. A free tool for individual lecturers, with a paid version for departments that integrates with their course system, is realistic. The project shows applied NLP with a human in the loop, which employers value more than a fully automatic demo.

Scope warning

Do not let the tool assign marks on its own. Do not try to grade essays or code; keep to answers of a few sentences. Anonymise student names before any answers leave the university's systems, and get the lecturer's and department's permission for the trial.

Learn what you need on Sefism

• Machine learning roadmap: clustering, embeddings and evaluation.
• Full stack development roadmap: building the grading screen and import.
• scikit-learn User Guide: clustering algorithms and outlier detection.
• Hugging Face Transformers Documentation: running sentence embedding models.
• Speech and Language Processing, 3rd Edition: background on text similarity.

Get help with this idea

The risk is a tool that saves no time because the groups are wrong. Tauseef can help you choose the evaluation, find a lecturer for the trial and keep the scope tight. If you are new here, create a free account first.

• Discuss this idea with a mentor
• Send a query to Tauseef
• Book a 1:1 session