On-Device Speech Recognition for an Under-Served Language
Adapt an open speech recognition model to a language or regional dialect it handles poorly, then get it running on a mid-range phone without a network connection. Useful where connectivity is unreliable and the people who need transcription most are the ones current models serve worst.
Advanced
Needs a team, real planning, and a supervisor who knows the area.1 semester, 2 to 3 students
5 to ship
3 optional extrasSuggested stack
What you should ship
- A labelled speech dataset of at least 10 hours, collected or assembled from open sources, with a documented collection method
- A fine tuned model with word error rate reported against the base model on a held out test set
- Quantised model running offline on an Android device, with measured latency and memory use
- A simple recording and transcription app demonstrating it
- Written analysis of where the model still fails, by speaker, accent and recording condition
If you have time left
- Speaker diarisation for two speaker recordings
- Punctuation and casing restoration as a second stage
- A correction interface whose edits feed back into the training set
The problem
Speech recognition works well for a small number of well resourced languages and noticeably worse for everything else, including most regional dialects. The gap is a data problem rather than an algorithmic one, which makes it tractable for a student project.
What you build
A fine tuning pipeline on top of an existing open model, a properly constructed evaluation set, and a quantised model that runs on a phone with no network connection.
Why on device matters
It is the part that makes this engineering rather than a notebook. Getting a model small and fast enough to run on a mid-range device involves real trade offs between accuracy, size and latency, and measuring those trade offs is a genuine contribution.
The hard part
The data. Ten hours of clean labelled speech is more work than anyone expects, and rushing it produces a model that scores well on your test set and fails on real speech. Plan the collection in the first three weeks, and make the test set representative of speakers who are not your project group.
How to evaluate it
Word error rate against the base model, on a test set with speakers who never appear in training. Break the result down by speaker and by recording condition, because an average figure hides the cases that matter.
Scope warning
Do not train from scratch. Fine tuning an existing open model is the correct approach and the only one that fits a semester. Be careful with data licensing and get consent from anyone you record.
Ideas and guidance, not finished projects
These are project ideas and scoping guidance, published free for students to use as a starting point. I do not build, write, or sell final-year projects, and I do not complete coursework for anyone. Take an idea, make it yours, and build it.
Not sure this one fits you?
I help students pick an idea that matches their skills and their deadline, then scope it down until it is finishable. Guidance only, never done for you.
Stuck on something specific?
Writing only gets you so far. If you want an answer to your situation rather than the general case, book a session and we will work through it together. Every session is free; a few slots open each week.
Follow along
New writing, resources and project ideas land here first.