Home Sound Alerts on a Vibrating Wristband for Deaf and Hard of Hearing People
Build a small listening device that recognises important household sounds, such as the doorbell, a pressure cooker whistle, a baby crying or a smoke alarm, entirely on the device, and sends a distinct vibration pattern to a wristband. Deaf and hard of hearing people often miss these sounds, and phone-based features are limited to a fixed list of mostly Western sounds. The project lets users teach the device sounds from their own home.
Intermediate
A small team, or one strong student willing to learn something new.1 to 2 semesters · 2 students
7 to ship
4 optional extrasSuggested stack
What you should ship
- A listening node that classifies at least six household sound classes on the device, with nothing recorded or uploaded
- Sound classes that matter locally, including the pressure cooker whistle and the doorbell, with a recorded dataset from several homes
- A way for users to add a new sound from a few examples recorded at home
- A wristband that receives BLE alerts and vibrates in a distinct pattern per sound
- Confidence thresholds and cooldowns so alerts are not repeated or triggered by television
- Accuracy, false alarms per hour, latency and battery life measurements
- A home trial with deaf or hard of hearing participants, designed with them
If you have time left
- Several listening nodes in different rooms with the room shown on the wristband
- A small display that shows the sound name in Urdu and English
- Name calling detection so family members can get attention
- A visual flashing light alert in addition to vibration
The problem: important sounds that never reach the person who needs them
For deaf and hard of hearing people, home is full of sounds they miss: the doorbell, someone knocking, a baby crying in the next room, the pressure cooker whistle that means the cooker needs turning down, a smoke alarm. Families work around it, but it costs independence and sometimes safety.
Phones now include sound recognition features, but they recognise a fixed list of sounds, many of which do not match South Asian homes, and the phone must be nearby and listening. Dedicated alerting systems use wired transmitters on each doorbell and alarm, which are costly and do not cover sounds such as a cooker whistle or a baby crying.
What already exists and the gap
- Sound recognition on iOS and Android alerts users to a set of built-in sounds, with limited support for custom sounds, and runs on a phone that must be close by.
- Visual alerting systems from hearing aid accessory makers connect a doorbell, phone or smoke alarm to flashing lights or vibrating pagers, using a transmitter on each source.
- Research prototypes such as SoundWatch from the University of Washington explored smartwatch-based sound recognition.
The gap is a low-cost, private, on-device system that covers the sounds of local homes and lets users teach it new ones, with alerts on an inexpensive wristband.
Inspiration
Jain and colleagues' paper "HomeSound: An Iterative Field Deployment of an In-Home Sound Awareness System for Deaf or Hard of Hearing Users" (CHI 2020) deployed sound sensing devices in deaf and hard of hearing participants' homes. Participants valued the awareness, but misclassifications and constant watch vibrations were not well received. That finding shapes this project: fewer, more reliable alerts, chosen by the user.
What you build
- The listening node sits in the kitchen or hallway and classifies audio continuously on the microcontroller; audio never leaves the device.
- The wristband receives an alert over Bluetooth Low Energy and vibrates in a pattern the user learns, such as two short pulses for the doorbell.
- The user picks which sounds to be alerted about and records a few examples of a new sound, such as their own doorbell.
- The family helps set it up through a simple companion app or buttons on the device.
The technically interesting part
- Few-shot new sounds. Training a full model for each household is impractical. Use a pretrained audio embedding model such as YAMNet on a PC or phone to build compact representations, then add new sounds from a few examples using a small classifier on top, and deploy that to the microcontroller.
- Local sounds. The pressure cooker whistle and South Asian doorbells are not well covered by common datasets. Record them in several homes with permission, and measure how well the model generalises to homes it has not seen.
- False alarms. Televisions play doorbells and babies crying. Use confidence thresholds, minimum durations and cooldowns, and measure false alarms per hour during normal household life.
- Low power on the wrist. The wristband should last days. Use BLE sleep modes and measure current carefully.
- Designing with users. Work with deaf and hard of hearing people from the start on which sounds matter and which vibration patterns are distinguishable. Organisations for deaf people can help you find participants.
How to evaluate it
- Classification: precision and recall per sound class, on homes not used for training.
- Few-shot: accuracy for a new sound with 3, 5 and 10 examples.
- False alarms: false alerts per hour during several hours of everyday household audio.
- Latency and battery: time from sound to vibration, and wristband battery life.
- Users: a week-long trial with feedback on usefulness and alert fatigue.
Suggested plan
- Weeks 1 to 3: interviews with deaf and hard of hearing users, choose sound classes, start recording.
- Weeks 4 to 7: models on a PC with YAMNet embeddings, few-shot method, evaluation.
- Weeks 8 to 10: on-device model, streaming classification, thresholds.
- Weeks 11 to 13: wristband, BLE, vibration patterns.
- Weeks 14 to 16: home trial, analysis, report. A second semester could add multiple rooms and the display.
Path to a product
Deaf and hard of hearing people in Pakistan and South Asia have few affordable assistive devices designed for local homes. A low-cost kit distributed through disability organisations, special education schools and hearing clinics fills a real need. The project shows on-device machine learning, low-power design and user-centred research, which assistive technology and IoT employers value.
Scope warning
Do not present the system as a replacement for a certified smoke or fire alarm; it is an awareness aid. Do not record or upload household audio beyond consented training sessions. Keep to six or so sound classes that users actually ask for.
Learn what you need on Sefism
- Embedded systems and IoT roadmap: audio capture, BLE and low-power design.
- Embedded systems and IoT cheat sheet: peripherals, sleep modes and wireless basics.
- Machine learning roadmap: transfer learning and honest evaluation.
- ESP-IDF Programming Guide: I2S microphones and BLE on the ESP32.
- Flutter Documentation: the optional companion app.
Get help with this idea
The hard parts are false alarms and designing with users, not just for them. Tauseef can help you plan the user research, the few-shot evaluation and a home trial that produces useful evidence. If you are new here, create a free account first.
Ideas and guidance, not finished projects
These are project ideas and scoping guidance, published free for students to use as a starting point. I do not build, write, or sell final-year projects, and I do not complete coursework for anyone. Take an idea, make it yours, and build it.
Not sure this one fits you?
I help students pick an idea that matches their skills and their deadline, then scope it down until it is finishable. Guidance only, never done for you.
Stuck on something specific?
Writing only gets you so far. If you want an answer to your situation rather than the general case, book a session and we will work through it together. Sessions are free for approved Sefism members, and a few slots open each week.
Follow along
New writing, resources and project ideas land here first.