Offline Urdu Voice Commands on a Microcontroller for Home Lights and Fans

The problem: voice control that needs the cloud and English

For an older person with arthritis, someone recovering from a stroke or a wheelchair user, getting up to switch a fan or light is a real barrier. Voice control should help, but mainstream assistants need reliable internet, send audio to the cloud, cost money for smart switches and speakers, and handle Urdu and Punjabi commands poorly. In many South Asian homes, power cuts also take the Wi-Fi down.

Keyword spotting, recognising a handful of fixed words, is a much smaller problem than full speech recognition, and it can run on an inexpensive microcontroller with no operating system. What is missing is a model trained for local languages and accents, and a design that works offline and safely controls mains appliances.

What already exists and the gap

• Cloud voice assistants from Amazon, Google and Apple need internet, and their Urdu support is limited or absent for home control.
• Edge Impulse and TensorFlow Lite for Microcontrollers make on-device keyword spotting possible, with English examples.
• The Speech Commands dataset from Google (Warden, 2018) contains English words only.
• Smart switches in local markets are Wi-Fi based and depend on a phone app or cloud assistant.

The gap is an offline, low-cost controller that understands a small set of Urdu and Punjabi commands from many speakers, and a public dataset that others can build on.

Inspiration

Zhang, Suda, Lai and Chandra's paper "Hello Edge: Keyword Spotting on Microcontrollers" (arXiv, 2017) showed that neural networks for keyword spotting, particularly depthwise separable CNNs, can fit within the memory and compute limits of microcontrollers while keeping accuracy high. Warden's Speech Commands paper showed how to build a keyword dataset carefully. Your project repeats both steps for local languages.

What you build

• A dataset of command words such as "pankha chalao", "batti band" and "sab band" recorded by many speakers of different ages and accents, with consent, plus noise from fans, televisions and street sounds.
• A model trained on spectrogram features, quantised to int8 and deployed on the microcontroller.
• The device listens continuously, detects a command, confirms with a beep or light and switches a relay.
• A user speaks naturally from across the room, without a phone, internet or account.

The technically interesting part

• Dataset design. The model is only as good as the speakers in your data. Balance age, gender and accent, include older speakers, and keep separate speakers in the test set so you measure performance on voices the model has never heard.
• False accepts. In a real home, the device hears hours of television and conversation. A device that switches the fan off during a drama is worse than useless. Measure false accepts per hour on long background recordings and tune thresholds and smoothing.
• Fitting the model. Compare model sizes and architectures for accuracy against memory and latency, and measure the effect of int8 quantisation.
• Streaming inference. Run the model on overlapping windows and smooth the outputs so one spoken command triggers exactly one action.
• Safe switching. Mains switching must use an enclosed, isolated relay or triac module, with proper wiring checked by a qualified electrician.

How to evaluate it

• Accuracy: per-command accuracy on held-out speakers, broken down by age group and gender.
• False accepts: false triggers per hour on several hours of household background audio.
• Latency: time from the end of the spoken command to the relay switching.
• Resources: flash, RAM and power use on the device.
• Users: a week-long home trial with two or three older users, with their feedback.

Suggested plan

• Weeks 1 to 4: choose commands with users, design the recording protocol, start recording.
• Weeks 5 to 9: feature extraction, first models on a PC, speaker-independent evaluation.
• Weeks 10 to 14: quantisation, deployment on the microcontroller, streaming inference.
• Second semester: relay hardware and enclosure, false accept tuning, home trials, dataset release and report.

Path to a product

Affordable offline voice control in local languages has clear demand among older people, people with disabilities and their families, and could be sold through electrical shops and disability organisations. The dataset itself is a valuable open contribution, and on-device machine learning is a growing specialisation in embedded engineering.

Scope warning

Keep to a small fixed vocabulary; do not attempt open speech recognition. Get informed consent from every speaker and do not collect names. Do not wire mains circuits yourself unless you are qualified; use certified modules and have the installation checked.

Learn what you need on Sefism

• Embedded systems and IoT roadmap: microcontrollers, peripherals and on-device processing.
• Machine learning roadmap: training, evaluation and avoiding leakage between speakers.
• PyTorch: Learn the Basics: neural network fundamentals, whichever framework you deploy with.
• ESP-IDF Programming Guide: I2S audio capture on the ESP32-S3.
• Is it worth doing a final year project in AI if I'm not good at math?: how much maths this really needs.

Get help with this idea

The dataset and false accepts decide whether this works in a real home. Tauseef can help you design the recording protocol, plan speaker-independent tests and keep the project within two semesters. If you are new here, create a free account first.

• Discuss this idea with a mentor
• Send a query to Tauseef
• Book a 1:1 session