On-Device SMS Scam Detector for Mobile Wallet Users in Roman Urdu

The problem mobile wallet users face with scam messages

Millions of people now keep money in mobile wallets such as JazzCash and Easypaisa, or in bank apps, and many are first-time digital finance users. Scam messages target them directly: "you have won a prize, call this number", "I sent money to you by mistake, please return it", "your account will be blocked, share the code you just received". A 2019 University of Washington study of SMS fraud in Pakistan found that most fraud messages ask the victim to call a number rather than send money in the message itself, which is why a warning at the moment of reading matters.

The same patterns appear in Kenya, India, Bangladesh and anywhere mobile money is common. The people most at risk, older relatives and first-time users, are the least likely to install a security app that needs configuration.

What already exists and the gap

• Google Messages scam detection: on-device AI that flags conversational scams from non-contacts. It launched in English first in the US, UK and Canada; coverage of Roman Urdu and local wallet scams is not documented.
• Truecaller: caller and SMS spam identification driven largely by crowdsourced number reputation. Scammers rotate numbers, and content-level explanation is limited.
• Operator and regulator reporting short codes: useful after the fact, but they do not warn the user while reading.
• Research datasets and prototypes for Roman Urdu smishing: published work exists, with datasets of around a thousand messages, but not as a maintained app.

The gap is a private, content-aware warning that understands how people here actually write (Roman Urdu with English mixed in, inconsistent spelling) and tells the user exactly what to do next.

What you build

An Android app with one main user and a small admin tool for you.

1. After install, the app asks for SMS permission (or, where that is restricted, notification access) and explains in plain Urdu and English that nothing leaves the phone.
2. Each incoming message is classified locally. Legitimate transaction alerts are left alone; likely scams trigger a notification with the reason ("asks you to share a code") and a "what to do" screen.
3. Users can mark mistakes. Corrections stay on the phone and adjust local thresholds.
4. Optionally, users can report a scam; they see and edit the redacted text before it is sent.

The technically interesting part

There are three hard problems, and you should pick one as your centrepiece.

Low-resource, code-mixed text. Roman Urdu has no standard spelling ("paise", "pesay", "paisay"). Compare character n-gram models, a small multilingual transformer distilled for mobile, and your hybrid with hand-built features. Show which generalises to spellings not seen in training.

False positives on real financial messages. Legitimate wallet and bank alerts contain the same words as scams: amounts, codes, links. A detector that flags these is uninstalled within a day. Report false positive rate on real bank alerts separately and design features (known sender IDs, template matching) to drive it down.

On-device constraints. Model size, cold-start time and battery. Measure them on a low-end phone, not just an emulator.

Android restricts SMS permissions for apps on Google Play, so document how your design handles this (for example default SMS app role or notification access) as part of the engineering.

How to evaluate it

• Classification: precision, recall and F1 on a held-out set, with a separate false positive rate on at least 300 legitimate financial alerts.
• Generalisation: accuracy on scams collected after the model was trained.
• Performance: inference time and model size on a budget Android device.
• Human outcome: in a user study with 15 or more people, the share who choose the safe action after seeing a warning, compared with a control message without a warning.

Suggested plan

1. Weeks 1 to 5: consent process, data collection, labelling guide, dataset.
2. Weeks 6 to 10: baseline models, feature engineering, error analysis.
3. Weeks 11 to 15: on-device export, Android app, permission handling.
4. Weeks 16 to 20: guidance screens, report flow, local feedback.
5. Weeks 21 to 26: user study, performance tests, write-up.

Path to a product

The first customer could be a microfinance bank, a wallet provider or a telecom operator that wants to reduce scam losses and support calls, licensing the model and warning screens as an SDK inside their own app. A consumer app with a free tier builds the reputation and the dataset of reported scams. Expanding to other code-mixed languages follows the same pipeline.

Scope warning

Do not upload users' inboxes or build a cloud classification service. Do not try to block calls or integrate with operator networks. Do not claim protection against every scam; show the error rates honestly.

Learn what you need on Sefism

• Mobile roadmap : Android app structure, permissions and background work.
• Machine learning roadmap : text classification, evaluation and error analysis.
• Machine learning cheat sheet : metrics and model choices at a glance.
• OWASP Mobile Application Security : how to keep the app itself and its local data secure.
• Android Developers: Develop for Android : official docs for SMS, notifications and on-device ML.

Get help with this idea

Scoping a security project is mostly about deciding what you will not defend against and how you will prove the rest works. If you want a second opinion on scope, stack or your evaluation plan, a mentor can review it with you; you still build it yourselves. New to Sefism? Create a free account first.

• Discuss this idea with a mentor
• Send a query to Tauseef
• Book a 1:1 session