Road Damage Mapping from Phone Video for Municipal Maintenance Teams

The problem: nobody has a map of the potholes

Potholes damage vehicles, cause motorbike crashes and get worse after every monsoon. Municipal works departments in Pakistani towns usually learn about damage from complaints, inspections by eye or newspaper photos. There is rarely a current map showing where the damage is, how bad it is and whether it is getting worse, so repair budgets go to the loudest complaint rather than the worst road.

Many vehicles already drive every road every day: rickshaws, delivery bikes, municipal vehicles. A phone on the dashboard could survey the roads as a side effect, and repeated passes over the same road would show not just where damage is, but how quickly it is spreading and whether last month's repair has already failed.

What already exists and the gap

• vialytics offers municipalities a smartphone-based road survey with AI damage detection and a management system, used in hundreds of municipalities in Europe and the US.
• RoadBotics, now part of Michelin, used windscreen-mounted phones to record video and produce colour-coded pavement maps.
• Open datasets and challenges for road damage detection include images from India and other countries.

These services are designed for municipalities in wealthy countries with marked roads and dedicated survey vehicles. The gap is an open, low-cost system tuned for South Asian roads, mixed surfaces and two- and three-wheelers, that solves the counting problem: one pothole, seen fifty times, must become one point on the map.

Inspiration

"RDD2022: A multi-national image dataset for automatic road damage detection" by Deeksha Arya, Hiroya Maeda, Sanjay Kumar Ghosh, Durga Toshniwal and Yoshihide Sekimoto, published in the Geoscience Data Journal, contains over 47,000 road images from six countries, including India, labelled for longitudinal, transverse and alligator cracks and potholes. It was released for the Crowd sensing-based Road Damage Detection Challenge. It is your starting dataset and benchmark.

What you build

• Drivers mount a phone and press record; the app does the rest and uploads on Wi-Fi.
• Municipal engineers see a map of damage with photos, severity and how long each point has been reported, and mark repairs.
• Councillors and residents can be shown a summary per area.

Each map point keeps the best photo of the damage, the dates it was seen, how many trips saw it and the current status. When a point stops being seen after a repair is marked, the system can confirm the repair; when it keeps being seen, the repair did not work. That simple loop, reported, repaired, confirmed, is what turns a detector into a maintenance tool.

The technically interesting part

• Domain shift. Local roads have unmarked edges, dust, patched surfaces, speed breakers and water. Measure how the RDD2022 model does on your frames, then fine-tune with a few hundred labelled local frames.
• From detections to objects. Consecutive frames see the same pothole. Track it within a trip, then merge sightings across trips using GPS, which can be off by several metres. This is the core engineering problem.
• Severity. A bounding box size depends on distance from the camera. Estimate a rough real size using the camera height and road geometry, or rank by relative measures.
• Privacy. Video captures faces and number plates. Blur them before storage and keep only frames with damage.

How to evaluate it

• Detection precision and recall on held-out local frames.
• Map accuracy: walk or drive a few routes, record every pothole by hand with GPS, and compare with the system's map: found, missed and false points.
• Deduplication: how many map points per real pothole after several trips.
• Engineer feedback: whether the ranked list matches their own sense of priority, and which points they would remove or add.
• Run time: minutes of processing per hour of video on a laptop or a small cloud machine.

Suggested plan

• Weeks 1 to 3: read the RDD2022 paper, train a baseline, record your first routes.
• Weeks 4 to 7: label local frames, fine-tune, add blurring.
• Weeks 8 to 10: tracking, GPS clustering and severity.
• Weeks 11 to 13: the web map and the ground truth survey.
• Weeks 14 to 15: report and demo.

Path to a product

Municipal corporations, cantonment boards, housing societies and highway authorities all maintain roads. A low-cost survey service with a map dashboard, or a model offered to ride-hailing and delivery companies whose drivers already cover the city, is realistic. The project is a complete computer vision and geospatial system, a strong portfolio piece.

Scope warning

Do not try to cover a whole city; a few well-surveyed routes are enough. Do not ask anyone to record while riding unsafely; mount the phone securely and never interact with it while moving. Blur people and plates before anything is stored.

Learn what you need on Sefism

• Machine learning roadmap: fine-tuning and honest evaluation.
• Ultralytics YOLO Documentation: training the damage detector and tracking.
• OpenCV-Python Tutorials: frame sampling and blurring.
• Label Studio: labelling local frames.
• PostgreSQL Documentation: storing points and running spatial queries with PostGIS.

Get help with this idea

The risk is a detector demo with no reliable map behind it. Tauseef can help you plan the ground truth survey, the deduplication approach and the municipal partnership. If you are new here, create a free account first.

• Discuss this idea with a mentor
• Send a query to Tauseef
• Book a 1:1 session