Traffic Flow Forecasting from Open Sensor Data

The problem: congestion that nobody forecasts at the network level

Congestion prediction feeds routing, signal timing, event planning and public transport scheduling. City traffic units in fast-growing cities like Lahore or Karachi often react to jams rather than anticipate them. Traffic on one road is caused by traffic on the roads feeding it a few minutes earlier, so a forecast that treats every sensor as an independent series misses the most useful signal.

For a student, the usual obstacle in transport projects is data. Here it is removed: several agencies publish loop-detector data openly, and research benchmarks built on it are well documented.

What already exists and the gap

• Google Maps, Waze and TomTom Traffic provide live and predicted traffic from phone and fleet data, but the models and data are closed.
• INRIX sells traffic analytics to agencies and businesses.
• California PeMS publishes the raw detector data behind the METR-LA (207 sensors, Los Angeles) and PEMS-BAY (325 sensors, Bay Area) research benchmarks.
• Research code such as DCRNN and later graph models is public, and surveys benchmark many of them.

The gap for a student is not a new architecture. It is a clean, honest pipeline that shows when graph structure actually beats a strong simple baseline, how missing data changes the answer, and what a planner would see on a map. Many published comparisons skip the simple baselines or the missing-data question.

What you build

• Pipeline: download, clean and store detector data; build the adjacency matrix from road distances.
• Baselines: historical average, persistence and a per-sensor model.
• Graph model: a spatio-temporal model that uses the adjacency, trained and tuned on your own splits.
• API and map: a FastAPI service serving predictions and a Leaflet map for planners, with predicted versus actual per sensor.

The technically interesting part

Two things.

The baseline you must beat. Historical average by time of day and day of week is surprisingly effective at this task. A project reporting an impressive error without comparing to it has not shown anything. Build it in week two, and report where the graph model wins (during congestion onset and incidents, typically) and where it does not.

Missing data. Real sensors fail. METR-LA has a noticeable share of missing readings; PEMS-BAY far fewer. How you impute gaps materially changes the results, and whether you evaluate on imputed values can inflate your scores. Document the strategy, exclude imputed targets from scoring, and show the effect.

How to evaluate it

• Horizon table: MAE, RMSE and MAPE at 15, 30 and 60 minutes for every model.
• Regime breakdown: errors in free-flow versus congested periods, where the differences actually matter.
• Temporal split: train on earlier weeks, test on later weeks. Never shuffle time.
• Ablation: graph model with and without adjacency, to prove the graph is doing the work.
• Planner check: show the map to a transport or civil engineering lecturer and record whether it answers a question they would ask.

Suggested plan

1. Weeks 1 to 2: data download, exploration, missing-data analysis.
2. Weeks 3 to 4: baselines and the evaluation script.
3. Weeks 5 to 9: graph model training, tuning and ablation.
4. Weeks 10 to 12: API, map dashboard, imputation study.
5. Weeks 13 to 14: report, model card and demo.

Path to a product

The realistic first customer is not a city. It is a logistics or ride-hailing startup that wants delivery-time estimates by corridor, or a research group at a university transport department. The honest path is consulting-style: adapt the pipeline to a client's own GPS or sensor data. Open-sourcing the clean pipeline also builds a portfolio for data science roles in mobility companies.

Scope warning

• One dataset, one prediction target (speed), one network.
• Do not build a routing or navigation app on top; forecasting and its evaluation is the project.
• Do not invent a new architecture. Use a published one and evaluate it well.
• Do not scrape commercial map services; stick to open data.

Learn what you need on Sefism

• Machine learning roadmap : time-series validation and the deep learning basics you need.
• Data analytics cheat sheet : pandas and plotting for the exploration phase.
• Google Colab : free GPUs to train the graph model.
• Kaggle Datasets : mirrors of traffic benchmarks and related weather data.
• Lyft Engineering : how a mobility company writes about forecasting and marketplace data.

Get help with this idea

The grade here comes from the baselines, the missing-data study and the honesty of the comparison. Tauseef can help you pick the dataset, set up the splits and plan an evaluation that shows exactly what the graph model adds. If you are new here, create a free account first.

• Discuss this idea with a mentor
• Send a query to Tauseef
• Book a 1:1 session