Offline-First Field Data Collection App with Conflict-Safe Sync

The problem field teams face without signal

Field work happens where the network is not. Health surveys, polio and vaccination campaigns, agricultural inspections, infrastructure audits and census work all involve people entering data for hours or days with no connectivity. Apps that assume a live connection lose that work or duplicate it, and a supervisor in the office cannot trust totals when the same household appears twice or an edit silently overwrote another.

What already exists and the gap

• ODK Collect is the open source standard for offline forms, with XLSForm definitions and sync to a server such as ODK Central.
• KoboToolbox builds on ODK tools and is free and hosted for humanitarian and research teams.
• SurveyCTO is a paid platform on the same standards, with case management and stronger support for research teams.

These are mature and you should not try to replace them. They are mostly built around one-time submissions; editing the same record on two devices is handled in limited ways, such as last-write-wins flags. The gap for a final year project is not a new market product but a deep understanding of the part these tools solve: offline-first sync with idempotent uploads and explicit conflict handling for records that change over repeat visits. That makes a strong portfolio piece and a base for a niche product.

What you build

Three roles:

• Form admin: publishes form schemas as JSON and assigns them to teams.
• Enumerator: downloads forms and assigned sites, collects data offline for days, edits records on repeat visits, syncs when possible.
• Supervisor: sees incoming submissions, resolves conflicts the policy cannot, and exports data.

Main flows: download forms, fill them offline with photos and GPS, queue changes in a local outbox, sync in the background, and handle a conflict when two enumerators edited the same site.

The technically interesting part

Offline-first inverts the normal design. The device is authoritative for its own work and the server is a replica. Every operation is written to a durable outbox and must be safe to repeat, because an upload can fail after the server processed it. Client-generated IDs and idempotent endpoints, the same pattern Stripe uses for retried API calls, prevent duplicates.

Conflict policy. Last write wins is simple and loses data. Field-level merge with a version per record is better: if two edits touch different fields, merge them; if they touch the same field, flag it for a human. State your policy, justify it, and show the supervisor screen that handles what it cannot resolve.

How to evaluate it

• Test plan: scripted scenarios, each run at least 20 times: airplane mode mid-submission, app killed with a queue pending, sync interrupted halfway, same record edited on two devices, and a schema change while forms are in progress. Report pass rates.
• Duplicates: inject random network failures during 500 uploads; the server must end with exactly one copy of each record.
• Field trial: two to four users collect data in a low-signal area for a day; report records lost (target zero), sync time and user feedback.
• Performance: sync time for a day's work, including photos, on a slow connection.

Suggested plan

1. Weeks 1 to 2: interview an NGO, health or agriculture field team; collect a real form.
2. Weeks 3 to 5: schema format, form renderer, local storage.
3. Weeks 6 to 8: outbox, idempotent API, basic sync.
4. Weeks 9 to 11: versions, conflict detection and resolution, form versioning.
5. Weeks 12 to 15: test plan, field trial, write-up.

Path to a product

The first customer is a local NGO or research group whose forms involve repeat visits, such as household follow-ups or farm inspections, which is where one-shot submission tools are weakest. Offer a hosted version priced per active enumerator per month, or a setup and support contract. Open-sourcing the sync engine is an equally valid path and a strong signal for backend and mobile roles.

Scope warning

Do not build a drag-and-drop form designer: a schema file edited as JSON is enough. Do not build dashboards and analytics beyond a submissions list and CSV export. Do not support every question type in XLSForm. The sync engine is the project.

Learn what you need on Sefism

• Mobile roadmap : Flutter, local databases and background work.
• Backend roadmap : API design, transactions and idempotency.
• System design cheat sheet : replication, consistency and conflict resolution concepts.
• roadmap.sh: API Design : versioning and idempotent endpoints.
• Guide to App Architecture (Android) : the offline-first data layer pattern, which carries over to Flutter.

Get help with this idea

The sync engine and its test plan are the project, so bring your conflict policy to the first conversation. Tauseef can help you tighten the scope, choose a stack your team can finish with, and design an evaluation your examiners will respect. If you are new here, create a free account first.

• Discuss this idea with a mentor
• Send a query to Tauseef
• Book a 1:1 session