Citation-Checked Legal Question Answering over Pakistan Code and India Code
Law students and junior lawyers in Pakistan and India increasingly ask AI tools legal questions, and those tools sometimes cite sections that do not exist or misquote them. Build a retrieval-augmented assistant over official statute text from Pakistan Code or India Code where every answer cites specific sections, and a separate checker verifies that each cited section exists and that quoted text matches the source before the answer is shown. Measure hallucinated citations before and after the checker.
Advanced
Needs a team, real planning, and a supervisor who knows the area.2 semesters · 2 to 3 students
7 to ship
4 optional extrasSuggested stack
What you should ship
- A corpus of 20 to 50 federal statutes from the official code website, parsed into sections with stable identifiers and version dates
- Hybrid retrieval (keyword plus vector) at section level, with amendments and definitions linked
- Answer generation that must cite section identifiers in a fixed format
- A citation checker that verifies every cited section exists, the quote matches, and the section is in force, and blocks or flags failures
- A test set of 150 or more questions written with law students, with expected sections
- An evaluation of retrieval recall, citation accuracy and answer correctness, judged by law students
- An interface that shows each answer beside the cited text
If you have time left
- Urdu or Hindi questions answered from English statutes
- Linking to leading judgments where a public, licensed source exists
- Comparing several models on the same test set
- Showing how a section changed after an amendment
The problem: confident answers with fake sections
Law is written in long statutes with frequent amendments. Students, junior lawyers, journalists and ordinary people now ask general AI chat tools legal questions, and the answers often sound right while citing the wrong section, an old version or a section that does not exist. In a legal setting, a wrong citation is worse than no answer, because it looks authoritative.
Pakistan Code and India Code, the official government websites for federal laws, publish statute text that can serve as a trustworthy base.
What already exists and the gap
- Pakistan Code and India Code publish federal statutes online.
- Indian Kanoon and commercial legal databases offer search over statutes and judgments.
- Legal AI products from large legal publishers add AI answers on top of their databases, priced for law firms.
The gap is an open, measurable assistant that treats citation correctness as a hard requirement, with a checker that is separate from the model and an honest evaluation.
Inspiration
A 2024 study by researchers at Stanford's RegLab and Human-Centered AI Institute, "Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools" by Varun Magesh and colleagues, found that commercial legal research tools built on retrieval still produced incorrect or misgrounded answers in a meaningful share of cases. Your project builds the verification layer they found missing and measures how much it helps.
What you build
- Law students ask questions and get answers with clickable sections and the exact text alongside.
- Teachers use the test set and results to discuss AI reliability in legal research.
- Developers get a pattern for verified citations that applies to any regulated domain.
The technically interesting part
- Parsing statutes. Sections, sub-sections, provisos, schedules and amendments need clean identifiers. This takes longer than the AI part.
- Hybrid retrieval. Legal queries mix exact terms ("section 489-F") with concepts ("bounced cheque"). Combine keyword and vector search and measure each.
- The checker. It must be deterministic: parse cited identifiers, look them up, compare quotes with fuzzy matching and check the version date. If anything fails, the answer is blocked or clearly flagged.
- Evaluation design. Write questions with law students, including trick questions with no answer in the corpus.
How to evaluate it
- Retrieval recall at k for the expected sections.
- Citation accuracy: share of cited sections that exist, are relevant and are quoted correctly, with and without the checker.
- Answer correctness judged by two law students independently, with agreement reported.
- Refusal quality: behaviour on questions the corpus cannot answer.
Suggested plan
- Weeks 1 to 5: collect and parse statutes, build identifiers.
- Weeks 6 to 10: retrieval and its evaluation.
- Weeks 11 to 16: answer generation and the citation checker.
- Weeks 17 to 22: test set with law students and evaluation.
- Weeks 23 to 28: interface, report and demo.
Path to a product
Law firms, legal aid organisations, law schools and compliance teams in Pakistan and India need reliable research tools at local prices. A verified-citation layer is a credible differentiator. The project is a strong AI engineering portfolio piece because it shows evaluation and safety, not just a chatbot.
Scope warning
The assistant is for research and education, not legal advice; say so clearly. Use only official, publicly available text and respect each site's terms. Do not include judgments unless you have a clear right to use them. Keep to federal statutes in one country first.
Learn what you need on Sefism
- AI engineering roadmap: retrieval-augmented generation and evaluation.
- Gen AI app architecture cheat sheet: where the checker sits in the pipeline.
- AI evals cheat sheet: building and scoring the test set.
- RAG Techniques: hybrid retrieval and re-ranking.
- pgvector: vector search inside PostgreSQL.
- DeepLearning.AI Retrieval Augmented Generation: the core ideas.
- Do I need a vector database for my AI project?: choosing where to store and search the legal text.
Get help with this idea
Teams spend the year on prompts and never build a real test set. Tauseef can help you plan the parsing, the checker and an evaluation law students can judge. If you are new here, create a free account first.
Ideas and guidance, not finished projects
These are project ideas and scoping guidance, published free for students to use as a starting point. I do not build, write, or sell final-year projects, and I do not complete coursework for anyone. Take an idea, make it yours, and build it.
Not sure this one fits you?
I help students pick an idea that matches their skills and their deadline, then scope it down until it is finishable. Guidance only, never done for you.
Stuck on something specific?
Writing only gets you so far. If you want an answer to your situation rather than the general case, book a session and we will work through it together. Sessions are free for approved Sefism members, and a few slots open each week.
Follow along
New writing, resources and project ideas land here first.