Hidden Instruction Scanner for Documents Before They Enter a RAG Knowledge Base
Build an ingestion-time scanner that inspects PDFs, Word files, web pages and spreadsheets before they are chunked into a RAG knowledge base, and finds text a human cannot see but a model will read: white or tiny text, invisible Unicode, instructions hidden in metadata, alt text and comments. Suspicious files are quarantined with a highlighted preview so a person decides. It closes the door that indirect prompt injection walks through.
Intermediate
A small team, or one strong student willing to learn something new.1 semester · 2 students
7 to ship
3 optional extrasSuggested stack
What you should ship
- Parsers for PDF, DOCX, HTML and XLSX that extract every text layer with its visual properties: colour against background, font size, position on or off the page, and layer
- Detectors for hidden text, invisible and tag Unicode characters, instructions in metadata, comments, alt text and hidden spreadsheet sheets
- A small classifier that scores whether visible or hidden text reads like an instruction to an AI system, in English, Urdu and Roman Urdu
- A quarantine queue with a side-by-side preview that highlights exactly what a human sees and what the model would see
- A plug-in step for an existing ingestion pipeline (a LlamaIndex or LangChain loader, or a plain HTTP call) that blocks, strips or passes each file
- A labelled test corpus of at least 300 documents, clean and attacked, built only from files you create
- An evaluation report: detection rate per hiding technique, false positives on ordinary university and business documents, and scan time per file
If you have time left
- A 'sanitised copy' mode that rebuilds the document from visible text only
- OCR comparison: render each page to an image, OCR it, and flag text present in the file but absent from the rendered page
- A browser extension that runs the same checks on a web page before it is pasted into a chatbot
The problem: your knowledge base reads what you cannot see
A retrieval-augmented generation (RAG) app answers questions from documents somebody uploaded: policy PDFs, supplier contracts, CVs, web pages. The model reads the extracted text, not the rendered page. That gap is exactly where indirect prompt injection lives. White text on a white background, a one-point font, a paragraph placed off the page, a comment in a Word file or a run of invisible Unicode characters are all invisible to a reviewer and perfectly readable to the model.
This is not hypothetical. Greshake and colleagues showed in 2023 ("Not what you've signed up for") that instructions planted in retrieved content can steer LLM-integrated apps. In July 2025 Nikkei Asia reported research preprints on arXiv carrying hidden prompts that asked AI reviewers to give only positive reviews. The OWASP Top 10 for LLM Applications (2025) lists prompt injection first, and names vector and embedding weaknesses as a separate risk.
Most teams test their chatbot's answers. Very few inspect the documents on the way in.
What already exists and the gap
- Prompt-attack classifiers such as Meta's Prompt Guard models, Microsoft's Prompt Shields in Azure AI Content Safety and Lakera Guard score a piece of text for injection, but they see the text after extraction and do not know it was invisible on the page.
- Document loaders in LlamaIndex and LangChain extract text faithfully, which includes the hidden parts.
- Red-team and regression tools such as garak and promptfoo test a running chatbot, not its ingestion pipeline.
The gap is a scanner that understands document rendering: it compares what a person sees with what the model receives and explains the difference.
Inspiration
Johann Rehberger's public write-ups on "ASCII smuggling" with Unicode tag characters showed that text can be invisible in a chat window yet fully read by a model. Combine that with the 2025 hidden-prompt preprints and you have a clear, testable research question: how many hiding techniques can you detect from the file alone, and at what false-positive cost on normal documents?
What you build
- A scanner service that receives a file, extracts every text run with its visual properties and flags anything a human would not see.
- An instruction scorer that looks at flagged text (and visible text) and asks: does this read like an instruction aimed at an AI?
- A review queue where an admin sees the rendered page with hidden text highlighted, then approves, strips or rejects.
- An ingestion hook so an existing RAG app can call the scanner before chunking.
The technically interesting part
- Rendering versus extraction. Compute contrast between text colour and the background beneath it, detect text clipped outside the visible page box, and catch zero-size fonts. PDFs are messy: test on files from Word, Google Docs, LaTeX and scanners.
- Unicode tricks. Tag characters, zero-width joiners and bidirectional controls. Urdu text legitimately uses some joiners, so a naive rule will flag every Urdu document. Learning where the line is makes a good chapter in the report.
- Low false positives. A scanner that quarantines a third of normal documents gets switched off. Measure on a clean corpus of syllabi, notices and business letters.
How you will evaluate it
- Detection rate per technique (colour, size, position, metadata, comments, Unicode) on your attacked corpus.
- False-positive rate on at least 150 ordinary documents in English and Urdu.
- An end-to-end test: a small RAG chatbot with and without the scanner, and how many planted instructions reach the model's context.
Path to a product
Any company putting customer or supplier documents into an AI assistant needs this check, and it slots in as one API call. A hosted scanner priced per document, an open source core with a paid review dashboard, or a plug-in for popular RAG frameworks are all realistic paths.
Scope warning
Create every attacked document yourself and never upload them to systems you do not own. Do not try to build a general prompt-injection classifier from scratch; reuse an open model for the scoring step and spend your time on the document side, where the gap is.
Learn what you need on Sefism
- AI security roadmap: how LLM apps fail, prompt injection and the defences that actually hold.
- AI security cheat sheet: the quick reference for injection, output handling and supply chain risks.
- GenAI app architecture cheat sheet: where ingestion, chunking and retrieval sit in a RAG app.
- OWASP Top 10 for LLM Applications: the risk list your report maps to.
- Retrieval Augmented Generation course: how ingestion pipelines are built.
- garak: an example of automated probing you can borrow ideas from.
- How to get into AI security: where a project like this leads.
Get help with this idea
The hard part is keeping false positives low while still catching clever hiding. Tauseef can help you plan the corpus, pick the techniques worth covering and design a fair evaluation. If you are new here, create a free account first.
Ideas and guidance, not finished projects
These are project ideas and scoping guidance, published free for students to use as a starting point. I do not build, write, or sell final-year projects, and I do not complete coursework for anyone. Take an idea, make it yours, and build it.
Not sure this one fits you?
I help students pick an idea that matches their skills and their deadline, then scope it down until it is finishable. Guidance only, never done for you.
Stuck on something specific?
Writing only gets you so far. If you want an answer to your situation rather than the general case, book a session and we will work through it together. Sessions are free for approved Sefism members, and a few slots open each week.
Follow along
New writing, resources and project ideas land here first.