RAG basics

What retrieval augmented generation is, the two halves of every pipeline and when it is the right tool.

Retrieval augmented generation

Also called RAG

What: A pattern where your code first searches your own documents for passages relevant to a question, then gives those passages to an LLM to answer from.
Why: It lets a model answer about private, recent or niche information it was never trained on, and lets you show sources.

Indexing and query pipelines

Also called offline and online

What: Indexing runs ahead of time: load, clean, chunk, embed and store documents. The query side runs per question: embed, retrieve, rerank, build the prompt, generate.
Why: Most quality problems live in indexing, even though people usually debug the prompt first.

Code

INDEX:  load -> clean -> chunk -> embed -> store
QUERY:  question -> rewrite -> embed -> retrieve
        -> rerank -> build prompt -> LLM -> answer
        + citations

RAG vs fine-tuning

What: RAG adds knowledge at question time; fine-tuning changes model weights to learn a style, format or narrow skill.
Why: For facts that change or must be cited, use RAG. For consistent tone or output format, consider fine-tuning or better prompts.

Watch out

Fine-tuning on company documents and expecting the model to recall them accurately with sources.

RAG vs long context

What: If all your content fits comfortably in the model context window, you can sometimes paste it in instead of retrieving.
Why: Long context is simpler for small, stable corpora; RAG wins on cost, latency and scale once content grows or changes often.

Start with a naive baseline

Also called naive RAG

What: Fixed-size chunks, one embedding model, top 5 by vector similarity and a simple grounded prompt.
Why: Get the baseline working with an eval set before adding rerankers, graphs or agents, so you can tell whether each change helps.

Knowledge base

Also called corpus

What: The collection of documents the system can retrieve from: docs, tickets, PDFs, wiki pages or database rows.
Why: Answer quality is capped by corpus quality: outdated or duplicated pages come back as confident wrong answers.

Go deeper