Loading and chunking

Turning messy documents into clean, well-sized chunks with the metadata you will need later for filters and citations.

Document parsing

Also called loaders, extraction

What: Extracting clean text from PDFs, HTML, slides and docs while keeping structure such as headings, tables and lists.
Why: Garbage in, garbage out: broken tables, headers repeated on every page and cookie banners become retrieved chunks.

Watch out

Never looking at the extracted text; always print a few parsed documents before embedding millions.

Chunking

Also called text splitting

What: Splitting documents into passages small enough to embed and retrieve precisely, but large enough to make sense alone.
Why: Chunks are what the retriever returns, so chunk quality decides what the model ever gets to see.

Fixed-size chunks with overlap

Also called chunk overlap, sliding window

What: Cutting text every N tokens or characters, with each chunk repeating the last part of the previous one.
Why: Simple and predictable; overlap reduces the chance that an answer is split across a boundary.

Python

def chunk(text, size=800, overlap=100):
    step = size - overlap
    return [text[i:i + size]
            for i in range(0, len(text), step)]

Structure-aware chunking

Also called recursive splitting, markdown splitting

What: Splitting on natural boundaries first, such as headings, then paragraphs, then sentences, and only cutting mid-sentence as a last resort.
Why: Chunks that follow the document outline read as complete thoughts and retrieve better than arbitrary slices.

Semantic chunking

What: Placing chunk boundaries where the meaning shifts, detected by comparing embeddings of neighbouring sentences.
Why: It can help with long unstructured text, but costs more to index; test it against structural chunking before adopting it.

Choosing chunk size

What: Small chunks match questions precisely but lose surrounding context; large chunks keep context but dilute the embedding and waste prompt space.
Why: There is no universal best size: try a few sizes against your eval set and keep the one with the best recall.

Chunk metadata

What: Fields stored with each chunk such as source URL, title, section heading, date, language, product version and access group.
Why: Metadata powers filters, citations, freshness rules and permission checks. Adding it later means re-indexing.

JSON

{
  "id": "handbook-leave-policy#3",
  "text": "Annual leave requests need...",
  "source": "https://intranet/handbook/leave",
  "heading": "Annual leave",
  "updated": "2026-08-14",
  "access": ["all-staff"]
}

Small-to-big retrieval

Also called parent document retrieval, sentence window

What: Embed small chunks for precise matching, but hand the model the larger parent section they came from.
Why: It combines accurate search with enough context for the model to answer properly.

Go deeper