Contextual retrieval

Chunks often lose meaning once separated from their document. A passage that says the window is 14 days for monthly plans doesn't say which product or policy it describes, so a query mentioning the product may never find it. Contextual retrieval fixes this by having a model read the full document and write a sentence or two explaining where each chunk fits, then prepending that note to the chunk before indexing it.

Anthropic published the technique in 2024 and reported fewer retrieval failures when the added context was used for both embeddings and keyword search, with further gains when reranking was added. Their results came from their own test sets, so measure on yours, but the idea is sound and widely applicable.

The cost is an extra model call per chunk at ingestion time, not at query time. Because every chunk of a document shares the same document text, prompt caching on that prefix makes the per-chunk calls much cheaper. A cheap, fast model is usually enough for writing the notes.

Start with the free version first: a deterministic header with document title and heading path. Add model-written context if your evals show chunks are still getting orphaned.