Long Context vs. RAG: When You Don't Need Retrieval

You've spent most of Module 5 building retrieval: embeddings, a vector store, hybrid search, chunking, reranking, freshness, permissions, and evals. Here's the uncomfortable question this lesson asks: for your feature, do you need any of it?

Every current Claude model has a 1-million-token context window, and other providers offer similar sizes. That's room for well over a thousand pages of text in a single request. If your entire knowledge base fits, you can skip retrieval and hand the model everything, and some of the hardest RAG problems (chunking, missed passages, questions that span documents) simply disappear. The trade-offs move to cost, latency, and attention instead. By the end of this lesson you'll be able to run the numbers for your own corpus and choose between stuffing the context, caching it, retrieving, or a mix.

The question has changed#

When context windows held a few thousand tokens, RAG wasn't a design choice; it was the only way to get your documents in front of a model. Now the question is a genuine trade-off between two ways of deciding what the model sees:

• Retrieval decides before the model call, using search. It's cheap per request and scales to any corpus size, but it can miss. A question whose answer isn't retrieved can't be answered, no matter how good the model is.
• Long context lets the model decide during the call, by reading everything. It can't miss a document that's in the window, it can connect facts across documents, and it needs no index. But you pay to process the whole corpus on every request, and the model has to find the needle itself.

Neither is universally better. A help center with 150 articles and a legal archive with 2 million documents need different answers, and so do a feature used ten times a day and one used ten thousand times an hour. The rest of this lesson is about telling them apart, starting with arithmetic.