Retrieval-augmented generation (RAG)

RAG pairs a search system with a model. When a question arrives, you retrieve the most relevant passages from your documents or database, put them in the prompt with instructions to answer from them, and have the model write the answer, ideally with citations. The model supplies reasoning and language; your data supplies the facts.

It's the standard way to build features over private or fast-changing information: help-center assistants, internal knowledge bots, document Q&A, and support reply drafting. It works around the knowledge cutoff, keeps answers specific to your product, and lets you update knowledge by updating the index, with no retraining.

Quality depends on two halves that fail differently. Retrieval can miss the right passage because of poor chunking, weak queries, or missing keyword matching. Generation can mishandle a passage it was given by ignoring it, misreading it, or adding outside facts. Evaluate them separately, with retrieval metrics like recall at k and generation metrics like faithfulness, so you fix the right thing.

RAG isn't always required. If your whole corpus fits comfortably in a long-context window and caching makes that affordable, sending the documents directly can be simpler.