A RAG Pipeline End to End

You have embeddings, a vector index, hybrid search, and nicely chunked documents. None of that is a feature yet. A user doesn't want ten search results; they want "Can I get a refund on an annual plan?" answered in one paragraph, from your documentation, with a link they can check. And when your docs don't cover the question, they want to hear that, not a confident paragraph made up to fill the space.

This lesson wires the pieces into a working question-answering feature for Deskmate's help center: take the question, retrieve the right chunks, assemble them into a prompt, generate an answer, and handle "I don't know" as a first-class outcome instead of an afterthought. By the end you'll have a pipeline you can ship, log, and improve one stage at a time.

The five stages of a RAG request#

Retrieval-augmented generation (RAG) means answering a question by first retrieving relevant text from your own data and then asking a model to answer using only that text. The model supplies language and reasoning; your documents supply the facts. Every RAG request moves through five stages:

Treat these as separate functions with separate logs. When an answer is wrong, the first question is always "which stage failed?" If the right chunk was never retrieved, no prompt change will fix it. If it was retrieved and the model ignored it, no retrieval change will help. The rest of Module 5 improves individual stages: reranking and query rewriting sharpen the first two, citations harden the answer stage, and retrieval evals tell you which stage to work on.

For this lesson, Deskmate's knowledge base is already chunked and embedded into the kb_chunks table from Building Semantic Search into Your App, with chunks built the way Chunking Strategies That Work describes. Here are the columns this lesson's code relies on, including a token_count per chunk:

CREATE EXTENSION IF NOT EXISTS vector;
 
CREATE TABLE kb_chunks (
  id            text PRIMARY KEY,          -- "<article_id>#<chunk_index>"
  article_id    text NOT NULL,
  article_title text NOT NULL,
  section       text NOT NULL,             -- heading path, e.g. "Refunds > Annual plans"
  chunk_index   integer NOT NULL,          -- position within the article
  url           text NOT NULL,
  content       text NOT NULL,
  token_count   integer NOT NULL,
  published     boolean NOT NULL DEFAULT true,
  embedding     vector(1536) NOT NULL
);
 
CREATE INDEX ON kb_chunks USING hnsw (embedding vector_cosine_ops);