Embeddings, Search & RAG
Grounding answers in your own data
Build retrieval into real products: embeddings and vector storage, semantic and hybrid search, document ingestion and chunking, reranking, grounded answers with citations, freshness and permissions, and how to measure whether retrieval works.
- Lessons
- 12
- Reading time
- 125 min
- Level
- Intermediate
- Access
- Full course
Lessons in this module
- 5.111 min
Embeddings in Practice
Generate, store, and compare embeddings in code; pick an embedding model; and build intuition for what similarity scores do and don't mean.
- 5.210 min
Choosing a Vector Store
pgvector, dedicated vector databases, and managed search services compared — with a clear default for most apps and the signals that you've outgrown it.
- 5.310 min
Building Semantic Search into Your App
Ship a "search by meaning" feature end to end with Postgres and pgvector: indexing, querying, filtering, and a results UI users understand.
- 5.410 min
Hybrid Search: Keywords Plus Vectors
Why pure vector search misses exact names, codes, and IDs — and how to combine full-text and vector results with reciprocal rank fusion.
- 5.511 min
Ingesting Documents: PDFs, HTML, and Office Files
Turn real-world files into clean, structured text: extraction tools, tables and layout, metadata worth keeping, and an ingestion pipeline you can re-run.
- 5.610 min
Chunking Strategies That Work
Fixed, structural, and semantic chunking compared; chunk size and overlap; and contextual chunk headers that make retrieval dramatically better.
- 5.711 min
A RAG Pipeline End to End
Wire retrieval and generation into a working question-answering feature: query, retrieve, assemble context, answer, and handle "I don't know."
- 5.811 min
Reranking and Query Rewriting
Two upgrades with outsized impact: rerank candidates with a cross-encoder or model, and rewrite vague or conversational questions into good search queries.
- 5.910 min
Citations and Grounded Answers
Make every claim traceable: citation formats, provider citation features, verifying quotes against sources, and UI that lets users check the work.
- 5.1011 min
Keeping Indexes Fresh and Permission-Aware
Handle updates, deletes, and re-embedding — and make sure users can only retrieve documents they're allowed to see.
- 5.1110 min
Evaluating Retrieval and RAG Quality
Measure retrieval separately from generation: recall@k, MRR, faithfulness, and answer relevance — with a small labeled set you can build in an afternoon.
- 5.1210 min
Long Context vs. RAG: When You Don't Need Retrieval
With million-token context windows, sometimes the best retrieval system is none. How to decide between stuffing context, caching it, and retrieving.
About this module
Retrieval-augmented generation, usually shortened to RAG, is how AI features answer from your own data instead of from whatever a model happened to learn in training. You search your documents for the passages that matter, put them in front of the model, and have it answer from them with citations a user can check. The same retrieval machinery also powers features with no text generation at all, such as search that understands meaning, related articles, and duplicate detection.
Most RAG tutorials stop at a demo: a few clean documents, one vector query, and an answer that looks right. Real products fail in less obvious places. PDFs extract with scrambled columns and flattened tables. Vector search misses exact error codes and product names. Small customers in a shared index get no results because of how filters interact with the index. Deleted or private documents keep showing up. And nobody can say whether a change made retrieval better or worse.
This module builds retrieval the way production apps need it, using Postgres and pgvector as the default store. You'll generate and store embeddings and learn what similarity scores do and don't mean, choose a vector store with clear signals for when to move, ship semantic search with filtering and a results UI, combine keyword and vector search with reciprocal rank fusion, ingest PDFs, HTML, and Office files into clean Markdown, and chunk documents with contextual headers that make retrieval more reliable.
Then you'll wire retrieval into a question-answering pipeline that knows when to say it doesn't know, improve it with reranking and query rewriting, make every claim traceable with citations, keep indexes fresh and permission-aware, and measure retrieval and answer quality with a small labeled set. The module closes with the question that matters in an era of million-token context windows: when you don't need retrieval at all. Examples are in TypeScript with Anthropic's Claude models for generation, built around a realistic help-desk app.
Questions about Embeddings, Search & RAG
What is RAG in AI?
RAG, or retrieval-augmented generation, is a pattern where your app first searches your own documents for passages relevant to a question, then sends those passages to a language model along with the question. The model answers from the retrieved text, ideally with citations. It lets AI features use private, current information the model was never trained on.
Do I need a vector database for RAG?
Usually not a separate one. If you already run Postgres, the pgvector extension stores embeddings next to your data, supports fast approximate nearest-neighbor indexes, and lets you filter with ordinary SQL. Dedicated vector databases make sense at very large scale or for vector-heavy workloads, but they add a second system you have to keep in sync.
What is hybrid search?
Hybrid search runs a keyword search and a vector (semantic) search for the same query and merges the results, commonly with reciprocal rank fusion. Vector search handles paraphrases and fuzzy questions, while keyword search catches exact strings such as error codes, IDs, and product names. Together they cover each other's blind spots.
What is the best chunk size for RAG?
There is no universal best size; it depends on your documents and questions. A common starting point is splitting on document headings and keeping chunks to a few hundred words, then comparing configurations on a set of real questions with known answers. Adding the document title and section path to each chunk often helps more than tuning the size.
Which embedding model should I use?
Shortlist two or three models, such as a hosted option from Voyage AI, OpenAI, Google, or Cohere, or an open-weight model you can run yourself, then test them on your own queries and documents. Public leaderboards are a useful starting point but don't reliably predict performance on your data. Remember that switching models later means re-embedding everything.
When should I use long context instead of RAG?
If your whole knowledge source fits comfortably in a model's context window and doesn't change per user, sending it directly, ideally with prompt caching, can be simpler and more accurate than retrieval. RAG wins when the corpus is large, changes often, needs per-user permissions, or when cost and latency per request matter.