loading…
The parts that let a model work with private, current, or domain-specific information before they are assembled into a full RAG architecture in Chapter 3.
A model's parameters are not a live database. Retrieval supplies task-relevant evidence at runtime without requiring that every fact be encoded in the model.
Retrieval is useful for private documents, frequently changing policies, product catalogs, support history, and any domain where answers should cite evidence. Chapter 2 focuses on the primitives; Chapter 3 combines them into a complete RAG pattern.
Documents must be converted into retrievable units. Parsing should preserve useful structure such as headings, tables, page numbers, and access-control metadata. Chunking should follow semantic boundaries where possible.
| Chunking choice | Trade-off |
|---|---|
| Very small chunks | Precise matches, but may lose surrounding meaning |
| Very large chunks | More context, but weaker ranking and higher token cost |
| Structure-aware chunks | Better coherence, requires better parsing |
Embeddings map text into vectors so semantically related content can be found even when wording differs. Lexical search matches exact or near-exact terms and is often better for IDs, names, error codes, and rare vocabulary.
Neither method wins universally. Hybrid retrieval combines semantic and lexical signals, then normalizes or fuses their rankings.
Metadata can filter by tenant, document type, date, language, product, or permission group before or during retrieval. Security filtering must be enforced by the retrieval layer, not left to the model.
A first-stage retriever prioritizes speed and recall. A reranker can then score a smaller candidate set more precisely. Context assembly removes duplicates, preserves source labels, and balances coverage against token budget.
Every indexed item needs provenance and a refresh strategy. Common failures include missing the correct document, retrieving a relevant but obsolete version, splitting an important table incorrectly, and presenting a generated answer without traceable sources.
Store source identifiers and locations so the application can render citations and evaluate whether the answer is grounded in the retrieved evidence.