RAG
also called Retrieval-Augmented Generation
Look things up, then put what you found in the prompt.
A pattern where relevant content is retrieved from an external source and injected into the prompt, so the model answers from supplied evidence rather than from memory. Retrieval plus generation.
RAG is the standard answer to "the model does not know about our data" and to "the model makes things up about our data". It keeps knowledge in a store you can update, permission and audit, instead of baked into weights.
Most RAG failures are retrieval failures, not generation failures. If the right passage never reaches the prompt, no amount of model quality will save the answer.
The three-stage shape is worth memorising because failures localise to a stage. Retrieval decides whether the answer is reachable at all; ranking decides whether it is in the top few; generation decides whether the model uses it honestly. A bad answer tells you which of the three to debug next.
Measuring retrieval separately from generation
Answers are wrong and the team keeps tweaking the generation prompt.
First compute retrieval hit rate on a set of questions with known source documents. If the right chunk is not in the top ten, no prompt will save it — the bug is in chunking or search, and the prompt work was the wrong hour spent.
Contextual chunks beat smaller chunks
Chunks lose the referent of a pronoun and retrieve poorly.
Prepend a short generated summary of where the chunk sits in its document before embedding. A chunk that says "this is from the 2024 EU pricing annex, discussing refunds" embeds far more usefully than three orphaned sentences.
Reaching for fine-tuning to teach facts that RAG would serve better and keep current.
- paperRetrieval-augmented generation for knowledge-intensive NLP tasks ↗
The original RAG formulation: combine a parametric model with a non-parametric dense index, so answers can be grounded in retrieved text.
- blogContextual retrieval in AI systems ↗
The practical hybrid-retrieval article: chunking, embeddings, BM25, rank fusion and reranking, plus prepending chunk context to reduce retrieval failures.
- docsFile search ↗
A hosted retrieval tool over vector stores with both semantic and keyword search — managed RAG, as opposed to building the pipeline yourself.
Embedding · Vector
A list of numbers representing meaning, so similar things land close together.
Vector Database · Vector store
Storage built for finding the nearest vectors, fast.
Chunking
Splitting documents into retrievable pieces.
Reranking · Reranker
A second pass that reorders candidates by actual relevance.
Hallucination
Fluent output that is not true.
Context
Everything the model can currently see.