Parametric Knowledge
What the model stores inside its weights.
Knowledge absorbed during training and encoded in the parameters, as opposed to knowledge supplied at inference time. Fast and free to use, but lossy, undatable and impossible to edit or attribute.
Parametric knowledge is why models can answer without retrieval at all, and why you cannot ask one to forget a fact. There is no delete button inside a weight matrix.
Three properties explain most of its problems: it is undated, so you cannot tell how fresh it is; it is unattributable, so you cannot point at where a claim came from; and it is uneditable, so correcting it means retraining. Retrieval exists precisely to route around all three.
Correcting a model is not like correcting a wiki
A model repeats an out-of-date policy that changed last year.
There is no page to edit. You can retrieve the current policy and instruct precedence, fine-tune it away, or wait for a future base model — which is why "put the source of truth in the context" is the standard answer.
- paperRetrieval-augmented generation for knowledge-intensive NLP tasks ↗
The original RAG formulation: combine a parametric model with a non-parametric dense index, so answers can be grounded in retrieved text.
- paperWhy language models hallucinate ↗
The statistical argument that training and grading regimes reward guessing over admitting uncertainty — so hallucination is an incentive problem, not a gap in knowledge.
Contextual Knowledge
What the model knows from the prompt, right now.
Knowledge Cutoff
The date the model’s knowledge stops.
Hallucination
Fluent output that is not true.
RAG · Retrieval-Augmented Generation
Look things up, then put what you found in the prompt.
Fine-tuning
Extra training on your own examples to change how the model behaves.