LLM
also called Large Language Model
A very large model whose only job, at heart, is predicting the next token.
A language model with billions of learned parameters, trained on enormous amounts of text so that predicting the next token forces it to internalise grammar, facts, style and a surprising amount of reasoning.
Everything an LLM does — answering, coding, summarising, calling tools — is generation, one token at a time, each time conditioned on everything before it. Keep that in mind and its behaviour stops being mysterious.
The single most useful thing to internalise is that the model is a next-token predictor, and that everything else — following instructions, using tools, refusing, apologising — was layered on top of that one objective by later training. When a model does something strange, the base objective is usually the explanation.
Predicting plausible text is not the same as knowing
An assistant invents a plausible-sounding function name that does not exist.
The completion was locally plausible — that is exactly what the objective rewards. The fix is architectural, not a stern instruction: give it the file to read, or a tool to check, so plausibility has something to be measured against.
Scaling bought in-context learning
Why a prompt with three examples works at all.
Larger models began doing few-shot learning from examples in the prompt without any weight updates, which turned prompting into a discipline overnight. It is also why "just describe the format" became a substitute for training a model — up to a point.
Treating the model as a database. It stores a lossy impression of its training data, not records.
- paperLanguage models are few-shot learners ↗
Introduces GPT-3 at 175B parameters and shows that scale alone produced in-context learning: performance from examples in the prompt, with no gradient updates.
- guideIntroduction to large language models ↗
A plain-language official introduction to models, tokens and context. Teaching material rather than origin research — good for the definition, not for a claim about the frontier.
- paperWhy language models hallucinate ↗
The statistical argument that training and grading regimes reward guessing over admitting uncertainty — so hallucination is an incentive problem, not a gap in knowledge.
Next-token Prediction
The single objective: guess what comes next, one token at a time.
Token
The chunk of text the model actually sees — and the unit you are billed in.
Transformer
The 2017 network architecture that all modern language models are built on.
Hallucination
Fluent output that is not true.
Foundation Model · Base model
A large model trained broadly, meant to be adapted to many downstream tasks.