Foundation Model
also called Base model
A large model trained broadly, meant to be adapted to many downstream tasks.
A model trained on a very large, general corpus that is intended to serve as a starting point for many applications, rather than being trained for one task. GPT, Claude, Gemini and Llama are foundation models.
The idea is a "pre-train once, adapt many times" economy. The general capability is bought once and then specialised through prompting, retrieval, or fine-tuning.
The word "foundation" describes an economic position as much as a technical one: the model is expensive to build once and cheap to adapt many times, so value concentrates in adaptation rather than in training. That is why most teams should assume they will never train a foundation model and will spend their effort on context, tools and evaluation instead.
Buying the base, owning the adaptation
A company wants to build a support assistant on its own documentation.
The base model is a commodity available from several providers. The defensible part is the retrieval corpus, the evaluation set drawn from real tickets, and the fine-tuned behaviour on their own tone. Those do not transfer when the provider changes — the base model does.
- paperLanguage models are few-shot learners ↗
Introduces GPT-3 at 175B parameters and shows that scale alone produced in-context learning: performance from examples in the prompt, with no gradient updates.
- blogIntroducing the next generation of Claude ↗
A first-party example of a model family positioned by capability, speed and cost, with vision input and a needle-in-a-haystack long-context evaluation.
- paperLlama 2: open foundation and fine-tuned chat models ↗
A released family from 7B to 70B plus chat variants, with the fine-tuning and safety work documented. A primary example of published weights — not a ruling on what "open source" should mean.
LLM · Large Language Model
A very large model whose only job, at heart, is predicting the next token.
Pretraining
The first, most expensive pass: predict text at internet scale.
Fine-tuning
Extra training on your own examples to change how the model behaves.
Frontier Model
A model at the current leading edge of capability.