Benchmark Contamination
also called Test set leakage
The test set was in the training data.
When benchmark questions or answers appear in a model’s training data, so a high score partly measures recall of the answers rather than the capability the benchmark intends to test. A known hazard for public benchmarks.
It is a good reason to hold a private evaluation set, and a reason to be suspicious of sudden jumps on widely-used public benchmarks. Overlap detection through n-gram or retrieval-based checks is the standard way to look for it.
The practical defence is a private set. Build a small evaluation from your own real inputs and never publish it, and it cannot leak into anyone’s training data. That set will be less impressive to quote and considerably more useful for decisions.
Checking a suspicious jump
A model improves sharply on a popular public benchmark.
Look for overlap between the test set and the training data before repeating the number. Where contamination is plausible, prefer a private evaluation for the decision — the public score is measuring something, but possibly not the capability it names.
- paperInvestigating data contamination in modern benchmarks for large language models ↗
Detecting test-set overlap between training data and benchmarks, and why a high score on a public benchmark may be partly recall of the answers.
- paperMeasuring massive multitask language understanding ↗
MMLU across 57 subjects. Read it to know what the benchmark tests — not as evidence that one number summarises capability.
Benchmark
A shared public test used to compare models.
Evaluation · Evals
How you know the system is actually working.
Golden Dataset · Eval set
A curated set of cases with known-good answers.
Eval Harness · Harness
The scaffolding that runs your test cases and scores the results.
Online Eval · Production evaluation
Measuring with real users, on real traffic.