Bias
also called Fairness
Systematic skew in outputs, usually inherited from data.
Systematic differences in a model’s behaviour across groups or contexts, reflecting patterns in training data, prompt wording or the design of the surrounding system.
Evaluation is the only way to see it: measure outcomes across relevant groups rather than assuming a model is neutral. Prompt phrasing and role assignment can shift results measurably.
Bias in these systems is rarely a property of the model alone; it emerges from the data, the prompt, the retrieval corpus and the interface together. That is why the useful mitigation is usually to measure outcomes across groups on your own traffic, where you can see the interaction, rather than to audit the base model.
The corpus that decided the answer
A hiring assistant suggests candidates similar to past hires.
The retrieval corpus is historical output, so it reproduces historical skew — a documented historical pattern becomes a recommendation. Measuring the distribution of suggestions across groups is the check that catches it, and it is a data decision rather than a model one.
- standardAI Risk Management Framework: Generative AI Profile (NIST AI 600-1) ↗
The official record for the cross-sector GenAI companion to the AI RMF. Use it as the governance vocabulary; the PDF itself was not machine-readable in this research pass, so do not quote definitions from here.
- paperJudging LLM-as-a-judge with MT-Bench and Chatbot Arena ↗
How well model judges track human preference, and the biases that distort them — position in the pair, and length. Required reading before trusting an automated grader.
- guidePeople + AI guidebook ↗
Human-centred design guidance for AI products: explainability, feedback, setting expectations and keeping a person in the loop. Cited as a reference — the page fetch returned only a landing stub, so do not quote specific claims here.
Evaluation · Evals
How you know the system is actually working.
Moderation
Filtering inputs and outputs against a policy.
Interpretability · Explainability
Working out why the model did that.
Human Eval · Human review
People scoring the output — the ground truth everything else is calibrated against.