Prompt Versioning
Treating prompts as versioned, reviewable artifacts.
Storing prompts in version control with an identifier per version, so every response can be traced to the exact prompt that produced it and changes can be reviewed and rolled back.
Without it you cannot reproduce an incident, compare two behaviours, or explain a regression. Prompts are configuration with code-level impact.
The minimum viable version is unglamorous: prompts in version control, an identifier recorded on every request, and the ability to roll back. That single change is what turns "quality dropped" from an investigation into a diff.
Attributing an incident
A quality drop is reported and no deploy happened.
With prompt versions recorded alongside traces, the change is found in minutes — an edit to the summary instructions last Tuesday. Without them, the investigation starts with the model provider and takes a week.
- docsEvaluation best practices ↗
Evals as structured tests: task-specific test sets, continuous evaluation, logging, and human calibration of automated scoring.
- docsLangSmith evaluation ↗
Distinguishes offline evaluation on datasets from online monitoring of production traffic, and shows failing production traces being promoted into regression datasets.
Prompt Template
A prompt with variables, versioned like code.
Observability
Being able to see what the system actually did.
Regression Test
A test that catches a specific past failure coming back.
Tracing
Following one request through every step it took.