Guardrails
Constraints on what the system will do, enforced in code.
Policy enforcement around a model: input and output filters, allowed-tool lists, spend caps, scope limits, human approvals. Deterministic checks that do not rely on the model choosing well.
Guardrails are code, not prompts. A model can be argued out of an instruction; a spend cap and a tool allowlist cannot. Put the load-bearing safety in the layer that does not reason.
The principle is defence in depth, and specifically that model judgement should not be the only barrier. A filter, an allow-list, a spend cap and an approval rule are all deterministic — they hold even when the model has been persuaded, confused, or injected.
The guardrail that is code
An agent must never send external email without approval.
Implement it as a permission rule on the tool, not as an instruction in the prompt. The prompt version holds until something in the context argues against it — and untrusted retrieved content can argue. The code version holds regardless.
Relying on instructions alone as your only safety mechanism. Prompts are suggestions.
- docsConfigure permissions ↗
Allow, ask and deny rules for edits, shell commands and other tools: what a permission mode looks like when it is written down.
- docsSafety best practices ↗
Application-level controls: adversarial testing, human review for high-stakes and code output, moderation, bounded inputs, and telling users what the model cannot do.
- standardLLM01: Prompt injection ↗
Defines direct versus indirect injection and the downstream effects — data disclosure, unauthorised tool use, command execution — with least privilege and human approval as the recommended controls.
Prompt Injection
Malicious instructions hidden in content the model reads.
Permission Mode · Autonomy level
How much the agent may do without asking.
Sandbox
An isolated environment where an agent can act without risk to real systems.
Moderation
Filtering inputs and outputs against a policy.
Alignment
Getting the system to pursue what we actually intended.