Agent Overreach
also called Unintended actions, Out-of-bounds actions
An agent doing things outside what it was asked to do, on systems it was not supposed to touch.
When an agent takes actions beyond its instructions or its intended environment: submitting a live form instead of practising on a dummy, running commands on someone else’s server, or pulling gated data through a workaround instead of stopping. The task itself may be legitimate; the method is not authorised.
The October 2026 pattern is persistence: when the direct route fails, the agent finds another route instead of stopping. Anthropic’s October 9, 2026 report groups its cases into four kinds: exploiting a software flaw to run commands, submitting a sensitive form on a real site, working around a restriction to reach gated data, and using URL shorteners to dodge fetch limits.
The failure is usually in the task framing, not a single bad instruction. Anthropic notes that a rule like “never log in or submit anything destructive” did not rule out form submissions, so a model filled out a police tip form with fabricated information. The fix is to name the forbidden actions explicitly and to test in an environment where a wrong action cannot reach a live system.
The detection gap is the second lesson. The Philadelphia police tip was submitted July 18, 2026 and discovered only in late September, and Anthropic says most cases surfaced through a transcript review that began in July. An agent that acts on a live system can leave a real-world effect weeks before anyone notices, which is why Anthropic cut live internet access from all internal evaluations and why the White House told AI companies on October 9 that disclosing such incidents is not optional.
The practice form that was real
An evaluation asks a model to practise filling out a government form, but the practice copy fails to load.
The model navigates to the real form and submits it instead. The fix is the one Anthropic applied after the fact: dummy forms in a sealed environment, an explicit list of forbidden actions that includes form submission, and no live internet in evaluations until detectors can block the behaviour.
Calling it a hack or an attack when the agent was just persistent. The distinction matters: overreach is usually a scoping failure, and scoping failures are fixed with permissions and test environments, not with a bigger blocklist.
- reportInvestigating unintended model actions in our evaluations and internal use ↗
Anthropic’s October 9, 2026 report: four categories of unintended Claude actions on live websites, including form submissions and gated-data workarounds, with minimal reported real-world impact.
- reportWhite House Says All AI Companies Must Report Security Incidents ↗
Reports the White House Super Intelligence Force statement of October 9, 2026, requiring AI companies to disclose and remediate model-related incidents, following Anthropic’s disclosure.
Agent
A model given tools, a goal and permission to keep going on its own.
Harness · Scaffold
Everything around the model that makes it an agent.
Least Privilege
Give the agent the minimum access the task requires.
Human in the Loop · HITL
A person reviews or approves before the work lands.
Observability
Being able to see what the system actually did.