Stateless
The model remembers nothing between requests.
A property of the inference API: each request is processed independently, with no memory of previous ones. Any apparent memory is the application resending history.
Understanding statelessness explains a whole class of bugs — why the model "forgot" your name, why two users’ requests can never contaminate each other, and why you must persist sessions yourself.
This is the fact that generates most architectural complexity in LLM applications. Because the model keeps nothing, your application must own conversation state, decide what to replay on each turn, and decide what to drop. Every "memory" feature is an implementation of that burden.
Scale without sticky sessions
Autoscaling an API behind a load balancer.
No request needs the same server as the last one, because nothing is remembered server-side. State lives in your database and is sent with each request, which is what makes horizontal scaling straightforward.
- docsAgents ↗
Compares a managed hosted agent, an in-app SDK loop, and the raw Responses API, including sessions, handoffs, approvals, sandboxing and compaction.
- docsContext windows ↗
What actually consumes the window — input, tools, thinking and output — plus context rot and compaction. Authoritative for this product, not for models generally.
Session
One continuous run of work with the agent, with its own accumulated history.
Stateful
Something that carries information forward — the app, usually, not the model.
Context
Everything the model can currently see.
Memory System
Storage that gives an assistant continuity across sessions.