Stateful Agents Need a Runtime, Not a Request Handler

Jul 21 2026 · updated Jul 28 2026 · 10 min · Sieon

On this page

Why this matters now
Reference architecture
Implementation details that change the design
1. Address sessions by domain identity
2. Keep resumable state close to execution
3. Treat WebSockets as transport, not state
4. Split long-running work from interactive turns
5. Put tool capability behind a runtime boundary
Rollout guidance
Pitfalls and tradeoffs
Practical takeaways
References

Most production agent failures are not model failures. They are runtime failures: the browser disconnects, the worker restarts, a tool call takes longer than expected, a human approval pauses the flow, or a sub-agent loses its partial state. If your agent is only a stateless request handler wrapped around an LLM call, every one of those events becomes an application-level recovery problem.

Cloudflare's Agents platform is interesting because it treats the agent session itself as durable infrastructure. The public docs describe each agent instance as a durable identity with local SQL storage, real-time connections, scheduled work, and recoverable execution on top of Cloudflare Durable Objects. That is the right mental model for senior engineers: the agent is not a prompt. It is a small, addressable service with state, transport, execution, and operational boundaries.

This article uses Cloudflare Agents and Durable Objects as a concrete reference architecture, not as a universal prescription. The broader lesson applies to any agent platform: once an agent touches real systems, the runtime contract matters as much as the model contract.

A caveat matters here: public docs describe platform behavior and API surfaces, not a benchmark of every workload. Treat the patterns below as engineering boundaries to evaluate in your own latency, cost, tenancy, and compliance environment.

Why this matters now

The first wave of agent applications over-indexed on tool definitions and orchestration loops. That was useful, but incomplete. A production agent also needs to answer harder runtime questions:

Where does session state live when the user reconnects?
What happens to a long-running model turn when the client tab closes?
How are approvals, retries, and scheduled follow-ups resumed after process eviction?
How do you isolate state by user, workspace, ticket, repository, or channel?
How do you expose browser, sandbox, MCP, and external API capabilities without flooding every prompt with tool schemas?

Cloudflare's current Agents docs frame the runtime around durable identity, state, sessions, routing, WebSockets, scheduling, fibers, and observability. The Agents API reference states that each agent instance can be addressed by a unique identifier and that the same name routes back to the same instance. That removes a common source of accidental distributed-systems complexity: reconstructing agent state from an external session store on every request.

The June 2026 Agents SDK changelog pushes the same direction. It highlights durable browser sessions, a Code Mode runtime with a durable execution log, approval-gated continuation, improved stream replay, and recovery across deploys, Durable Object evictions, and connection churn. Those are runtime properties, not prompt-engineering tricks.

Reference architecture

A practical agent runtime separates channels, agent identity, durable state, tool execution, and background workflow. The architecture below is deliberately small enough to implement, but explicit about the failure boundaries.

flowchart LR
  U["User or system event"] --> C["Channel adapter"]
  C --> R["Agent router"]
  R --> A["Durable agent instance"]
  A --> S["Local state and session SQL"]
  A --> T["Tool runtime"]
  T --> B["Browser or sandbox"]
  T --> M["MCP servers"]
  T --> E["External APIs"]
  A --> W["Workflow or queue"]
  W --> A
  A --> O["Logs, metrics, traces"]

The important design choice is the center of gravity. The durable agent instance owns the session identity and the local coordination state. Channel adapters should be thin. Tools should be capability providers. Workflows and queues should handle long-running or asynchronous work. Observability should describe the agent turn, not just the HTTP request.

Cloudflare maps this pattern onto existing primitives. The Agent class extends an abstraction layered on Durable Objects. Durable Objects are globally addressable compute instances with colocated durable storage, including SQLite-backed storage that Cloudflare now documents as generally available for new Durable Object classes. Agents add state, WebSockets, scheduling, callable methods, queue methods, fibers, sub-agents, MCP client APIs, and higher-level chat abstractions.

That stack matters because agent sessions are rarely pure request-response transactions. They are closer to state machines with human and machine interrupts.

Implementation details that change the design

1. Address sessions by domain identity

The Agents API reference describes instances as separate micro-servers addressed by a unique identifier such as a user ID, email, ticket number, or other entity. Use that seriously. Do not address every session by a random chat ID if the operational boundary is actually a repository, customer, incident, inbox, or workflow run.

Good instance keys make downstream logic simpler:

user:${userId} for a personal assistant with long-lived state.
repo:${owner}/${repo} for code review or maintenance agents.
ticket:${ticketId} for support triage.
incident:${incidentId} for incident response coordination.
workflow:${runId} for isolated background execution.

The key should reflect the unit that needs serialized coordination. If two requests must not race each other, route them to the same instance. If they can scale independently, split them across instances.

2. Keep resumable state close to execution

Durable Objects combine compute with attached durable storage. Cloudflare's docs call out strongly consistent storage colocated with the object, plus SQLite APIs for structured state. For agents, that is useful for more than chat history. Store resumable execution state, approval status, tool-call checkpoints, compact session summaries, and idempotency keys near the code that advances the agent.

A minimal schema often starts with four tables:

CREATE TABLE turns (
  id TEXT PRIMARY KEY,
  status TEXT NOT NULL,
  started_at TEXT NOT NULL,
  finished_at TEXT,
  summary TEXT
);

CREATE TABLE tool_calls (
  id TEXT PRIMARY KEY,
  turn_id TEXT NOT NULL,
  name TEXT NOT NULL,
  status TEXT NOT NULL,
  input_json TEXT NOT NULL,
  output_json TEXT,
  idempotency_key TEXT
);

CREATE TABLE approvals (
  id TEXT PRIMARY KEY,
  turn_id TEXT NOT NULL,
  status TEXT NOT NULL,
  requested_action TEXT NOT NULL,
  decided_at TEXT
);

CREATE TABLE memory_snapshots (
  id TEXT PRIMARY KEY,
  scope TEXT NOT NULL,
  content TEXT NOT NULL,
  created_at TEXT NOT NULL
);

This is not a recommendation to build your own database layer for everything. It is a reminder that an agent runtime needs first-class recovery data. If the model retries a side-effecting operation after a reconnect, the application should know whether the action already happened.

3. Treat WebSockets as transport, not state

Cloudflare Agents support real-time connections and WebSocket lifecycle callbacks. The v0.12.4 changelog also describes chat recovery improvements where server turns continue when the browser or client stream is interrupted, while an explicit stop() can still cancel the server turn.

That distinction is production-critical. A WebSocket is a delivery path, not the source of truth. The source of truth is the durable turn state. If a client disconnects, the runtime should be able to continue, pause, or cancel based on policy, not because the TCP connection disappeared.

For user-facing chat, this changes UX and reliability. A refresh should not necessarily kill a model turn. A mobile network transition should not duplicate a payment action. A deploy should not erase the approval state. These behaviors require runtime state, not better message wording.

4. Split long-running work from interactive turns

Cloudflare Workflows provide durable multi-step execution, retries, waiting for events, and pauses that can last minutes, hours, or weeks. Queues provide guaranteed delivery, buffering, batching, retries, and dead letter queues. Agents expose scheduling, queue, and workflow-related APIs in the runtime surface.

Use the right primitive for each shape:

Work shape	Better primitive	Why
User is actively waiting for a streamed answer	Agent turn or chat runtime	Preserves conversational context and streaming UX
Tool call needs approval before a side effect	Agent turn plus approval record	Keeps human decision tied to the exact turn
Multi-step process can outlive the chat	Workflow	Durable steps, retry policy, external event waits
Burst of independent jobs	Queue	Backpressure, retries, dead letter handling
Periodic follow-up	Agent schedule or workflow schedule	Avoids cron glue inside prompts

The mistake is forcing all of these into the same loop. A senior production system lets the interactive agent delegate durable background execution instead of pretending every task fits inside one model response.

5. Put tool capability behind a runtime boundary

The June 2026 Agents SDK changelog describes Code Mode as a runtime with connectors and a durable execution log. Instead of giving the model a huge prompt full of tool definitions, the model gets a single codemode tool, discovers capabilities, writes code against typed globals, and resumes after approval-gated actions by replaying completed calls from the durable log.

That pattern is important beyond Cloudflare. Tool schemas are still contracts, but production agents increasingly need a capability runtime behind those schemas. Browser automation, sandboxed code, MCP tools, and external APIs should expose:

least-privilege connectors;
typed interfaces;
audit logs;
idempotency keys;
approval gates for sensitive operations;
replay or recovery semantics;
budget and timeout controls.

Without that boundary, every tool call becomes a bespoke integration with bespoke failure behavior.

Rollout guidance

Start by moving one high-value agent path from stateless request handling to durable session handling. Do not migrate the whole product at once.

Pick a session identity. Choose the domain key that needs coordination, such as user, ticket, repository, or incident.
Define the turn lifecycle. Use explicit statuses like accepted, running, waiting_for_approval, resuming, succeeded, failed, and canceled.
Persist tool-call records. Store input, output, status, idempotency key, and external object references.
Separate cancel from disconnect. Decide whether a client abort cancels the server turn or only detaches the stream.
Move long work to workflows or queues. Keep the agent responsible for orchestration and user-facing decisions, not every retry loop.
Instrument the agent boundary. Trace turns, tool calls, approvals, retries, and recovery events. HTTP latency alone is not enough.
Test recovery deliberately. Kill connections, redeploy during a tool call, force a retry, deny an approval, and replay a completed side effect.

A useful readiness test is simple: can the user close the tab halfway through a tool-using turn, come back later, and see a correct state without duplicate side effects? If not, you do not have a production runtime yet.

Production tip: write this test as an automated scenario, not a demo script. Start a turn, persist a pending tool call, sever the client connection, reconnect through the same agent identity, then assert that the turn resumes or cancels according to policy and that no external side effect runs twice.

Pitfalls and tradeoffs

Durable identity can become a bottleneck. If one instance serializes too much unrelated work, you have created an artificial hot shard. Pick instance keys that match coordination needs, not organizational charts.

Local state is not global analytics. SQLite inside a durable instance is excellent for session-local coordination. It is not a replacement for a warehouse, search index, or cross-tenant reporting system.

Recovery logic must be idempotent. Durable execution reduces lost work, but it does not absolve external APIs from duplicate protection. Use idempotency keys and store external resource IDs.

Approval UX is part of the runtime. A paused tool call needs a stable approval object, clear user-facing explanation, and a safe resume path. Do not hide approvals in transient prompt text.

Tool discovery can increase blast radius. Code Mode and connector models are powerful because they reduce prompt bloat and improve flexibility. They also require capability scoping, logging, and policy checks. The runtime boundary must be stricter than the prompt boundary.

Preview every failure mode, not just the happy path. Reconnect, deploy, eviction, provider error, and user cancellation should all have visible behavior in tests. If the only test is a complete chat turn on localhost, you have not tested the runtime.

Practical takeaways

The direction is clear: production agents are becoming durable, addressable services. Cloudflare's Agents platform makes that explicit by building agents on Durable Objects and exposing runtime primitives for state, sessions, routing, WebSockets, scheduling, queues, workflows, sub-agents, MCP, browser automation, sandboxed code execution, and observability.

You do not need to adopt that exact stack to apply the lesson. You do need to stop treating agents as stateless request handlers once they operate real systems.

The engineering checklist is straightforward:

Give each agent session a durable identity.
Persist turn state and tool-call state close to execution.
Separate transport disconnect from semantic cancellation.
Use workflows and queues for work that outlives the interactive turn.
Put browser, sandbox, MCP, and external API access behind capability runtimes.
Test recovery as a first-class product behavior.
Keep taxonomy, access policy, and image attribution boring at publish time: this article uses no external image URLs, and its only visual is the original Mermaid architecture diagram above.

The model may decide what to do next. The runtime decides whether the system can survive doing it.