Agent Deadlines Should Travel With the Work

An agent timeout is usually added too late.

The team notices that a run sometimes hangs. Someone adds a thirty-second HTTP timeout around the model call. A tool wrapper gets a separate timeout. The worker has a queue visibility timeout. The browser automation layer has its own wait limit. The result looks safer, but the system still wastes work after the user has already moved on.

The problem is not that one timeout is missing. The problem is that the agent has no shared idea of whether the work is still worth doing.

A deadline is different from a timeout. A timeout is a local setting that says, "stop this operation after this long." A deadline is a runtime budget that says, "the whole run must be useful before this point in time." In production agent systems, that budget has to travel with the work. It should be visible to the planner, tool router, worker queue, retry policy, approval gate, and observability pipeline.

The memorable rule is simple: a deadline is not patience. It is permission to keep spending resources.

Distributed systems already learned this lesson

This is not a new idea in infrastructure. gRPC defines a deadline as the point in time past which the client is no longer willing to wait for a response. Its documentation is blunt about the default: without an explicit deadline, a client can wait effectively forever. It also explains why servers need to know when to stop, because continuing after the client has given up wastes resources and hurts latency.

That last part matters for agents. A model call can return late. A tool can keep executing after the run is no longer valuable. A retrieval job can continue burning database and embedding capacity after the answer path has already failed. A browser worker can keep navigating a page after the user-visible deadline has expired.

If each layer only owns its own timeout, nobody owns the remaining budget.

The stronger design is to propagate a deadline through the run. gRPC has a formal version of this for RPC calls that fan out to other services. OpenTelemetry has a related concept in context propagation: execution metadata needs to cross process boundaries if traces are going to tell a coherent story. Agent runtimes need the same instinct. The deadline belongs in the run context, not in scattered SDK options.

Why agents make local timeouts worse

A normal request handler often has one obvious unit of work. An agent run does not. It is a chain of decisions and side effects:

1. classify the task
2. plan the next step
3. call a model
4. retrieve context
5. call tools
6. wait for external APIs
7. retry failures
8. ask for approval
9. resume after interruption
10. write a final answer or artifact

Each step can look reasonable in isolation. Together, they can violate the user's budget.

A five-second model timeout, a ten-second retrieval timeout, and three tool retries do not create a reliable fifteen-second agent. They create a run whose worst-case behavior depends on the hidden multiplication of attempts, queue waits, network backoff, and recovery work.

That is why a deadline should be carried as runtime state:

{
  "run_id": "run_7f3c",
  "started_at": "2026-08-04T13:01:00Z",
  "deadline_at": "2026-08-04T13:01:45Z",
  "remaining_budget_ms": 45000,
  "budget_owner": "user_visible_response",
  "on_expiry": "cancel_tools_and_return_partial"
}

This object is not meant to be a product schema. It is the contract the runtime should enforce. Every component that spends time reads from the same budget. Every component that schedules more work asks the same question: is this still useful before the deadline?

The budget ledger beats the retry counter

Retries are often configured as if time were infinite. Three attempts sounds conservative until each attempt can invoke a model, a vector database, an HTTP API, and a worker queue.

Temporal's failure-detection model is a useful contrast. Temporal separates workflow execution, activity timeouts, retry policies, cancellation, and non-retryable failures. The important lesson is not that every agent should run on Temporal. The lesson is that retry behavior must be tied to explicit execution semantics. Some failures deserve another attempt. Some failures should fail fast. Some work should stop because the timeout threshold has made success irrelevant.

For agents, a deadline-aware retry policy looks less like this:

Local policy	Hidden failure
max_retries: 3	Retries continue after the user-visible budget is gone.
tool_timeout: 10s	A late tool can consume the entire run budget by itself.
model_timeout: 30s	The planner has no time left to validate or recover.

It should look more like this:

Runtime question	Better behavior
How much budget remains?	Choose smaller model, smaller retrieval, or partial answer.
Is the operation idempotent?	Retry only if repeated side effects are safe.
Is the error non-retryable?	Fail fast on bad input, auth errors, or invalid tool arguments.
Will recovery fit inside the deadline?	Prefer explicit partial failure over stale success.

The retry counter is a local limit. The deadline is the system-level budget ledger.

Tool calls need deadline checks at both edges

Claude's tool-use documentation describes the core loop clearly: the model asks for a tool, the client executes the tool, and the result is sent back. That boundary is where many agent systems lose control. The model is not executing the database query, browser action, deployment command, or ticket update. The runtime is.

That means the runtime needs two deadline checks.

First, before the tool starts, it should decide whether the call still fits. If the run has eight seconds left, starting a tool with a known p95 of twenty seconds is not resilience. It is denial.

Second, after the tool returns, the runtime should decide whether the result is still usable. A tool result that arrives after the deadline may be valuable for logs, compensation, or caching. It should not silently continue the user-facing plan as if the budget still existed.

A practical tool contract should include fields like:

{
  "tool": "search_docs",
  "idempotent": true,
  "expected_p95_ms": 1200,
  "deadline_at": "2026-08-04T13:01:45Z",
  "cancel_signal": "run_7f3c.cancelled",
  "late_result_policy": "record_but_do_not_plan"
}

The exact names do not matter. The boundary does. If a tool can spend time or create side effects, it needs the deadline and the cancellation signal.

Durable agents need deadline persistence too

Durable execution frameworks make agent runs resumable. LangGraph, for example, documents durable execution through checkpoints and stores that persist run state across a thread or across threads. This is a good direction because long-running agents cannot be treated as one fragile HTTP request.

But durability without deadlines can preserve the wrong thing. A resumed run should not only know what it was doing. It should know whether the original budget still makes sense.

There are three different cases:

Resume case	Runtime decision
User-visible deadline is still open	Continue, but recompute remaining budget.
Deadline expired but side effects need cleanup	Run compensation or reconciliation only.
Deadline expired and no cleanup is needed	Mark the run expired and stop planning.

This distinction prevents a common failure mode: an agent wakes up, reads a checkpoint, and continues a plan whose original reason has disappeared. Durable execution should preserve intent and limits, not just messages.

Observability should show budget burn

If deadlines are runtime state, traces should expose them. Otherwise the team only sees that a run timed out, not where the budget went.

A useful trace for an agent run should answer:

• What deadline did the run start with?
• How much time was spent before the first model call?
• Which tool consumed the largest share of remaining budget?
• Did any retry start after the run had too little budget to recover?
• Were late results recorded, ignored, or used incorrectly?
• Did cancellation reach the spawned work?

OpenTelemetry context propagation is useful precedent here. Trace context exists because a distributed operation cannot be understood if each service keeps its own private view of the work. Agent deadlines need the same visibility. A timeout event without the propagated deadline is just a symptom. A trace with budget burn shows the cause.

A small budget ledger can be more useful than another prompt metric:

Span	Started with	Spent	Remaining	Decision
classify intent	45s	1s	44s	continue
retrieve context	44s	4s	40s	continue
call model	40s	18s	22s	continue
execute tool	22s	16s	6s	skip retry
final answer	6s	3s	3s	return partial with caveat

That table changes debugging. The question stops being "which timeout fired?" and becomes "which decision spent the budget?"

An original diagram of the runtime contract

flowchart LR
  U["User request"] --> R["Run context"]
  R --> D["deadline_at"]
  R --> P["Planner"]
  P --> M["Model call"]
  P --> T["Tool router"]
  T --> W["Worker queue"]
  W --> X["External side effect"]
  D --> P
  D --> M
  D --> T
  D --> W
  D --> O["Trace spans"]
  X --> C["Late result policy"]
  C --> O

The diagram is intentionally boring. That is the point. The deadline should not be an exotic feature. It should be part of the basic run context that every expensive or side-effecting operation receives.

Implementation pattern

A production-oriented agent runtime does not need a complicated scheduler to start. It needs a small set of invariants.

First, create the deadline once at ingress. Convert user expectations, product SLOs, cron windows, or background-job policies into a single deadline_at timestamp.

Second, pass that value through every model, retrieval, tool, worker, and approval boundary. Do not let downstream components invent their own unrelated budgets unless they derive from the remaining time.

Third, make every retry ask whether recovery still fits. If not, return a partial answer, mark the run expired, or schedule a separate background reconciliation task with a new owner and a new deadline.

Fourth, record budget burn in traces. A deadline that is not observable becomes another hidden setting.

Fifth, treat late results as a separate state. Late results may be useful for cache warming, audit logs, compensation, or future runs. They should not automatically re-enter the active plan.

What this changes tomorrow

The practical change is small but high leverage. Stop reviewing agent reliability by listing timeouts. Review it by following the deadline.

Ask these questions in design review:

• Where is the run deadline created?
• Which components receive it?
• Which components can spend time without seeing it?
• Which retries are disabled when the remaining budget is too small?
• Which side effects can continue after cancellation?
• What happens to a late tool result?
• Can the trace prove that cancellation propagated?

If the team cannot answer those questions, the agent does not have a runtime budget. It has a pile of timeout settings.

The decision rule is this: when an agent can call tools, retry work, or resume later, do not configure timeouts one component at a time. Propagate a deadline through the run, make every expensive boundary spend from it, and treat expired work as a state the system handles deliberately.

Sources

• gRPC Deadlines
• OpenTelemetry Context Propagation
• Temporal TypeScript Failure Detection and Timeouts
• LangGraph Durable Execution
• Claude Tool Use Overview

References

1. gRPC Deadlines
2. OpenTelemetry Context Propagation
3. Temporal TypeScript Failure Detection and Timeouts
4. LangGraph Durable Execution
5. Claude Tool Use Overview