Agent Budgets Belong in the Runtime

Agent reliability is not only about better prompts or smarter retries. Production agents need a runtime budget governor that tracks tokens, steps, tools, time, cost, and side effects before the loop runs away, because every agent decision consumes scarce operational capacity.

The real failure mode is unbounded work

Most broken agent runs do not look dramatic at first. The model asks for one more search, one more file read, one more summarization pass, one more retry after a malformed response. Each individual action is defensible. The system failure is that no component owns the total shape of the work.

Frameworks already expose pieces of the problem. LangGraph documents GRAPH_RECURSION_LIMIT as the error raised when a graph reaches its maximum number of steps before hitting a stop condition, often because of a cycle, while also noting that complex graphs can raise the configured recursion_limit when they legitimately need more iterations LangGraph. That is a useful safety rail, but it is still only one dimension of the run.

A production agent can stay under a step limit and still fail operationally. It can fit into the context window but burn the user's cost budget. It can finish quickly but call a write-capable tool too early. It can pass a guardrail after the model already spent tokens and touched systems. OpenAI's Agents SDK guardrail docs make that tradeoff explicit: parallel input guardrails reduce latency, but if a guardrail fails, the agent may have already consumed tokens and executed tools; blocking guardrails complete first and are better when avoiding token use or side effects matters OpenAI Guardrails.

That distinction is the core lesson. Budgeting is not a dashboard concern after the run finishes. It is a runtime control plane concern while the run is still making choices.

Budget every scarce resource, not just tokens

Token budgets matter because they are easy to measure, but they are not the whole contract. Anthropic's token counting docs describe preflight token counts as a way to manage rate limits and costs, make routing decisions, and fit prompts to a target length Anthropic Token Counting. That is a good start. It tells you whether the initial request is likely to fit.

Agent loops expand after the initial request. Tool results add text. Handoffs add context. Retry prompts add failure transcripts. Memory retrieval adds snippets that seemed cheap in isolation. A budget governor has to track the full run, not just the first model call.

I usually split budgets into six buckets:

Budget	What it protects	Example enforcement
Step budget	Infinite loops and runaway plans	Stop after 12 planner or tool turns
Token budget	Context, latency, and model cost	Reserve 20 percent for final answer synthesis
Tool budget	External system pressure	Allow 5 search calls and 2 database reads
Time budget	User experience and worker capacity	Return partial results after 45 seconds
Money budget	Product margin and tenant fairness	Cap estimated spend per request or account
Side-effect budget	Safety and reversibility	Allow reads freely, require approval for writes

The important move is to treat these as contracts, not hints buried in a system prompt. A model can be told to be frugal, but the runtime has to enforce the ceiling. Anthropic's task budgets show one useful advisory pattern: the model can receive a token budget for the full agentic loop, including thinking, tool calls, tool results, and output, and can see a running countdown so it can finish gracefully Anthropic Task Budgets. That can improve behavior, but hard production limits still belong in deterministic application code.

Put the governor between the planner and the tools

A budget governor should sit on the hot path. If it is only a log processor, it can explain why the incident happened, but it cannot prevent the incident.

A simple architecture looks like this:

flowchart TD
    U["User request"] --> R["Run policy"]
    R --> G["Budget governor"]
    G --> M["Model call"]
    M --> P["Planner decision"]
    P --> G
    G --> T["Tool dispatcher"]
    T --> O["External systems"]
    O --> G
    G --> S["Final answer or graceful stop"]
    G --> X["Trace and usage sink"]

The governor needs to see four things before each action:

1. The run policy for the user, product tier, workflow, and risk level.
2. The current budget ledger, including consumed and reserved amounts.
3. The proposed action, including tool name, expected cost, side-effect class, and timeout.
4. The stop behavior if the action is denied.

That last point is where many implementations fall apart. If the only stop behavior is "raise an exception," engineers will quietly increase the limits to keep the product usable. A better governor returns a typed decision such as allow, defer, summarize_now, ask_for_approval, or handoff_to_human.

The runtime should also reserve budget for the final answer. If an agent spends every token gathering evidence, it often fails right when it needs to synthesize. I prefer to reserve a fixed percentage plus a minimum floor. For example, a research agent with a 60,000 token budget might reserve 8,000 tokens for synthesis and citations. Every retrieval or tool result is admitted only if the remaining budget still protects that reserve.

Make budget state observable

You cannot tune what you cannot see. OpenAI's Agents SDK usage docs describe run-level tracking for requests, input tokens, output tokens, total tokens, and per-request usage entries, with usage aggregated across model calls, tool calls, and handoffs OpenAI Usage. That is the kind of ledger a governor needs.

The same information has to appear in traces. OpenAI's tracing docs cover traces, spans, custom tracing processors, and sensitive data controls OpenAI Tracing. OpenTelemetry also has a Generative AI semantic conventions area covering GenAI spans, agent spans, events, metrics, and provider-specific conventions OpenTelemetry. The exact attribute names will vary by implementation, but the operational shape should be consistent.

At minimum, I want these fields in every agent run trace:

{
  "agent.budget.step.limit": 12,
  "agent.budget.step.used": 9,
  "agent.budget.input_tokens.limit": 50000,
  "agent.budget.input_tokens.used": 38210,
  "agent.budget.tool_calls.limit": 8,
  "agent.budget.tool_calls.used": 6,
  "agent.budget.final_reserve_tokens": 8000,
  "agent.stop.reason": "final_answer"
}

Those fields answer questions that generic latency charts cannot answer. Did the agent fail because the model was wrong, because retrieval returned too much text, because a tool loop ran too long, or because the configured final-answer reserve was too small? Without budget state in the trace, every incident review turns into prompt archaeology.

Budget observability also keeps teams honest about retries. A retry is not free work. It should decrement the same ledger as any other model call. If a schema repair loop needs three attempts, the trace should show that the request spent three model calls on formatting before doing user-visible work.

Fail gracefully instead of retrying blindly

A budget governor should not just say no. It should help the agent finish in a useful state.

For read-heavy workflows, graceful stop usually means summarizing what was found, stating what was not checked, and asking whether to continue with a larger budget. For write-capable workflows, graceful stop may mean returning a draft plan rather than executing changes. For incident-response workflows, it may mean escalating to a human with the trace, the exhausted budget, and the highest-confidence findings.

Here is the policy I use as a default:

Budget exhausted	Bad behavior	Better behavior
Steps	Increase recursion limit automatically	Stop with loop diagnosis and last state
Tokens	Drop citations or truncate the final answer	Compress evidence, preserve final reserve
Tools	Retry a failing external API until timeout	Return partial result and mark tool unavailable
Time	Keep worker running after user gives up	Emit progress summary and resumable run ID
Side effects	Let the model decide whether to proceed	Require approval or move to dry-run mode

This is where blocking guardrails matter. If a request is not allowed to call a write tool, run that check before the agent starts spending tokens or touching systems. OpenAI's guardrail docs explicitly call out blocking input guardrails as useful for cost optimization and avoiding potential tool side effects OpenAI Guardrails. Treat budget checks the same way.

A minimal Python budget governor

The implementation does not have to be complicated. Start with a small ledger and put it around every model and tool call. This example is dependency-free so the control flow is visible.

from dataclasses import dataclass
from enum import Enum

class Decision(str, Enum):
    ALLOW = "allow"
    SUMMARIZE_NOW = "summarize_now"
    ASK_FOR_APPROVAL = "ask_for_approval"
    STOP = "stop"

@dataclass
class Budget:
    step_limit: int
    tool_limit: int
    token_limit: int
    final_reserve_tokens: int
    steps_used: int = 0
    tools_used: int = 0
    tokens_used: int = 0

    def remaining_tokens_after_reserve(self) -> int:
        return self.token_limit - self.final_reserve_tokens - self.tokens_used

@dataclass
class Action:
    kind: str
    estimated_tokens: int = 0
    side_effect: bool = False

class BudgetGovernor:
    def __init__(self, budget: Budget):
        self.budget = budget

    def decide(self, action: Action) -> Decision:
        if action.side_effect:
            return Decision.ASK_FOR_APPROVAL
        if self.budget.steps_used >= self.budget.step_limit:
            return Decision.SUMMARIZE_NOW
        if action.kind == "tool" and self.budget.tools_used >= self.budget.tool_limit:
            return Decision.SUMMARIZE_NOW
        if action.estimated_tokens > self.budget.remaining_tokens_after_reserve():
            return Decision.SUMMARIZE_NOW
        return Decision.ALLOW

    def record(self, action: Action, actual_tokens: int = 0) -> None:
        self.budget.steps_used += 1
        self.budget.tokens_used += actual_tokens
        if action.kind == "tool":
            self.budget.tools_used += 1

budget = Budget(
    step_limit=8,
    tool_limit=4,
    token_limit=20000,
    final_reserve_tokens=3000,
)
governor = BudgetGovernor(budget)

next_action = Action(kind="tool", estimated_tokens=1200, side_effect=False)
decision = governor.decide(next_action)

if decision == Decision.ALLOW:
    governor.record(next_action, actual_tokens=980)
elif decision == Decision.SUMMARIZE_NOW:
    print("Return a partial answer with checked and unchecked work.")
elif decision == Decision.ASK_FOR_APPROVAL:
    print("Ask for approval before a side-effecting action.")
else:
    print("Stop the run safely.")

The useful part is not the class design. It is the placement. The governor is consulted before every expensive or risky action, and it records actual usage after the action returns. In a real system, the record method should also emit trace attributes and metrics.

The next step is to make the estimates less naive. A retrieval tool can estimate returned tokens from document size. A shell tool can carry a timeout and output cap. A web-search tool can budget both the request and the summarization that follows. A write tool can carry a side-effect class that routes through approval or dry-run mode.

Rollout checklist

Do not try to solve every budget dimension on day one. Roll it out in layers:

1. Add run-level token and request accounting.
2. Add step and tool-call ceilings with graceful stop behavior.
3. Reserve final-answer tokens before admitting retrieval results.
4. Classify tools by read, write, external spend, and irreversible side effect.
5. Add budget fields to traces and dashboards.
6. Create product-tier policies for normal, elevated, and admin workflows.
7. Review budget-exhausted runs weekly and tune thresholds from real traces.

The signal you want is not "the agent never hits a limit." If no run ever hits a limit, the limits are probably decorative. The signal you want is that exhausted runs are understandable, resumable, and safe.

FAQ

Is a recursion limit enough?

No. A recursion or step limit protects against one class of runaway loop, and LangGraph documents how that limit catches graphs that do not reach a stop condition LangGraph. Production systems also need token, tool, time, cost, and side-effect limits.

Should budgets be hard limits or hints to the model?

Use both. Advisory budgets can help the model self-regulate, as Anthropic's task budget feature is designed to do across an agentic loop Anthropic Task Budgets. Hard limits should still live in the runtime when cost, safety, or external side effects are involved.

Where should usage accounting live?

Usage accounting should live at the agent run boundary, not inside one tool or one model wrapper. OpenAI's Agents SDK tracks usage across all model calls in a run, including tool calls and handoffs OpenAI Usage. That is the right shape for budgeting.

What should the agent return when it runs out of budget?

Return a useful partial result: what was completed, what was not checked, why the run stopped, and what budget would be needed to continue. Silent truncation and blind retries both hide the real failure from the user and the operator.

References

1. LangGraph: GRAPH_RECURSION_LIMIT
2. OpenAI Agents SDK: Guardrails
3. OpenAI Agents SDK: Usage
4. OpenAI Agents SDK: Tracing
5. Anthropic: Token counting
6. Anthropic: Task budgets
7. OpenTelemetry: Generative AI semantic conventions