On this page
- The decision rule: retry only when the ledger changes
- Retry budgets are control-plane state
- What an attempt ledger records
- A runnable retry loop contract
- Tradeoffs and failure modes
- Instrumentation and review
- FAQ
- How is an attempt ledger different from max_turns or recursion_limit?
- When should an agent retry a failed tool call?
- What should I log for every retry attempt?
- Is unlimited retry ever safe?
- References
A retry is not another chance for the agent to improvise. It is a budgeted control-plane decision. Production loops should spend another turn only after recording what failed, what changed, which policy allows the retry, and which stop condition will end the run if progress does not appear.
The decision rule: retry only when the ledger changes
The most useful rule I have found for production agent loops is simple: do not retry unless the ledger changes.
A loop can hit a hard framework fuse for many reasons. LangGraph raises GRAPH_RECURSION_LIMIT when a graph reaches the maximum number of steps before hitting a stop condition, often because a cycle never exits, while complex graphs can intentionally raise the limit when the design really needs more steps LangGraph. The OpenAI Agents SDK describes Runner.run as a loop over model calls, final-output checks, handoffs, and tool calls, and it can raise MaxTurnsExceeded when max_turns is exceeded OpenAI Agents SDK. AutoGen makes the same operational point from another direction: multi-agent conversations can continue forever, so the runtime needs termination conditions AutoGen.
Those mechanisms are fuses. They tell you that the loop must stop eventually. They do not tell you whether the next retry is justified.
The attempt ledger is the missing middle layer. Before a loop spends another model turn, repeats a tool call, or re-enters a graph edge, the runtime writes one small record:
- what failed
- what state was observed
- what input, prompt, tool arguments, or wait interval changed
- which retry budget is being spent
- which condition will stop the next attempt
If none of those fields changes, the retry is usually just a more expensive way to repeat the same failure. Raising max_turns, increasing a graph recursion limit, or adding another fallback model may hide the incident, but it does not create new information.
Decision rule: a retry may spend one more loop iteration only when it records a new cause, a changed input, a budgeted wait, or an explicit escalation path.
Retry budgets are control-plane state
A senior agent team should treat retry policy as runtime state, not as exception-handling decoration.
Workflow engines already model this explicitly. Temporal retry policies include fields such as initial interval, backoff coefficient, maximum interval, maximum attempts, and non-retryable errors Temporal. Temporal also documents a default Activity retry policy that uses exponential backoff with a 2.0 coefficient, starts at a one-second interval, caps at a 100-second maximum interval, and treats maximum attempts of zero as unlimited Temporal. AWS Step Functions defaults to failing the whole execution when a state reports an error, but lets a state define Retry and Catch fields to retry transient failures or move to a fallback path AWS Step Functions. Kubernetes Jobs expose failure and backoff behavior so the controller can decide whether failed pods should be retried or counted toward job failure Kubernetes.
Agent loops need the same discipline, but across more dimensions. A single run may consume:
| Budget | What it limits | Why it matters |
|---|---|---|
| Turn budget | Model and tool loop iterations | Prevents silent infinite reasoning loops |
| Token budget | Prompt and completion usage | Caps cost and context drift |
| Time budget | Wall-clock duration | Protects queues, user waits, and SLAs |
| Tool budget | External calls and side effects | Prevents duplicate writes and rate-limit storms |
| Retry budget | Attempts per failure class | Separates transient recovery from stuck behavior |
AutoGen exposes termination conditions for message count, token usage, timeout, handoff, and source matching AutoGen. That variety is important. A loop can be safe under one budget and unsafe under another. A run might have turns remaining but no tool budget. It might have tokens remaining but no useful state change. It might have time remaining but a non-retryable policy error.
This is why retry policy belongs beside the loop scheduler, not inside a helper function. The scheduler is the component that can see the graph step, the agent turn, the tool result, the remaining budgets, and the user-facing objective. It can decide whether the next attempt is recovery, exploration, or thrashing.
What an attempt ledger records
An attempt ledger is not a full transcript. It is a compact control record that answers one review question: why was the loop allowed to continue?
A practical schema has eight fields:
run_id: the stable identifier for the user-visible task.attempt: the monotonic attempt number for this failure class.state_digest: a hash of the relevant state, not the whole prompt.cause: the failure or uncertainty that triggered the retry.action: the recovery action, such as backoff, tool-argument repair, model retry, graph resume, or human escalation.budget: the remaining turn, token, time, and tool budgets.delay: the wait interval when the policy uses backoff.stop_reason: the condition that will stop or escalate the next attempt.
OpenTelemetry's generative AI semantic conventions give teams a public vocabulary for model-operation telemetry, including spans and events around model interactions that can carry model, provider, request, response, usage, and operation attributes OpenTelemetry. The OpenTelemetry registry also maintains generative AI attributes that can be attached to telemetry OpenTelemetry Registry. The point is not to invent a private log dialect for every agent framework. The point is to attach retry decisions to traceable runtime events.
A ledger entry should be small enough to read during incident review. If it becomes another transcript, engineers will ignore it. If it omits budgets and stop reasons, it cannot explain why the runtime kept going.
A good ledger entry sounds like this:
{
"run_id": "support-triage-4812",
"attempt": 2,
"state_digest": "b4f1e9c712e8a031",
"cause": "tool_timeout",
"action": "retry_with_backoff",
"budget": {"turns": 3, "tool_calls": 1, "seconds": 45},
"delay": "PT8S",
"stop_reason": "no-progress"
}
That record is enough to audit the next turn. It says the runtime is not merely asking the model again. It is spending a specific budget in response to a specific failure with a specific stop condition.
A runnable retry loop contract
Here is a minimal Python contract I would start with before integrating any specific agent framework. It separates retry policy from loop execution and blocks identical no-progress attempts. The example uses only the Python standard library.
from __future__ import annotations
from dataclasses import dataclass
from hashlib import sha256
from typing import Literal
StopReason = Literal[
"continue",
"success",
"no-progress",
"non-retryable",
"budget-exhausted",
"needs-human",
]
@dataclass(frozen=True)
class Budget:
turns: int
tool_calls: int
seconds: int
@dataclass(frozen=True)
class AttemptEntry:
run_id: str
attempt: int
state_digest: str
cause: str
action: str
budget: Budget
delay_seconds: int
stop_reason: StopReason
def digest_state(state: str) -> str:
return sha256(state.encode("utf-8")).hexdigest()[:16]
def should_retry(previous: AttemptEntry | None, current: AttemptEntry) -> bool:
if current.stop_reason != "continue":
return False
if current.budget.turns <= 0 or current.budget.tool_calls <= 0:
return False
if previous is None:
return True
changed_state = current.state_digest != previous.state_digest
changed_cause = current.cause != previous.cause
changed_action = current.action != previous.action
changed_delay = current.delay_seconds > previous.delay_seconds
return changed_state or changed_cause or changed_action or changed_delay
def next_attempt(
run_id: str,
attempt: int,
state: str,
cause: str,
budget: Budget,
) -> AttemptEntry:
if cause in {"permission_denied", "schema_rejected"}:
return AttemptEntry(
run_id,
attempt,
digest_state(state),
cause,
"escalate",
budget,
0,
"non-retryable",
)
delay = min(2 ** attempt, 30)
return AttemptEntry(
run_id,
attempt,
digest_state(state),
cause,
"retry_with_backoff",
budget,
delay,
"continue",
)
if __name__ == "__main__":
first = next_attempt(
"run-1",
1,
"tool timed out before result",
"tool_timeout",
Budget(turns=3, tool_calls=2, seconds=60),
)
second = next_attempt(
"run-1",
2,
"tool timed out before result",
"tool_timeout",
Budget(turns=2, tool_calls=1, seconds=45),
)
print(should_retry(None, first))
print(should_retry(first, second))
The second retry is allowed only because the delay changed and budget remains. If the same state, cause, action, delay, and budget repeat, the runtime should stop with no-progress or needs-human. In a LangGraph deployment, that check can run before re-entering a cyclic edge. In an OpenAI Agents SDK deployment, it can run around the runner loop that alternates model calls, handoffs, and tool calls OpenAI Agents SDK. In an AutoGen deployment, it can complement termination conditions rather than replacing them AutoGen.
The key design choice is that the ledger is not owned by the model. The model can propose a recovery action, but the runtime owns the retry permission.
Tradeoffs and failure modes
Attempt ledgers add friction, and that friction is intentional.
The first tradeoff is latency. Temporal-style retry policies use explicit backoff fields because retry timing is part of reliability behavior Temporal. Agent loops often skip that step and immediately ask the model to try again. That feels responsive until the runtime hammers the same tool, burns the same context window, or creates a queue burst. Backoff is not only for HTTP clients. It is for any control loop that needs to distinguish transient failure from persistent failure.
The second tradeoff is side-effect safety. AWS Step Functions separates retry behavior from fallback transitions through Retry and Catch AWS Step Functions. Agent tools need the same separation. Retrying a read-only retrieval call is different from retrying a purchase, a message send, a database update, or a deployment step. The ledger should record whether the previous attempt reached the external system and whether the next attempt is idempotent.
The third tradeoff is context quality. A retry can improve the run only when it adds information or changes the plan. Re-prompting with the same context after a schema error may work once. Re-prompting after the same schema error five times is not recovery. It is missing validation. The ledger makes that visible because repeated entries will show the same state digest and cause.
The fourth tradeoff is human escalation. AutoGen's termination-condition model includes multiple ways to stop a conversation, including time, token, message, and handoff conditions AutoGen. Production systems should make needs-human a first-class stop reason, not an embarrassing failure mode. A human handoff after two clear ledger entries is better than an invisible ten-turn loop that finally returns a vague apology.
Instrumentation and review
The operational question is not, "Did the agent eventually answer?" The better question is, "Would we allow that recovery path again?"
To answer it, emit one span or event for each attempt decision. OpenTelemetry's generative AI conventions are a reasonable place to anchor model-operation telemetry OpenTelemetry. Add application attributes for the fields the public conventions do not own yet: agent.loop.attempt, agent.loop.cause, agent.loop.action, agent.loop.state_digest, agent.loop.remaining_turns, and agent.loop.stop_reason.
Then review retries like production incidents:
- Which failure classes consumed the most attempts?
- Which retries changed state and which repeated identical inputs?
- Which tools are retried without idempotency keys?
- Which runs hit framework fuses such as graph step limits or maximum turns?
- Which stop reasons led to useful human intervention?
This also changes how teams tune caps. If a LangGraph run hits a recursion limit, the docs say complex graphs may need a higher limit, but an unexpected limit usually indicates a cycle that does not stop LangGraph. The ledger tells you which case you have. If an OpenAI Agents SDK run hits MaxTurnsExceeded, the runner docs tell you the loop exceeded its maximum turn count OpenAI Agents SDK. The ledger tells you whether those turns were productive recovery or repeated indecision.
That distinction is the heart of Loop Engineering. Caps keep systems from running forever. Ledgers help teams decide whether another iteration deserves to exist.
FAQ
How is an attempt ledger different from max_turns or recursion_limit?
max_turns and recursion limits are fuses. The OpenAI Agents SDK can raise MaxTurnsExceeded when a run exceeds its turn limit OpenAI Agents SDK, and LangGraph can raise GRAPH_RECURSION_LIMIT when a graph exceeds its step limit before a stop condition LangGraph. The ledger explains why each retry was allowed before those fuses blew.
When should an agent retry a failed tool call?
Retry when the cause looks transient, the next attempt changes timing or inputs, the tool call is safe to repeat, and budget remains. Temporal retry policies explicitly model intervals, backoff, maximum attempts, and non-retryable errors Temporal. Agent tools should be held to the same standard.
What should I log for every retry attempt?
Log the run id, attempt number, state digest, cause, recovery action, remaining budgets, delay, and stop reason. OpenTelemetry's generative AI conventions provide a public base for model-operation spans and events OpenTelemetry, while application-specific fields can capture loop policy decisions.
Is unlimited retry ever safe?
Only when a stronger outer control surface exists. Temporal documents that maximum attempts of zero is evaluated as unlimited for Activity retries Temporal, but production agent loops should still pair any unlimited inner policy with time, token, tool, side-effect, and human-escalation boundaries.