Graph Routers Need Decision Contracts

The most dangerous line in an agent graph is often not the tool call. It is the branch that decides which node runs next.

A router can send a customer issue to a human, retry a failed retrieval step, execute a tool, skip a safety review, or end the run. In a notebook, that branch feels like ordinary control flow. In production, it is a control-plane decision. If the decision is prompt-shaped, unrecorded, and hard to replay, the graph will eventually do the wrong thing and leave almost no evidence.

The rule is simple: route from recorded facts, not from ambient prompts.

A production router should not only return goto="human_review". It should also leave behind a decision record that explains what it saw, what destinations were allowed, why this destination won, and which inputs made the choice stable enough to replay.

Routers are part of the runtime contract

LangGraph models workflows as state, nodes, and edges. Nodes receive state, do work, and return updates. Edges determine which node executes next, including conditional branches. The docs summarize the mental model clearly: nodes do the work, edges tell what to do next.

That split is useful, but it can hide an operational problem. The edge is where business policy, model uncertainty, retry state, human review, and side-effect safety meet.

Consider a support agent graph:

graph TD
    A[Read ticket] --> B[Classify]
    B --> C{Router}
    C -->|question| D[Retrieve docs]
    C -->|billing risk| E[Human review]
    C -->|bug report| F[Create issue]
    D --> G[Draft response]
    E --> G
    F --> G

The router is not merely choosing a prettier diagram path. It decides whether a human sees the case, whether an external issue is created, and which evidence later nodes inherit. If it changes behavior after a retry or replay, the graph may look healthy while the business process drifts.

The five-field decision contract

Treat every non-trivial router as a small API. The API has five fields.

Field	Purpose	Failure it prevents
input_snapshot	The raw state facts used for the decision	Hidden prompt context changing routes
allowed_destinations	The finite set of legal next nodes	Hallucinated or deprecated destinations
selected_destination	The chosen next node	Unobservable branch behavior
reason_code	A stable, low-cardinality explanation	Logs that are impossible to aggregate
evidence_ids	Pointers to state facts, retrieved docs, tool results, or approvals	Decisions that cannot be audited

This is intentionally boring. A router contract should be easier to test than the model call that feeds it. The creative work can happen inside classification or planning nodes. The router should convert the current state into a bounded decision.

Record classifications before routing

A common failure pattern is putting classification, explanation, and branching inside one LLM prompt, then returning only the destination. That makes the graph feel compact, but it removes the evidence later engineers need.

Prefer this shape:

1. A classifier node writes raw structured facts into graph state.
2. A router node reads only those facts and the retry or approval state.
3. The router writes a decision record and returns a typed destination.

LangGraph already encourages state to hold raw data rather than formatted prompt text. That principle matters most at routing boundaries. Prompt formatting can evolve. State facts and decision records are the compatibility layer.

from typing import Literal, TypedDict

Destination = Literal["retrieve_docs", "human_review", "create_issue", "end"]

class Classification(TypedDict):
    intent: Literal["question", "billing", "bug", "unknown"]
    urgency: Literal["low", "medium", "high", "critical"]
    confidence: float
    evidence_ids: list[str]

class RouteDecision(TypedDict):
    router: str
    allowed_destinations: list[Destination]
    selected_destination: Destination
    reason_code: str
    evidence_ids: list[str]

class TicketState(TypedDict):
    ticket_id: str
    classification: Classification
    route_decisions: list[RouteDecision]

The important move is not the syntax. The important move is that routing becomes inspectable state. When the graph is checkpointed, streamed, replayed, or reviewed after an incident, the decision is available without reconstructing a hidden prompt transcript.

Make destinations typed and boring

LangGraph supports routing with commands that can update state and select a destination. In production, make the destination set explicit. Do not let a model invent node names. Do not let a router return arbitrary strings. Do not let a default branch swallow unknowns silently.

from typing import Literal
from langgraph.types import Command

Destination = Literal["retrieve_docs", "human_review", "create_issue", "end"]
ALLOWED: list[Destination] = ["retrieve_docs", "human_review", "create_issue", "end"]

def route_ticket(state: TicketState) -> Command[Destination]:
    c = state["classification"]

    if c["confidence"] < 0.75:
        dest: Destination = "human_review"
        reason = "low_confidence"
    elif c["urgency"] == "critical" or c["intent"] == "billing":
        dest = "human_review"
        reason = "risk_review"
    elif c["intent"] == "bug":
        dest = "create_issue"
        reason = "bug_report"
    elif c["intent"] == "question":
        dest = "retrieve_docs"
        reason = "answerable_question"
    else:
        dest = "human_review"
        reason = "unknown_intent"

    decision: RouteDecision = {
        "router": "route_ticket.v1",
        "allowed_destinations": ALLOWED,
        "selected_destination": dest,
        "reason_code": reason,
        "evidence_ids": c["evidence_ids"],
    }

    return Command(
        update={"route_decisions": state.get("route_decisions", []) + [decision]},
        goto=dest,
    )

This pattern gives you three useful properties.

First, a branch has a stable name. risk_review is easier to alert on than a paragraph of model justification.

Second, the router update travels with the same graph state that downstream nodes already read.

Third, route changes become testable. If the classifier produces the same facts, the router should produce the same destination and reason.

Test the router without the graph

You do not need a full LangGraph runtime to test the core routing contract. Keep a stdlib test that exercises the pure decision logic before wiring it into the graph.

VALID_DESTINATIONS = {"retrieve_docs", "human_review", "create_issue", "end"}
VALID_REASONS = {
    "low_confidence",
    "risk_review",
    "bug_report",
    "answerable_question",
    "unknown_intent",
}

def validate_decision(decision: dict) -> None:
    assert set(decision["allowed_destinations"]) == VALID_DESTINATIONS
    assert decision["selected_destination"] in VALID_DESTINATIONS
    assert decision["reason_code"] in VALID_REASONS
    assert decision["router"].startswith("route_ticket.")
    assert isinstance(decision["evidence_ids"], list)

def test_billing_routes_to_human_review() -> None:
    state = {
        "ticket_id": "t_123",
        "classification": {
            "intent": "billing",
            "urgency": "high",
            "confidence": 0.91,
            "evidence_ids": ["email:body", "classifier:v4"],
        },
        "route_decisions": [],
    }

    command = route_ticket(state)
    decision = command.update["route_decisions"][-1]
    validate_decision(decision)
    assert command.goto == "human_review"
    assert decision["reason_code"] == "risk_review"

This does not replace graph-level integration tests. It catches the class of failures that should never need a graph to reproduce: new destination strings, missing evidence, silent default branches, and reason codes that cannot be aggregated.

Design for replay and interrupts

Persistence changes the meaning of routing. LangGraph checkpointers persist thread-scoped graph state for continuation, time travel, human-in-the-loop workflows, and fault tolerance. Interrupts pause execution, save state, and later resume with a command. The docs also warn that side effects before an interrupt should be idempotent because interrupted nodes can rerun.

Routers should follow the same discipline.

A replay-safe router should make the same decision from the same recorded facts. If it depends on current time, random sampling, model temperature, a live feature flag, or mutable external policy, record the resolved value before routing. Better yet, move those inputs into a policy snapshot node, then route from the snapshot.

Bad pattern:

# The route can change when the policy service changes.
def route(state):
    if policy_client.requires_review(state["customer_id"]):
        return Command(goto="human_review")
    return Command(goto="retrieve_docs")

Safer pattern:

def load_policy_snapshot(state):
    return {
        "policy_snapshot": {
            "version": "support-routing-2026-09-18",
            "requires_review": True,
            "evidence_ids": ["policy:support-routing-2026-09-18"],
        }
    }

def route_from_snapshot(state):
    policy = state["policy_snapshot"]
    if policy["requires_review"]:
        dest: Destination = "human_review"
        reason = "policy_review"
    else:
        dest = "retrieve_docs"
        reason = "policy_allows_auto_answer"

    return Command(
        update={"route_decisions": [{
            "router": "route_from_snapshot.v1",
            "allowed_destinations": ["retrieve_docs", "human_review"],
            "selected_destination": dest,
            "reason_code": reason,
            "evidence_ids": policy["evidence_ids"],
        }]},
        goto=dest,
    )

The second pattern is less magical, which is the point. Replays should reveal what happened, not ask the runtime to remember what the world looked like yesterday.

Observe route drift as a production metric

Streaming and trace data are useful because they make graph execution visible while it runs. But raw traces are not enough. Route decisions should become metrics.

Track at least four signals:

• destination distribution by router version
• reason-code distribution by router version
• human-review rate by customer segment or workflow type
• replay mismatch rate, meaning same input snapshot hash but different destination

The replay mismatch metric is the canary. A mismatch does not always mean a bug. It may mean a policy rollout changed the expected path. But if you cannot distinguish intentional drift from accidental drift, the router contract is incomplete.

Tradeoffs

Decision contracts add state. They also add schema maintenance. That cost is real.

The payoff is that graph behavior becomes debuggable at the branch where incidents usually start. You can answer questions like:

• Did the graph send this case to a human because confidence was low or because policy required it?
• Did a replay follow the original route?
• Which router version introduced the spike in issue creation?
• Did the model change classification, or did the deterministic router change policy?

For senior teams, that separation is the difference between observing an agent and operating one.

Rollout checklist

Before shipping a production graph router, ask these questions:

1. Is the destination set finite and typed?
2. Does the router write a decision record into state?
3. Are reason codes stable and low-cardinality?
4. Does every decision cite evidence already present in state?
5. Are mutable policy inputs snapshotted before routing?
6. Can the router be tested without the full graph runtime?
7. Can operations track route drift by router version?

If the answer is no, the graph may still work. It is just not ready to be trusted under replay, interrupts, retries, and human review.

The decision rule

Graph engineering is not only about nodes, checkpoints, and side effects. It is also about the branch decisions that connect them.

A router is safe when it is boring enough to audit. Give it typed destinations, recorded evidence, stable reason codes, and replay-safe inputs. Then the graph can evolve without turning every incident review into archaeology.

References

1. LangGraph Graph API overview
2. LangGraph Persistence
3. LangGraph Interrupts
4. LangGraph Thinking in LangGraph
5. LangGraph Streaming