Graph State Needs Garbage Collection

Graph state needs garbage collection because every checkpointed field eventually becomes a replay, cost, privacy, or migration surface. The production rule is simple: keep raw state durable, keep prompts derived, and make every growing collection declare retention, compaction, deletion, and audit semantics before it reaches production.

The hidden production problem: state never shrinks by accident

LangGraph makes state explicit: a workflow is built from shared State, executable Nodes, and routing Edges that move execution forward based on current state (LangGraph Graph API). That explicitness is powerful because nodes can return state updates and reducers decide how those updates merge into the accumulated state (LangGraph Graph API). It also creates a failure mode that prompt-only agent designs can hide: once a graph field grows, the runtime will keep carrying it until some contract says otherwise.

In toy demos, a growing messages list looks like context. In production, it is more than context. It is checkpoint data, replay input, debugging material, privacy scope, migration surface, and sometimes the payload a future node uses to call a tool. LangGraph persistence is designed to keep useful information beyond one run through checkpointers and stores (LangGraph Persistence). If the graph can resume after failure, fork for time travel, or pause for human review, then stale state can resume too.

The engineering mistake is treating state cleanup as an LLM context-window trick. Trimming the prompt before a model call helps cost and latency, but it does not automatically define what should remain durable, what should be summarized, what should be deleted, or what should move to long-term memory. Long conversations can overflow context windows, and even long-context models can become distracted by stale or off-topic content while latency and cost rise (LangChain Short-term Memory). The graph problem is broader: stale data can influence non-LLM nodes, tests, routers, replays, and audits.

The memorable rule is: checkpoint the current job, not the whole attic.

State retention is a graph contract, not a prompt trick

LangGraph separates short-term and long-term persistence. Checkpointers persist a thread's graph state as checkpoints for conversation continuity, human-in-the-loop workflows, time travel, and fault tolerance (LangGraph Persistence). Stores persist application-defined data outside graph state for cross-thread memory such as user preferences, facts, and shared knowledge (LangGraph Persistence). That separation is the first retention boundary.

A graph should not use checkpointed state as the universal warehouse. Thread state is for the current execution context. Store data is for durable memory that must outlive one thread. Derived prompt text is for one node invocation. Logs and traces are for observation. When these roles collapse into one giant state object, cleanup becomes politically hard because every downstream node might depend on accidental history.

The design question for every field is not only, "Does this help the next LLM call?" The stronger question is, "Who owns this field's lifecycle?" LangGraph's own design guidance asks whether data needs to persist across steps, and it recommends deriving data when possible instead of storing it (Thinking in LangGraph). It also recommends storing raw data rather than formatted prompts so nodes can format context on demand and state schemas can evolve without prompt-template churn (Thinking in LangGraph).

That guidance turns garbage collection into a contract. A field that stores raw source facts might be compacted into stable citations. A field that stores retrieved chunks might be evicted after a final answer is drafted. A field that stores tool receipts might be retained longer than chat messages because replay safety depends on it. A field that stores prompts should probably not exist at all.

Classify every field before it grows

I use four field classes when reviewing production graphs.

Field class	Examples	Retention default	Failure if wrong
cursor	current step, selected route, pending approval id	keep until thread completes	resume lands in the wrong phase
evidence	retrieved document ids, source snippets, tool readbacks	compact after decision	stale facts keep influencing later nodes
effect_receipt	external write id, idempotency key, payment or ticket receipt	keep as audit data	replay repeats an external side effect
conversation	user and assistant messages	trim, summarize, or delete under policy	context cost grows and old intent distracts the model

This classification fits LangGraph's mechanics. State schemas define what nodes and edges can read, and reducers define how node updates are applied (LangGraph Graph API). A reducer is not just a merge helper. It is the first place to encode whether a field appends forever, overwrites with the latest value, merges by id, or rejects growth that lacks a retention decision.

Message history deserves special handling because it is the easiest field to let grow forever. LangGraph provides a prebuilt MessagesState with a messages key that uses the add_messages reducer (LangGraph Graph API). The memory documentation describes trimming messages before a model call, deleting messages from state permanently, summarizing earlier messages, managing checkpoints, and custom filtering (LangGraph Memory). Deleting specific messages uses RemoveMessage with a state key that uses the add_messages reducer, such as MessagesState (LangGraph Memory).

The production decision is not "trim or summarize." The production decision is which lifecycle each message has. A user instruction that defines task scope may need a summary receipt. A tool output that contains a temporary signed URL may need immediate deletion after extraction. A model draft may be safe to discard after review. A human approval decision may need a durable audit record.

A garbage-collection contract for graph state

For each growing key, I want a short contract that can be reviewed with the graph schema:

1. Owner: which node or subgraph owns this key's lifecycle.
2. Growth mode: append-only, merge-by-id, overwrite, bounded list, or external store pointer.
3. Retention trigger: max tokens, max items, phase boundary, approval completion, external receipt, or age.
4. Compaction behavior: summarize, extract facts, keep ids only, move to store, or delete.
5. Audit evidence: how an operator proves data was retained or removed under policy.
6. Replay rule: what happens if the node reruns after compaction.

This contract is small enough to fit beside a TypedDict or Pydantic schema. It also forces teams to separate state retention from model prompting. Prompt formatting should happen inside nodes, while state stores raw data (Thinking in LangGraph). If a prompt template is saved in checkpointed state, every future replay inherits a formatted artifact that should have been derived.

The replay rule matters because LangGraph persistence makes checkpointed state operationally meaningful. Checkpointers persist a thread's graph state, and stores persist data outside graph state for cross-thread use (LangGraph Persistence). If a garbage collector summarizes messages, the replay path must know whether to use the summary, reconstruct context from evidence ids, or stop with a migration error. Silent context loss is worse than a loud resume failure.

A good retention contract also names what does not belong in graph state. Database connections, temporary caches, formatted prompt strings, duplicate rendered markdown, and external API payloads that can be re-read by id are common offenders. Store stable ids, state-changing receipts, and raw facts. Derive everything else at the edge of the node that needs it.

Minimal retention wrapper and contract test

Here is a small stdlib-only test sketch. It does not depend on LangGraph. The goal is to make the retention policy executable before it is wired into a real reducer or cleanup node.

from dataclasses import dataclass
from typing import Literal

GrowthMode = Literal["append", "merge_by_id", "overwrite", "bounded"]
Action = Literal["keep", "summarize", "delete", "ids_only"]

@dataclass(frozen=True)
class RetentionPolicy:
    key: str
    owner: str
    growth_mode: GrowthMode
    max_items: int
    compact_to: Action
    audit_key: str
    replay_rule: str


def apply_policy(key: str, values: list[dict], policy: RetentionPolicy) -> tuple[list[dict], dict]:
    if key != policy.key:
        raise ValueError("policy key mismatch")
    if policy.growth_mode not in {"append", "merge_by_id", "overwrite", "bounded"}:
        raise ValueError("unknown growth mode")
    if len(values) <= policy.max_items:
        return values, {"key": key, "action": "keep", "removed": 0}

    overflow = len(values) - policy.max_items
    kept = values[overflow:]
    receipt = {
        "key": key,
        "action": policy.compact_to,
        "removed": overflow,
        "owner": policy.owner,
        "audit_key": policy.audit_key,
        "replay_rule": policy.replay_rule,
    }
    return kept, receipt


def test_policy_is_bounded_and_audited() -> None:
    policy = RetentionPolicy(
        key="messages",
        owner="conversation_gc_node",
        growth_mode="bounded",
        max_items=3,
        compact_to="summarize",
        audit_key="state_gc_receipts",
        replay_rule="use_summary_plus_last_messages",
    )
    values = [{"id": str(i), "content": f"message {i}"} for i in range(6)]
    kept, receipt = apply_policy("messages", values, policy)
    assert [item["id"] for item in kept] == ["3", "4", "5"]
    assert receipt["removed"] == 3
    assert receipt["audit_key"] == "state_gc_receipts"
    assert receipt["replay_rule"] == "use_summary_plus_last_messages"


test_policy_is_bounded_and_audited()

In a LangGraph implementation, this policy usually becomes a cleanup node, a reducer guard, or a middleware-like step before the model node. For message history, the actual deletion path should use the documented RemoveMessage mechanism with a state key backed by add_messages, such as MessagesState (LangGraph Memory). For non-message fields, the same contract can produce updated bounded lists, compacted summaries, or store pointers.

The important part is not this exact code. The important part is that state growth becomes testable. If a graph cannot prove which keys grow, when they shrink, and what audit evidence remains, it is not ready for long-lived threads.

Observability: prove the graph is shrinking safely

Garbage collection without observability becomes data loss with nicer naming. LangGraph's streaming API exposes graph execution through modes such as updates, values, messages, custom, checkpoints, tasks, and debug (LangGraph Streaming). Use those projections to make state-retention behavior visible.

The metrics I want are boring and useful:

• state_items_by_key: current item count for each growing key.
• state_bytes_by_key: approximate serialized size for each key.
• gc_actions_total: count by key, action, owner, and reason.
• gc_removed_items_total: number of records removed or summarized.
• replay_after_gc_total: replays that resumed after compaction.
• missing_receipt_total: garbage-collection actions without audit evidence.

The traces should be equally concrete. A cleanup node should emit whether it kept, summarized, deleted, or moved data. A replay should record whether it used full state, summary plus recent messages, evidence ids, or long-term store recall. A migration should fail loudly if it encounters compacted state without a supported replay rule.

This is where checkpoint retention also enters the design. The persistence docs show production Postgres checkpointers and explicitly suggest considering a cron job to delete checkpoints older than N days (LangGraph Persistence). Application-level state garbage collection and checkpoint-store retention are different layers, but they should agree. There is little value in deleting messages from active state while retaining every historical checkpoint forever without policy.

Rollout checklist

Use this checklist before allowing a graph to hold long-lived production threads:

• Every growing state key has an owner and a retention trigger.
• Message history has a trim, delete, summarize, or custom filtering policy grounded in the documented memory patterns (LangGraph Memory).
• Raw data is persisted, while formatted prompts are derived inside nodes (Thinking in LangGraph).
• Short-term thread state and long-term cross-thread store data are separated (LangGraph Persistence).
• Effect receipts outlive conversational clutter.
• Cleanup actions emit audit receipts.
• Replay after cleanup has a tested behavior.
• Checkpoint retention is documented separately from in-state compaction.
• Streaming or trace data can show state size before and after cleanup.

The larger lesson is that graph state is an API between time slices. Today's node writes it, tomorrow's resumed run reads it, next month's migration interprets it, and next quarter's incident review asks why it was still there. Treating garbage collection as part of the graph contract makes that API survivable.

FAQ

Is trimming messages enough for graph-state garbage collection?

No. Trimming helps control what the model sees, and the memory docs describe trimming as one common solution for long conversations (LangGraph Memory). Graph-state garbage collection also covers durable checkpoints, non-message fields, replay behavior, audit receipts, and long-term store boundaries.

Should I summarize or delete old messages?

Use the retention contract to decide. Summarize when downstream nodes need the distilled meaning. Delete when the information is temporary, sensitive, or no longer required. The docs describe both summarizing earlier messages and deleting messages from state permanently as valid memory-management strategies (LangGraph Memory).

What belongs in a LangGraph Store instead of checkpointed state?

Use stores for long-term, cross-thread data such as user preferences, facts, and shared knowledge, and use checkpointers for thread-scoped graph state (LangGraph Persistence). If data must survive across conversations rather than one thread, it is probably store data.

How do I know whether a field should be persisted?

Ask whether the value must persist across steps. LangGraph's design guidance says to include data in state when it needs to persist and derive it when possible instead of storing it (Thinking in LangGraph). That question should be part of schema review, not an afterthought.

References

1. LangGraph Graph API
2. LangGraph Persistence
3. LangGraph Memory
4. Thinking in LangGraph
5. LangGraph Streaming
6. LangChain Short-term Memory