Agent Memory Needs a Write Contract

Most agent memory failures do not start with retrieval. They start earlier, at the moment the system decides that something a user said, a tool returned, or an agent inferred should become durable state.

That decision is easy to hide. A prototype can append every message to a conversation, summarize when the context window gets large, and call the result memory. A more advanced prototype can store extracted facts in a vector database and retrieve them later. Both can feel impressive in a demo.

Production systems need a stricter boundary. Conversation history, retrieved context, user profile facts, tool results, and long-term agent memory are different kinds of state. They have different scopes, owners, expiration rules, privacy risks, correction paths, and failure modes. If they all enter the same bucket, the agent eventually starts remembering things it should have ignored, forgetting things it needed, or treating stale guesses as durable facts.

The fix is not just a better embedding model. Agent memory needs a write contract.

The Engineering Problem

Memory is usually discussed from the read side: how to retrieve the right facts, rank them, compress them, and fit them into the model context. That matters, but it assumes the memory store already contains the right data.

The harder production question is: what is allowed to be written?

A memory write contract answers that question before a fact becomes durable. It defines:

• which inputs are eligible for memory
• who or what is allowed to request a write
• what scope the memory belongs to
• how the system records provenance
• how conflicts, corrections, deletion, and expiration work
• when the memory becomes visible to future runs

Without that contract, memory becomes a side effect of interaction. Any user phrase can become a profile update. Any tool result can become a fact. Any model inference can become future context. The system looks stateful, but it is not governed.

That is where practical failures begin.

Why Saving the Transcript Fails

A transcript is not memory. It is an event log of interaction.

Keeping a transcript is useful for resuming a session, debugging a run, or giving the model local conversational continuity. LangGraph describes this as short-term, thread-scoped memory: state associated with a conversation, persisted through checkpointers so the thread can be resumed. The OpenAI Agents SDK similarly provides sessions that store conversation history across multiple agent runs, so application code does not need to manually rebuild the full input list every turn.

That is session continuity, not durable knowledge.

The transcript has three production problems when it is treated as memory.

First, it is too broad. It contains requests, false starts, intermediate tool outputs, jokes, failed guesses, private data, outdated preferences, and instructions that were relevant only once. Most of that should not be promoted into future context.

Second, it is too local. A single thread can preserve what happened in that thread, but many useful memories need cross-thread scope. A user's stable preference, an organization's integration setting, or an agent's procedural instruction may need to be available in a later session. LangGraph separates this long-term memory from thread state by storing user-specific or application-level data across sessions, organized in stores and namespaces.

Third, it is too expensive to replay. Long conversations can exceed a model's context window, and even when they fit, long context does not guarantee good attention over the right details. If the system depends on replaying everything, it has no control over which facts become operationally important.

A production agent needs both artifacts: an event log for traceability and a memory store for durable, scoped facts. The mistake is pretending they are the same thing.

What the Memory System Actually Writes

A memory write is not one operation. It is a classification and governance decision.

LangGraph's memory documentation uses a useful distinction between semantic, episodic, and procedural memory. Semantic memory stores facts and concepts, such as user or organization preferences. Episodic memory stores past events or actions. Procedural memory stores instructions or learned behavior, such as how the agent should perform a task.

These categories matter because they should not share the same write path.

A semantic preference might be updated from a user's explicit correction: "Use concise summaries for weekly reports." An episodic memory might be derived from a completed workflow: "The October billing report was exported to the finance folder." A procedural memory might require review because it changes how the agent behaves in future tasks.

The storage shape also changes the risk. A profile-style memory keeps a continuously updated JSON document for an entity. It is easy to read, but updates become error-prone as the profile grows. A collection-style memory stores narrower documents, which can reduce information loss, but creates search, deduplication, update, and deletion complexity.

That is the core trade-off: profiles make composition simple and mutation risky; collections make mutation safer and composition harder.

A write contract makes that trade-off explicit.

A Memory Write Contract

A useful contract is small enough to enforce on every write and strict enough to reject ambiguous state.

MemoryWrite {
  candidate: extracted claim or event
  source: user | tool | system | human_review
  evidence: message_id, tool_call_id, document_uri, or trace_id
  memory_type: semantic | episodic | procedural
  scope: thread | user | org | app | agent
  visibility: immediate | after_review | next_session
  confidence: asserted | inferred | observed | corrected
  ttl: duration or none
  conflict_policy: reject | supersede | append | require_review
  deletion_key: stable key used for correction and erasure
}

The exact schema can vary, but the contract should force five decisions.

First, source. A fact directly asserted by a user is not the same as a model inference. A tool result is not the same as a preference. If the memory store cannot distinguish them, the agent will eventually cite its own guesses as if they came from the user.

Second, scope. Thread memory, user memory, organization memory, and application memory should have separate namespaces. LangGraph stores long-term memories as JSON documents under custom namespaces, often including user or organization identifiers. That is not just an implementation detail. It is how a production system prevents one tenant, project, or session from contaminating another.

Third, visibility. Some writes should be visible immediately. Others should be created in the background and only become available after deduplication, validation, or review. A user correction may need immediate effect. A model-summarized preference may need a delayed path.

Fourth, repair. Every memory needs a stable deletion or correction key. If a user says, "Actually, that preference changed," the system should update the previous memory, not add a contradictory new one and hope retrieval ranks the latest version first.

Fifth, provenance. The memory should point back to the event, message, document, or tool call that justified the write. Provenance is how operators debug wrong memory, how users get explanations, and how deletion requests can be honored.

The contract turns memory from a blob of remembered text into a controlled state transition.

Hot-Path Versus Background Writes

LangGraph identifies two broad write paths: creating memories in the hot path during runtime, or creating them in the background.

Hot-path writes are attractive because they make new memories available immediately. If a user corrects the agent's name pronunciation or changes a durable preference, the next response should respect that. Hot-path writes can also be transparent: the UI can say, "I will remember that."

The cost is latency and cognitive load. The agent now has to answer the user while deciding what to commit to memory. If memory writing is exposed as a tool, the model may overuse it, underuse it, or use it inconsistently. The main task and the memory task compete for attention.

Background writes avoid that problem. A separate worker can inspect completed turns, extract candidates, deduplicate them, validate schemas, and write only stable memories. This keeps the primary interaction fast and separates application logic from memory management.

The cost is freshness. If the background job runs too rarely, other threads may not see new context. If it runs too aggressively, it can write redundant or low-quality memories before the interaction is settled. Background memory also needs a clear trigger: after every turn, after a session ends, on a schedule, after explicit user consent, or only when the application marks an event as durable.

The production answer is usually both.

Use the hot path for explicit, high-confidence corrections and preferences that must affect the current interaction. Use the background path for summarization, deduplication, profile maintenance, and cross-session memory formation. Both paths should enforce the same write contract.

flowchart TD
  A[Conversation or tool event] --> B{Eligible for memory?}
  B -- no --> L[Keep in event log only]
  B -- yes --> C[Build MemoryWrite candidate]
  C --> D{Hot path needed?}
  D -- yes --> E[Validate scope and consent]
  E --> F[Write with provenance]
  D -- no --> G[Queue background review]
  G --> H[Deduplicate and resolve conflicts]
  H --> F
  F --> I[Future retrieval]
  F --> J[Correction and deletion index]

The diagram's important detail is not the queue. It is the rejection branch. Most interaction data should remain in the event log only.

Failure Modes and Trade-offs

A memory system without a write contract usually fails in predictable ways.

The first failure is over-remembering. The agent stores casual statements as durable preferences. A user says, "Make this one shorter," and every future report becomes terse. The system confused a local instruction with a stable preference.

The second failure is inferred identity. The model turns patterns into facts: the user asked three Kubernetes questions, so the memory says the user is a Kubernetes administrator. That may be useful as a retrieval hint, but it should not be stored as an asserted profile fact without confirmation.

The third failure is conflict accumulation. The user corrects a preference, but the old memory remains. Retrieval now returns both, and the model decides which one to trust. That pushes a database consistency problem into the prompt.

The fourth failure is scope leakage. A memory learned in one workspace appears in another. This can be a privacy bug, a security bug, or simply a source of wrong behavior. Namespaces and tenant-aware session storage are not optional in multi-user systems.

The fifth failure is unreviewed procedural drift. Procedural memories change how the agent behaves. If a model can freely rewrite those rules, the system has created a hidden self-modifying runtime. Procedural memory should usually require stricter review than semantic memory.

The trade-off is that stricter contracts slow down memory formation. Some useful details will be rejected. Some memories will wait for background processing. Some user experiences will feel less magical. That is acceptable. In production, predictable correction beats magical contamination.

Practical Implementation Considerations

Start by separating four stores that prototypes often collapse:

State	Purpose	Typical scope	Write rule
Event log	Debugging and replay	run or thread	append every relevant event
Session state	Multi-turn continuity	thread	checkpoint runtime state
Retrieved context	External knowledge	request	read-only or source-owned
Long-term memory	Durable adaptation	user, org, app, agent	contract-gated writes

This separation aligns with how modern agent frameworks already describe context. The OpenAI Agents SDK context docs note that the LLM only sees data made available through the conversation history, instructions, input, tool results, or retrieval. MCP resources similarly represent application-controlled context that a server exposes for clients to read. Not all useful context should become model-authored memory. Some data should remain external, addressable, and controlled by the system of record.

Then make the write path boring.

Use explicit memory types. Require a scope. Store provenance. Validate JSON. Deduplicate before writing collection memories. Use stable keys for profile fields. Keep correction and deletion operations first-class. Track whether a memory came from direct user assertion, tool observation, model inference, or human review.

For high-risk writes, use admission control:

def admit_memory(candidate, actor, policy):
    if candidate.scope not in policy.allowed_scopes(actor):
        return Reject("scope_not_allowed")
    if candidate.memory_type == "procedural" and not actor.has_review_grant:
        return Review("procedural_memory_requires_review")
    if candidate.confidence == "inferred" and candidate.scope in {"user", "org"}:
        return Review("inferred_profile_fact")
    if not candidate.evidence:
        return Reject("missing_provenance")
    return Accept(candidate.with_deletion_key())

This is not complex infrastructure. It is a gate that prevents the model from using memory as an untyped scratchpad.

What I Would Do in Production

I would treat memory writes like database writes from an unreliable client.

The model can propose a memory candidate. It should not be the final authority on scope, retention, or permission. Application code should enforce the contract, attach provenance, and decide whether the write is immediate, queued, rejected, or sent to review.

For most agents, I would use three lanes.

Lane one is session continuity. Store conversation state with a checkpointer or session backend. Optimize it for resume, interruption recovery, and local continuity.

Lane two is durable user or organization memory. Store narrow semantic facts in namespaced documents. Prefer explicit user assertions and corrections. Treat inferred facts as candidates, not commitments.

Lane three is operational memory. Store episodic events and procedural changes separately. Episodic events should expire or summarize. Procedural changes should have review, tests, or at least a stronger approval rule than ordinary preferences.

Finally, I would expose memory to users and operators. Show what was remembered. Let users correct it. Log why it was written. Make deletion real. A memory system that cannot explain or repair itself eventually becomes a source of silent errors.

Key Takeaway

Agent memory is not the transcript, and it is not just a vector store. It is a governed write path for durable state.

The production question is not "Can the agent remember?" It is "What is allowed to become memory, under which scope, with which evidence, and how can it be corrected later?"

Answer that with a write contract, and memory becomes a reliability feature. Ignore it, and memory becomes another way for the agent to accumulate hidden state it cannot safely operate.

FAQ

Is this only necessary for personal assistants?

No. Personal assistants make the risk obvious, but the same issue appears in support agents, internal copilots, sales assistants, coding agents, and multi-agent workflows. Any system that persists facts across runs needs a write policy.

Should every memory write require user confirmation?

No. Requiring confirmation for everything creates friction and teaches users to ignore prompts. Use confirmation or review for high-impact writes: identity facts, sensitive preferences, cross-tenant state, procedural changes, or inferred claims. Low-risk session continuity can remain automatic.

Is a vector database enough for agent memory?

A vector database can help retrieve memories, but it does not decide what should be remembered, how conflicts are resolved, or when memory should be deleted. Those are contract and lifecycle problems, not embedding problems.

When should memory be written in the background?

Use background writes when the memory can wait for deduplication, summarization, validation, or review. Use hot-path writes when an explicit correction or preference must affect the current or next interaction immediately.

References

1. LangGraph Memory overview
2. LangGraph Persistence
3. OpenAI Agents SDK Sessions
4. OpenAI Agents SDK Context management
5. Model Context Protocol Resources