Context Engineering: Combining Claude-Style Task State Compression with ChatGPT-Style Long-Term Memory

Jul 14 2026 · updated Jul 28 2026 · 16 min · Sieon

Early AI chat products can get surprisingly far by sending the entire conversation history back to the model on every request. If a user only sends a handful of messages, replaying the full chat transcript is simple and usually good enough.

That approach breaks down once the conversation becomes real work.

A long-running chat may grow from 10 messages to 100, 300, or 1,000 messages. If the product keeps sending every previous message into every new LLM call, the system starts paying a compounding tax.

More input tokens
→ higher TTFT
→ slower responses
→ higher cost
→ more chances for the model to miss the important context

The fix is not just “store chat history.” The fix is a Context Engineering layer: a system that decides what context the model should receive for this specific request.

The central idea is context compaction. Instead of either replaying everything or forgetting everything, we preserve the useful meaning of older interactions in a smaller, more structured form.

In practice, there are four useful compaction strategies:

1. Conversation summarization
2. Hierarchical compaction
3. Semantic memory extraction
4. Retrieval + compaction

Each strategy solves a different part of the problem.

Conversation summarization compresses older messages into a rolling summary. It is useful for preserving the current goal, decisions, constraints, and open tasks in a single conversation.

Hierarchical compaction is what you need when one summary is no longer enough. Instead of one flat summary, you maintain chunk summaries, topic summaries, and a global conversation summary. This is useful for long project conversations or multi-day agent sessions.

Semantic memory extraction is different. It does not summarize the conversation. It extracts stable facts that are likely to be useful later: user preferences, project constraints, technology choices, writing style, or repeated decisions.

Retrieval + compaction searches older raw messages only when the current question needs them. The retrieved snippets may then be inserted as raw evidence or compressed again before being added to the final prompt.

A good production system usually combines all four.

This article walks through a hybrid architecture that combines Claude-style task state compression with ChatGPT-style long-term memory.

Claude and ChatGPT emphasize different kinds of context

From a product and agent-runtime perspective, Claude and ChatGPT often feel different in how they preserve context.

The internal implementations are not public, and the behavior is partly model-level and partly product-level. Still, the distinction is useful when designing your own system.

Dimension	Claude-style approach	ChatGPT-style approach
Main strength	Preserving current task state	Preserving user and cross-conversation memory
Primary goal	Continue a long-running task	Personalize future interactions and recall past facts
Typical information	Current goal, decisions, failed attempts, remaining work	User preferences, projects, repeated facts, prior conversations
Scope	Current conversation or work session	User-level memory across conversations
Example	“This refactor already tried approach A and still needs tests.”	“This user prefers Korean coordination and architecture-first explanations.”

Claude Code often feels strong at task continuity. During a coding session, it tends to preserve what files were inspected, what approaches failed, what decisions were made, and what remains to do. That is not just ordinary chat memory. It is compressed task state.

ChatGPT’s memory and past-conversation features feel closer to user continuity. They can preserve preferences, recurring projects, common technology stacks, and style expectations across conversations.

These are not competing ideas. Production AI systems need both.

Claude-style component
= conversation-level task state compression

ChatGPT-style component
= user-level memory and historical retrieval

The goal is to get three properties at the same time:

1. Task continuity
   The assistant can continue the current work without losing decisions or constraints.

2. User continuity
   The assistant can adapt to the user's stable preferences and long-running projects.

3. Exact historical recall
   The assistant can retrieve what was actually said when exact past details matter.

The high-level architecture

The Context Builder sits between storage and the LLM.

User question
    ↓
Context Builder
    ├─ 1. Conversation Summary
    ├─ 2. Recent Raw Messages
    ├─ 3. Relevant Old History
    ├─ 4. User Long-term Memory
    ├─ 5. Current Question
    └─ 6. Token Budgeting
            ↓
        Final Context
            ↓
            LLM
            ↓
       Save assistant response

The Context Builder does not own every piece of data. It reads from several stores and assembles the final model input.

Context Builder
    │
    ├─ Load conversation summary
    ├─ Load recent messages
    ├─ Load user memory
    ├─ Decide whether retrieval is needed
    ├─ Search older messages
    ├─ Deduplicate overlapping context
    ├─ Rerank by importance
    └─ Apply token budget

This prevents the system from replaying the entire conversation every time. Older context is compacted, recent context stays raw, and exact old history is retrieved only when needed.

1. Conversation Summary

A Conversation Summary is the compressed task state of the current conversation.

Imagine a chat with 100 messages. Instead of sending all 100 messages back to the model, the system can summarize the older portion like this:

Current goal:
- Reduce chat latency caused by replaying full history.

Confirmed findings:
- Every new request currently includes the full previous conversation.
- Input tokens and TTFT increase as the conversation grows.

Decisions:
- Keep all raw messages in the database.
- Use a token-based recent window.
- Compact older messages using threshold-based rolling summaries.

Open questions:
- Whether historical retrieval should use pgvector.
- Whether a separate cache layer is necessary at the initial stage.

This is not a generic summary. It is a structured working state.

A useful conversation summary should preserve:

current goal
confirmed facts
decisions
constraints
failed attempts
open questions
next steps

This is the Claude-style part of the architecture. The system is not preserving every sentence. It is preserving the information required to continue the work.

2. Recent Raw Messages

Recent messages should usually remain raw.

The reason is simple: recent messages contain references that are hard to resolve from summaries alone.

User: Where would the separate vector database fit?
Assistant: For the first version, Postgres + pgvector may be enough...
User: So we do not need a separate vector DB from day one?
Assistant: Right...
User: But Claude seems to behave differently from GPT.

Recent conversation often includes expressions like:

that
second option
the thing we just discussed
do it that way
no, I meant the other one

A summary may lose the exact referent. Raw recent messages preserve it.

Use a token budget rather than a fixed message count.

Recent messages budget: 4,000 to 8,000 tokens

A token-based window handles both short casual messages and long code-heavy turns more reliably than “last 20 messages.”

3. Relevant Old History

Relevant Old History is used when the user asks for something specific from the past.

For example:

What exact function did we identify as the cause of the CPU spike last time?

The conversation summary might only say:

Full-history replay was identified as a latency source.

But the user is asking for an exact function name or code path. A summary is not enough. The Context Builder must search older raw messages.

Current question
    ↓
Historical retrieval
    ↓
Relevant message groups

The system should usually retrieve message groups, not isolated messages.

matched message: 152
included context: messages 150–154

This preserves the surrounding question and answer.

A useful rule is:

Summary
= what happened

Retrieval
= what was said exactly

4. User Long-term Memory

User Long-term Memory is different from Conversation Summary.

A Conversation Summary belongs to one conversation.

conversation_id = 123

User Long-term Memory belongs to the user across conversations.

user_id = 456

Examples:

- User prefers Korean coordination with English technical terms.
- User works on AI agent and AI infrastructure projects.
- User commonly uses Postgres and pgvector for early-stage AI infrastructure.
- User prefers architecture-first implementation plans.

These categories have different scopes:

Context type	Scope	Example
Conversation Summary	Current conversation	A separate vector DB was deferred for the MVP.
User Memory	User-level	The user prefers architecture-first explanations.
Retrieved History	Specific past conversation	The exact compaction threshold discussed earlier.
Recent Messages	Immediate local context	What “that approach” refers to in the last few turns.

Long-term memory improves personalization, but it is not required to fix the first latency problem. If the immediate problem is full-history replay, the first priority is:

1. Conversation Summary
2. Recent Raw Messages
3. Optional Retrieved History
4. User Long-term Memory

5. Current Question

The current question should be included verbatim.

Current question:
"How does this structure work?"

The current question is the anchor. The Context Builder should use it to decide which context matters.

If the user asks for a general explanation, retrieval may not be needed.

Explain this structure again.

If the user asks for exact past wording, retrieval probably is needed.

Give me the exact email sentence we drafted three weeks ago.

Retrieval should be conditional. Running retrieval on every request adds latency and often inserts irrelevant context.

Final LLM input example

The final prompt may look like this:

SYSTEM
You are an assistant for ...

USER MEMORY
- User works on AI infrastructure.
- User prefers architecture-first explanations.

CONVERSATION SUMMARY
- Full-history replay is causing increasing latency.
- The current proposal is rolling summary + recent window.
- A separate cache layer is deferred until DB reads become a measured bottleneck.

RELEVANT OLD HISTORY
- An earlier discussion identified buildApiMessages as the shared history path.

RECENT CONVERSATION
User: Claude seems to behave differently from GPT.
Assistant: Claude Code appears more task-state-oriented...
User: Then how does this structure work?

CURRENT QUESTION
Explain this structure.

The model receives only the context needed for this turn. It does not receive the entire raw transcript.

Four context compaction strategies

The four compaction strategies are worth separating because they fail in different ways.

1. Conversation summarization

This is the simplest approach. Older messages are compressed into a rolling summary.

Raw messages 1–80
    ↓
Conversation Summary

The benefit is speed of implementation. It quickly reduces full-history replay.

The downside is information loss. If one summary is repeatedly rewritten, details can blur over time. Code names, exact numbers, file paths, and dates may disappear.

Conversation summarization is good for task state. It is not enough for exact recall.

2. Hierarchical compaction

Hierarchical compaction avoids placing all old context into one summary.

Instead, the system summarizes chunks, then summarizes those summaries.

Messages 1–50      → Chunk Summary A
Messages 51–100    → Chunk Summary B
Messages 101–150   → Chunk Summary C

Chunk Summary A+B+C
        ↓
Global Conversation Summary

This is useful for long-running project conversations.

A practical hierarchy might look like this:

message chunk summary
→ topic-level summary
→ conversation-level summary
→ project-level summary

The tradeoff is complexity. You need parent-child relationships, range cursors, invalidation rules, and re-summarization policies.

A good MVP starts with rolling summaries and adds hierarchical compaction once conversations become long enough to justify it.

3. Semantic memory extraction

Semantic memory extraction does not summarize the whole conversation. It extracts stable facts.

For example:

User: For the first version, I want Postgres to handle raw messages, summaries, and retrieval indexes.

This can become a durable memory:

Project constraint:
- The first version should keep raw messages, summaries, and retrieval indexes in Postgres.
- Avoid a separate vector database unless retrieval load justifies it.

Good memory candidates include:

user preference
project fact
technical constraint
business rule
style preference
repeated decision

Not every fact belongs in memory.

Good memory candidates:

User prefers Korean coordination with English technical terms.
The product uses Postgres and pgvector for message storage and retrieval.
The team wants architecture-first implementation plans.

Poor memory candidates:

A test failed three times today.
We assumed an 8,000-token compaction threshold in this draft.
This answer included one comparison table.

Temporary task state belongs in the conversation summary, not long-term memory.

4. Retrieval + compaction

Retrieval + compaction is the most practical hybrid.

The system does not send all old messages. It searches for relevant old message groups, then inserts the useful parts into the final context.

Current question
    ↓
Vector search + keyword search
    ↓
Relevant message groups
    ↓
Optional local compaction
    ↓
Final context

This is useful for questions like:

What API name did we choose last time?
Which function did we identify as the CPU spike source?
Give me the exact sentence from the email draft.

Retrieved content should not be inserted blindly. The Context Builder should:

deduplicate results
include adjacent messages
remove irrelevant parts
locally summarize if needed
fit the final token budget

Retrieval finds the evidence. Compaction makes it fit.

Storage model

A clean storage model separates raw records, summaries, memories, and retrieval indexes.

Database
├─ conversations
├─ conversation_messages
├─ conversation_summaries
├─ user_memories
├─ memory_source_links
└─ message_embeddings

conversation_messages

This table stores the full raw transcript.

all user messages
all assistant responses
tool results
attachment references

Example schema:

conversation_messages
- id
- conversation_id
- sequence_number
- role
- content
- token_count
- metadata
- created_at

This is the source of truth. Summaries reduce prompt size, but raw messages should remain available for retrieval, auditing, and reprocessing.

conversation_summaries

This table stores the rolling conversation summary and the compaction cursor.

conversation_summaries
- conversation_id
- summary
- summarized_until_seq
- version
- updated_at

summarized_until_seq records how far the summary has progressed.

summary covers messages 1–100
summarized_until_seq = 100

This makes the next compaction job deterministic.

user_memories

This table stores stable user-level or project-level facts.

user_memories
- id
- user_id
- memory_type
- content
- confidence
- source_message_id
- status
- created_at
- updated_at

Typical memory types:

preference
project
constraint
stable_fact

User memory can be deferred if the first goal is only to reduce conversation latency.

message_embeddings

Semantic retrieval needs an embedding index.

With Postgres, pgvector can store embeddings directly beside the rest of the data.

message_embeddings
- message_id
- conversation_id
- user_id
- sequence_number
- embedding vector
- created_at

The embedding row should reference the raw message. The raw message remains canonical.

Write path

The request path should save raw data first, then build context.

User message
    ↓
Save to primary database
    ↓
Run Context Builder
    ↓
Call LLM
    ↓
Save assistant response

Background jobs can run separately:

new message saved
    ├─ enqueue embedding job
    ├─ update token counts
    └─ check compaction threshold

Embedding generation and summary compaction should usually be off the critical response path.

Compaction flow

Compaction merges older raw messages into the rolling summary.

If unsummarized messages outside the recent window exceed a threshold, such as 8,000 tokens, the system can trigger compaction.

Existing Summary
+
newly old message range
    ↓
Compaction LLM
    ↓
Updated Summary
    ↓
Optimistic Lock UPDATE

Example state:

Current summary: messages 1–80
Recent window: messages 101–120
New compaction target: messages 81–100

Compaction input:

Summary of 1–80
+
Raw messages 81–100

Compaction output:

Summary of 1–100

Then the cursor moves:

summarized_until_seq = 100

Use optimistic locking to avoid concurrent workers overwriting each other.

UPDATE conversation_summaries
SET summary = ?,
    summarized_until_seq = ?,
    version = version + 1
WHERE conversation_id = ?
  AND version = ?;

Retrieval flow

Retrieval is not needed for every request.

It should run when the current question asks for exact old details, prior decisions, specific code, filenames, dates, or earlier wording.

Current Question
    ↓
Decide whether retrieval is needed
    ↓
Generate query embedding
    ↓
Vector search
    +
Keyword search
    ↓
conversation/user filter
    ↓
top relevant message groups
    ↓
Context Builder

Semantic search alone is not enough. Keyword search is better for exact artifacts:

error code
file name
function name
port number
email address
specific date
exact phrase

A hybrid retrieval system is better:

Retrieval
├─ Semantic search
└─ Lexical search

The result should be a message group, not just one message.

matched message: 152
included context: messages 150–154

Context Builder logic

Not every step belongs in the database.

Several responsibilities are application logic inside the Context Builder.

Deciding whether retrieval is needed

def should_retrieve(question, recent_messages):
    if has_explicit_past_reference(question):
        return True

    if asks_for_exact_previous_detail(question):
        return True

    if topic_shift_detected(question, recent_messages):
        return True

    return False

The recent messages come from storage, but the decision is application logic.

Deduplication

The same information may appear in the summary, recent messages, and retrieved history.

Summary:
pgvector was discussed.

Retrieved message:
pgvector was discussed.

Recent message:
pgvector was discussed.

Deduplicate using message IDs, source ranges, and normalized text.

Reranking

Retrieved context can be scored with several signals:

semantic similarity
keyword match
recency
same conversation
explicit user reference
source reliability

Example:

final_score =
    0.45 * semantic_score
  + 0.25 * keyword_score
  + 0.15 * recency_score
  + 0.15 * reference_score

The goal is not to insert everything found. The goal is to insert the most useful context for the current question.

Token budgeting

The final context should reserve room for the model response.

Example budget:

System Prompt       2,000
Summary             2,000
User Memory         1,000
Retrieved History   3,000
Recent Messages     6,000
Current Question    variable
Response Reserve    4,000

Default priority:

1. Current Question
2. Recent Raw Messages
3. Conversation Summary
4. Retrieved History
5. User Memory

If the user asks for an exact past detail, retrieved history may temporarily move higher.

Example Architecture: Postgres + pgvector

For a clean MVP, Postgres is a strong default example.

It supports raw message storage, rolling summaries, user memory, semantic retrieval, keyword retrieval, and token-aware context assembly without introducing several systems at once.

Postgres
├── conversations
├── conversation_messages
├── conversation_summaries
├── user_memories
├── memory_sources
├── message_embeddings       -- pgvector
└── full-text / trigram index -- keyword retrieval

Context Builder
= reads from Postgres and assembles the final LLM context

Postgres is not always the fastest possible vector system. The point is that early context engineering usually benefits more from correctness, simplicity, and operational clarity than from splitting every concern into a separate database.

With Postgres, the role split is straightforward.

Context Builder
    │
    ├─ Load summary
    │      → Postgres: conversation_summaries
    │
    ├─ Load recent messages
    │      → Postgres: conversation_messages
    │
    ├─ Load user memory
    │      → Postgres: user_memories
    │
    ├─ Decide whether retrieval is needed
    │      → Context Builder logic
    │
    ├─ Search older messages
    │      → Postgres + pgvector
    │      → Postgres Full-Text Search / pg_trgm
    │
    ├─ Deduplicate and rerank
    │      → Context Builder logic
    │
    └─ Apply token budget
           → Context Builder logic

Postgres as the source of truth

Postgres stores durable conversation state.

Postgres
├── conversations
├── conversation_messages
├── conversation_summaries
├── user_memories
├── memory_sources
└── message_embeddings

The key design rule is simple:

Postgres = source of truth
Context Builder = context assembly logic
LLM = reasoning and response generation

pgvector for semantic retrieval

Semantic retrieval can start with pgvector inside Postgres.

message_embeddings
- message_id
- conversation_id
- user_id
- sequence_number
- embedding vector
- created_at

Retrieval flow:

Current question
    ↓
Generate embedding
    ↓
pgvector similarity search
    ↓
Relevant message IDs
    ↓
Fetch raw messages and adjacent context from conversation_messages

Full-text search for exact terms

Vector search is not enough by itself.

Exact terms often matter more than semantic similarity:

error code
file name
function name
port number
email address
specific date
exact API name

Postgres Full-Text Search and pg_trgm can cover many of these cases without adding another system.

Retrieval
├── Semantic search: pgvector
└── Lexical search: Postgres FTS / pg_trgm

Recommended final structure

A simple production-ready version can look like this.

┌──────────────────────────────────────┐
│              Postgres                │
│                                      │
│ raw messages                         │
│ conversation summaries               │
│ user memories                        │
│ pgvector message embeddings          │
│ full-text / trigram indexes          │
└──────────────────┬───────────────────┘
                   │
                   ▼
┌──────────────────────────────────────┐
│           Context Builder            │
│                                      │
│ load summary                         │
│ load recent messages                 │
│ load user memory                     │
│ decide whether retrieval is needed   │
│ hybrid search                        │
│ deduplicate                          │
│ rerank                               │
│ apply token budget                   │
└──────────────────┬───────────────────┘
                   │
                   ▼
                  LLM

Request flow

1. Save the user message to Postgres
2. Run the Context Builder
3. Load summary and recent messages from Postgres
4. Decide whether historical retrieval is needed
5. If needed, run pgvector + keyword retrieval
6. Fetch raw message groups from conversation_messages
7. Deduplicate and apply token budget
8. Call the LLM
9. Save the assistant response to Postgres
10. Run embedding and compaction jobs asynchronously

The biggest win is still reducing LLM input size.

Postgres reads         40ms
Hybrid retrieval       80ms
LLM input processing    4,000ms

If the prompt is too large, changing databases will not fix the main problem. Reducing the input from 80,000 tokens to 10,000 tokens will.

MVP implementation order

The first version does not need every feature.

Postgres
├── raw messages
├── rolling conversation summary
└── optional pgvector embeddings

Context Builder
├── summary
├── recent messages by token budget
├── optional retrieved history
└── current question

Phase 1 removes full-history replay.

Raw messages in Postgres
+ Rolling Conversation Summary
+ Token-based Recent Window
+ Current Question

Phase 2 adds conditional retrieval.

Conditional Historical Retrieval
+ pgvector semantic search
+ Postgres keyword search
+ raw message group fetch

Phase 3 adds long-term memory.

User Long-term Memory
+ cross-conversation personalization

This gives a clean path from a simple MVP to a more capable memory system without introducing unnecessary infrastructure too early.

Final take

Context Engineering is not about storing more data. It is about selecting the right context for the current LLM call.

Claude-style task state compression keeps the current conversation moving. ChatGPT-style long-term memory and historical retrieval preserve user continuity and exact recall.

The combined architecture looks like this:

Conversation Summary
+ Recent Raw Messages
+ Optional Retrieved History
+ User Long-term Memory
+ Current Question
+ Token Budgeting

Compaction is not a single technique.

Conversation summarization
= compress current task state

Hierarchical compaction
= compress long conversations across multiple levels

Semantic memory extraction
= extract stable facts into memory

Retrieval + compaction
= find relevant old context and compress it into the current prompt

For an MVP, Postgres + pgvector is enough to demonstrate the architecture cleanly.

Postgres
= raw messages, summaries, user memories, pgvector embeddings, keyword indexes

Context Builder
= retrieval decisions, deduplication, reranking, and token budgeting

The first problem to solve is not storage migration. It is full-history replay.

A good AI chat architecture does not resend the entire conversation forever. It combines summaries, a recent-message window, retrieval, memory, and token budgeting to assemble only the context that matters for the next response.