A frog mechanic in coveralls and a backwards cap standing beside a hydraulic car press in a scrapyard
blog

Conversation Compaction: Keep Long-Running AI Agents on Track

Shrink the transcript, preserve the work. 🗜️

Agent Workflows
Harness Engineering
A frog mechanic in coveralls and a backwards cap standing beside a hydraulic car press in a scrapyard

A practical guide to conversation compaction for AI agents: trimming, rolling summaries, structured state, retrieval, prompt compression, and the failure modes that quietly break long-running tasks.

11 min read

Start a coding task with an AI agent. Let it inspect the repository, read logs, discuss three approaches, change direction twice, run the tests, and debug the failure.

By turn 40, the transcript contains everything — except a clear picture of what matters now.

Conversation compaction fixes that. It replaces old, noisy history with a smaller representation that preserves the state required to continue the task: goals, decisions, constraints, facts, artifacts, and open work.

This guide explains the main compaction techniques, where each one fails, and a practical architecture you can use for long-running agents.


Compaction Is Not Just Summarization

A conversation is an append-only event log. The model receives some portion of that log on every turn. As it grows, three things happen:

  • Cost rises because more input tokens are processed.
  • Latency rises because the model has more context to ingest.
  • Signal density falls because resolved discussions compete with current instructions.

Remember! 💡 Compaction is a state-management operation.

💡 Pro tip: Optimize for recoverability, not compression ratio. If the agent can explain the current goal, the decisions already made, and the next step after compaction, the result is useful.


Five Techniques You Can Combine

TechniqueWhat it doesBest forMain risk
Sliding windowKeeps only the newest messagesStateless chat, simple assistantsSilently loses old requirements
Rolling summaryReplaces the whole conversation with a summary (It can be a good idea to keep latest turns too)General conversationsSummary drift
Structured stateExtracts facts into a fixed schemaAgents and workflowsSchema misses unexpected details
Retrieval-backed memoryStores history externally and recalls relevant partsLarge histories and researchRetrieval misses or returns noise
Prompt compressionRemoves low-information text or tokensLarge documents and tool outputDamaged code, JSON, or meaning

These are layers, not competitors. A robust system usually combines at least three.

1. Sliding window

Keep the system instructions and the most recent N messages. Drop everything in the middle.

This is cheap and deterministic. It is also brutally lossy. LangChain’s own trimming example shows the failure clearly: once the message containing a user’s name is removed, the agent can no longer answer a later question about it.

Use a sliding window for freshness, not memory.

2. Rolling summary

When the conversation crosses a token threshold, summarize the oldest section and keep the latest turns verbatim (Optional):

[system instructions]
[summary of turns 1–36]
[turns 37–44 verbatim]
[new user message]

This preserves narrative continuity at a fraction of the size. The weakness is recursive loss: if you repeatedly summarize a summary, small errors become durable facts.

3. Structured state

For task-oriented agents, prose is not enough. Extract the state into fields with explicit semantics:

{
  "goal": "Add conversation compaction to the support agent",
  "constraints": ["Keep the last 12 messages verbatim"],
  "decisions": [
    {
      "decision": "Use a structured summary plus retrieval",
      "reason": "We need both continuity and source recovery"
    }
  ],
  "artifacts": ["src/agent/memory.ts"],
  "openQuestions": ["Which embedding model should index old turns?"],
  "nextActions": ["Add compaction evaluation cases"]
}

Now you can validate the output, merge fields deliberately, and render only the sections needed by the next call.

⚠️ Warning: Never let the summary overwrite higher-priority instructions. System rules, security boundaries, and user constraints should remain authoritative inputs, not mutable memories.

4. Retrieval-backed memory

Do not force every tool result, document, and decision into the active prompt. Store the original content outside the conversation, then retrieve relevant fragments when needed.

A useful split is:

  • Working memory: recent turns and current task state.
  • Episodic memory: previous actions and their outcomes.
  • Semantic memory: stable facts, preferences, and domain knowledge.
  • Artifact store: files, logs, diffs, URLs, and full tool outputs.

The compacted state should point to evidence rather than paraphrase all of it. Tests failed in run test-184 is cheaper and safer than embedding 600 lines of output, as long as the agent can retrieve test-184 later.

5. Prompt compression

Prompt compression removes redundant sentences or low-information tokens before the context reaches the main model. Microsoft’s LLMLingua research reported up to 20× compression with little performance loss in its evaluated tasks. LongLLMLingua made compression question-aware and reported 1.4×–2.6× end-to-end acceleration for roughly 10K-token prompts compressed by 2×–6×.

Those results are promising, but prompt compression solves a different problem from conversation state. It can reduce bulky natural language. It should not be trusted to rewrite source code, signed reasoning blocks, exact quotations, tool-call arguments, or machine-readable data.


A Practical Compaction Architecture

The diagram below shows how each technique transforms growing conversation history into a compact working set. Trigger compaction before the context is full — a threshold around 60–75% of the usable input budget is a sensible starting point.

flowchart LR
    T(["Token\nthreshold\nreached"])

    T --> SW
    T --> RS
    T --> SS
    T --> RM
    T --> PC

    subgraph SW ["① Sliding Window"]
        SW1["Full history"] -->|"Drop old"| SW2["Latest N\nmessages"]
    end

    subgraph RS ["② Rolling Summary"]
        RS1["Old turns"] -->|"Summarize"| RS2["Prose summary\n+ recent tail"]
    end

    subgraph SS ["③ Structured State"]
        SS1["Conversation"] -->|"Extract"| SS2["{goal · constraints\ndecisions · openWork}"]
    end

    subgraph RM ["④ Retrieval Memory"]
        RM1["History +\ntool output"] -->|"Archive + embed"| RM2["Working memory\n+ evidence pointers"]
    end

    subgraph PC ["⑤ Prompt Compression"]
        PC1["Verbose text"] -->|"Compress"| PC2["Dense text\n~20× smaller"]
    end

If you use OpenAI’s Responses API, the platform now supports server-side and standalone compaction. Server-side compaction can run automatically after a configured token threshold and returns an encrypted compaction item that can be passed into later requests. That removes some orchestration work, but you should still evaluate whether your application-specific constraints survive the process.


Sources and Further Reading

Link copied to clipboard