feat(ai-compaction): pluggable context-window compaction

.changeset/ai-compaction.md

@@ -0,0 +1,13 @@
+---
+'@tanstack/ai-compaction': minor
+---
+
+Add `@tanstack/ai-compaction` — context-window compaction as a `chat()`
+middleware. `withCompaction({ maxTokens, strategy })` runs a pluggable
+`CompactionStrategy` before each model call, so compaction is incremental and
+rolling. Three strategies ship built in: `evictOldest` (drop old messages, the
+default), `summarizeOldest` (replace them with an LLM summary), and
+`clearToolResults` (stub old tool output, keep the messages). Combine them with
+`composeStrategies`, which escalates through strategies until the transcript is
+back under budget. Strategies preserve tool-call/result pairing and never touch
+the system prompt.

.changeset/compaction-persistence-integration.md

@@ -0,0 +1,8 @@
+---
+'@tanstack/ai': minor
+'@tanstack/ai-compaction': minor
+'@tanstack/ai-persistence': patch
+---
+
+Keep canonical chat history separate from compacted provider context. Reuse
+validated compaction checkpoints through an optional persistence metadata store.

docs/advanced/compaction.md

@@ -0,0 +1,220 @@
+---
+title: Compaction
+id: compaction
+order: 3
+description: "Keep long chats under the context limit with @tanstack/ai-compaction. withCompaction runs a pluggable strategy before each model call: evict, summarize, or clear old tool output."
+keywords:
+  - tanstack ai
+  - compaction
+  - context window
+  - middleware
+  - token limit
+  - summarize history
+---
+
+A long chat or a multi-step agent loop keeps adding messages. At some point the transcript passes the model's context limit and the call fails. You want the conversation to keep working without hitting that wall.
+
+`withCompaction` shrinks provider context before each model call. When the context passes `maxTokens`, a **strategy** rewrites what the model sees. The canonical transcript does not change. Add this [`ChatMiddleware`](./middleware) to the `middleware` array of any `chat()` call.
+
+## Install
+
+```bash
+pnpm add @tanstack/ai-compaction
+```
+
+## Quick start
+
+The default strategy drops the oldest messages once the transcript passes `maxTokens` and keeps the recent ones.
+
+```typescript
+import { chat, toServerSentEventsResponse } from "@tanstack/ai";
+import { openaiText } from "@tanstack/ai-openai";
+import { withCompaction } from "@tanstack/ai-compaction";
+
+export async function POST(request: Request) {
+  const { messages } = await request.json();
+
+  const stream = chat({
+    adapter: openaiText("gpt-5.5"),
+    messages,
+    middleware: [withCompaction({ maxTokens: 100_000 })],
+  });
+
+  return toServerSentEventsResponse(stream);
+}
+```
+
+## Pick a strategy
+
+Pass `strategy` to change how the history shrinks. Three are built in.
+
+| Strategy | What it does | Cost |
+|----------|--------------|------|
+| `evictOldest` (default) | Drop the oldest messages, leave a marker | No extra model call |
+| `summarizeOldest` | Replace the oldest messages with an LLM summary | One summarize call |
+| `clearToolResults` | Stub the content of old tool results, keep the messages | No extra model call |
+
+### evictOldest
+
+Cheapest. Keeps the recent tail, drops the older head, and leaves a short marker in its place. This is the default, so you only name it to tune `keepRecentTokens`.
+
+```typescript
+import { withCompaction, evictOldest } from "@tanstack/ai-compaction";
+
+withCompaction({
+  maxTokens: 100_000,
+  strategy: evictOldest({ keepRecentTokens: 40_000 }),
+});
+```
+
+### summarizeOldest
+
+Keeps the gist of old turns instead of dropping them, at the cost of one summarization call. Pass a `summarize` callback. It gets the messages about to be dropped and returns the summary text. Wire it to `summarize()` or any model call.
+
+```typescript
+import { chat, summarize, toServerSentEventsResponse } from "@tanstack/ai";
+import { openaiText, openaiSummarize } from "@tanstack/ai-openai";
+import { withCompaction, summarizeOldest } from "@tanstack/ai-compaction";
+import type { ModelMessage } from "@tanstack/ai";
+
+async function summarizeHistory(messages: Array<ModelMessage>): Promise<string> {
+  const text = messages
+    .map((m) => `${m.role}: ${typeof m.content === "string" ? m.content : ""}`)
+    .join("\n");
+
+  const { summary } = await summarize({
+    adapter: openaiSummarize("gpt-5.5"),
+    text,
+  });
+  return summary;
+}
+
+export async function POST(request: Request) {
+  const { messages } = await request.json();
+
+  const stream = chat({
+    adapter: openaiText("gpt-5.5"),
+    messages,
+    middleware: [
+      withCompaction({
+        maxTokens: 100_000,
+        strategy: summarizeOldest({ summarize: summarizeHistory }),
+      }),
+    ],
+  });
+
+  return toServerSentEventsResponse(stream);
+}
+```
+
+### clearToolResults
+
+Best for agent loops. Tool output (file reads, command output) is usually most of the tokens. This strategy replaces the content of old tool results with a stub and keeps every message and its tool-call pairing in place. The conversation shape does not change.
+
+```typescript
+import { withCompaction, clearToolResults } from "@tanstack/ai-compaction";
+
+withCompaction({
+  maxTokens: 100_000,
+  // Keep the 5 most recent tool results in full, stub the older ones.
+  strategy: clearToolResults({ keepRecentToolResults: 5 }),
+});
+```
+
+### Write your own
+
+A strategy is a function. It gets the messages and the budget, and returns the rewritten messages, or `null` to change nothing. It runs only when the estimate is over `maxTokens`.
+
+```typescript
+import { withCompaction } from "@tanstack/ai-compaction";
+import type { CompactionStrategy } from "@tanstack/ai-compaction";
+
+// Keep only the last message.
+const keepLastOnly: CompactionStrategy = (messages) => {
+  if (messages.length <= 1) return null;
+  return messages.slice(-1);
+};
+
+withCompaction({
+  maxTokens: 100_000,
+  strategy: keepLastOnly,
+  strategyKey: "keep-last-v1",
+});
+```
+
+Set `strategyKey` when you combine a custom strategy with persistence. Change
+the key when the strategy can produce different output. This prevents an old
+checkpoint from using stale behavior.
+
+## Combine strategies
+
+`composeStrategies` runs several strategies in order and **escalates**: it stops as soon as the result is back under `maxTokens`. Put the cheap, targeted strategy first and a broad fallback last. Here it clears old tool output first, and only drops old messages if that was not enough.
+
+```typescript
+import {
+  withCompaction,
+  composeStrategies,
+  clearToolResults,
+  evictOldest,
+} from "@tanstack/ai-compaction";
+
+withCompaction({
+  maxTokens: 100_000,
+  strategy: composeStrategies(clearToolResults(), evictOldest()),
+});
+```
+
+## Options
+
+### withCompaction
+
+| Option | Type | Default | Description |
+|--------|------|---------|-------------|
+| `maxTokens` | `number` | - | **Required.** Compact when the estimated tokens across `messages` pass this. |
+| `strategy` | `CompactionStrategy` | `evictOldest()` | How to shrink the messages. |
+| `estimateTokens` | `(message: ModelMessage) => number` | characters / 4 | Per-message token estimate. Pass a real tokenizer if you need exact counts. |
+| `strategyKey` | `string` | built-in strategy identity | Stable checkpoint identity. Set it for custom strategies, custom estimators, `summarizeOldest`, or a custom eviction marker. |
+| `onCompact` | `(info: CompactionInfo) => void` | - | Runs after each compaction. `info` is `{ before, after, messagesBefore, messagesAfter }` (token and message counts). |
+
+### Strategy options
+
+| Strategy | Options |
+|----------|---------|
+| `evictOldest` | `keepRecentTokens` (default `maxTokens / 2`), `marker` |
+| `summarizeOldest` | `summarize` (**required**), `keepRecentTokens`, `summaryRole` |
+| `clearToolResults` | `keepRecentToolResults` (default `3`), `stub` |
+
+The token count is a rough `characters / 4` estimate. It is good enough to trigger on, not exact. Pass `estimateTokens` for provider-accurate counts.
+
+## What it keeps safe
+
+- **The system prompt is never dropped.** `chat()` keeps it separate from `messages`, so compaction only touches the conversation.
+- **Tool calls stay paired with their results.** The built-in strategies never leave an orphaned tool result, so the request stays valid.
+- **It runs before every model call.** Compaction is incremental: as the chat keeps growing it compacts again, and a later `summarizeOldest` pass folds an earlier summary into the new one.
+- **The canonical transcript stays complete.** Compaction writes provider-only context. Persistence and other middleware still read `ctx.messages`.
+
+## Compaction and persistence
+
+Compaction and server-side [`withPersistence`](../persistence/chat-persistence)
+use two message views:
+
+- `messages` is the complete canonical transcript. Persistence saves this view.
+- `providerMessages` is temporary model context. Compaction rewrites this view.
+
+Middleware order does not change this split. Dropped, summarized, and stubbed
+content remains in the message store.
+
+If the persistence adapter has a `metadata` store, compaction also saves a small
+checkpoint. The next request validates the canonical prefix, restores the last
+compacted result, and adds only new messages. A changed prefix or strategy key
+invalidates the checkpoint.
+
+The default strategy, standard `evictOldest`, `clearToolResults`, and safe
+compositions get a strategy key automatically. Set `strategyKey` for
+`summarizeOldest`, custom strategies, custom estimators, or custom marker
+functions. Without a metadata store or safe key, compaction stays stateless.
+
+## Next steps
+
+- [Middleware](./middleware): the full hook reference and how middleware composes
+- [Built-in Middleware](./built-in-middleware): ready-made middleware that ships in `@tanstack/ai`

docs/advanced/middleware.md

@@ -157,14 +157,19 @@ const dynamicTemperature: ChatMiddleware = {
 
 | Field | Type | Description |
 |-------|------|-------------|
-| `messages` | `ModelMessage[]` | Conversation history |
+| `messages` | `ModelMessage[]` | Canonical conversation history. Persistence and `ctx.messages` use this field. |
+| `providerMessages` | `ModelMessage[]` | Temporary context sent to the provider. Defaults to `messages`. |
 | `systemPrompts` | `string[]` | System prompts |
 | `tools` | `Tool[]` | Available tools |
 | `metadata` | `Record<string, unknown>` | Request metadata |
 | `modelOptions` | `Record<string, unknown>` | Provider-native options — this is where sampling params (`temperature`, `top_p` / `topP`, the provider's `max*Tokens` key) now live, alongside every other model-specific knob. See [Moving Sampling Options into modelOptions](../migration/sampling-options-to-model-options). |
 
 When multiple middleware define `onConfig`, the config is **piped** through them in order — each receives the merged config from the previous middleware.
 
+Return `providerMessages` when a transform must affect only the model call. For
+compatibility, returning `messages` also updates provider input unless the same
+result sets `providerMessages` explicitly.
+
 ### onStructuredOutputConfig
 
 Called once at the start of the final structured-output adapter call — only when `chat()` was invoked with `outputSchema` **and** `supportsCombinedToolsAndSchema()` does not return `true` for the current model/options. Pipes through middleware in order, like `onConfig`, but with access to the **JSON Schema** being sent to the provider. Use this hook when you need to transform the schema (e.g., inject `$defs`, strip vendor-incompatible keywords) or apply structured-output-specific behavior (e.g., suppress system prompts on the final call).
@@ -195,7 +200,8 @@ const injectDefs: ChatMiddleware = {
 
 | Field | Type | Description |
 |-------|------|-------------|
-| `messages` | `ModelMessage[]` | Conversation history sent to the final call |
+| `messages` | `ModelMessage[]` | Canonical conversation history |
+| `providerMessages` | `ModelMessage[]` | Temporary context sent to the final call |
 | `systemPrompts` | `SystemPrompt[]` | System prompts on the final call |
 | `metadata` | `Record<string, unknown>` | Request metadata |
 | `modelOptions` | `Record<string, unknown>` | Provider-native options — this is where sampling params (`temperature`, `top_p` / `topP`, the provider's `max*Tokens` key) now live, alongside every other model-specific knob. See [Moving Sampling Options into modelOptions](../migration/sampling-options-to-model-options). |
@@ -1140,6 +1146,7 @@ import type {
 ## Next Steps
 
 - [Built-in Middleware](./built-in-middleware) — `toolCacheMiddleware`, `contentGuardMiddleware`, `otelMiddleware`
+- [Compaction](./compaction): keep long chats under the context limit with `withCompaction`
 - [OpenTelemetry](./otel) — emit traces and metrics via `otelMiddleware`- [Tools](../tools/tools) — Learn about the isomorphic tool system
 - [Agentic Cycle](../chat/agentic-cycle) — Understand the multi-step agent loop
 - [Streaming](../chat/streaming) — How streaming works in TanStack AI

docs/config.json

@@ -278,7 +278,7 @@
           "label": "Chat Persistence",
           "to": "persistence/chat-persistence",
           "addedAt": "2026-08-04",
-          "updatedAt": "2026-08-19"
+          "updatedAt": "2026-08-26"
         },
         {
           "label": "Client Persistence",
@@ -348,7 +348,7 @@
           "label": "Store Reference",
           "to": "persistence/store-reference",
           "addedAt": "2026-08-04",
-          "updatedAt": "2026-08-14"
+          "updatedAt": "2026-08-26"
         },
         {
           "label": "How Persistence Works",
@@ -537,14 +537,20 @@
           "label": "Middleware",
           "to": "advanced/middleware",
           "addedAt": "2026-04-15",
-          "updatedAt": "2026-08-19"
+          "updatedAt": "2026-08-26"
         },
         {
           "label": "Built-in Middleware",
           "to": "advanced/built-in-middleware",
           "addedAt": "2026-06-03",
           "updatedAt": "2026-07-21"
         },
+        {
+          "label": "Compaction",
+          "to": "advanced/compaction",
+          "addedAt": "2026-08-24",
+          "updatedAt": "2026-08-26"
+        },
         {
           "label": "Locks",
           "to": "advanced/locks",

docs/persistence/chat-persistence.md

@@ -91,6 +91,18 @@ generation hooks. [How persistence works](./internals) has the rest.
   middleware loads the stored transcript and the run picks up from there, so the
   client does not have to re-send history.
 
+## Compaction keeps the transcript complete
+
+Do you add [`withCompaction`](../advanced/compaction) to the same `chat()`? The
+saved thread remains canonical. Compaction changes only the provider context,
+not `ctx.messages`. The message store keeps dropped content, summaries do not
+replace old turns, and cleared tool output remains available for reloads.
+
+If your adapter provides `stores.metadata`, `withPersistence` exposes it to
+other middleware. Compaction uses it automatically for validated checkpoints.
+See
+[Compaction and persistence](../advanced/compaction#compaction-and-persistence).
+
 ## What gets persisted, and when
 
 `withPersistence` writes at **four** moments so a reload never loses a turn:

docs/persistence/store-reference.md

@@ -269,6 +269,12 @@ composite identity. A stored `null` is indistinguishable from absence at the typ
 level, so wrap a value you must persist as `null` (e.g. `{ value: null }`), or
 reject nullish values outright the way the SQLite store above does.
 
+`withPersistence` also provides this store through the core
+`MetadataCapability`. Middleware can use it for derived state without depending
+on `@tanstack/ai-persistence`. For example, `withCompaction` stores validated
+context checkpoints here. Do not place the canonical transcript in metadata;
+the `messages` store owns it.
+
 ## GenerationRunStore
 
 The generation counterpart to `RunStore`. Keyed by its own `runId`, with