diff --git a/.changeset/ai-harness-cli.md b/.changeset/ai-harness-cli.md index 122cb06865..479bf06e29 100644 --- a/.changeset/ai-harness-cli.md +++ b/.changeset/ai-harness-cli.md @@ -4,8 +4,9 @@ Add `@tanstack/ai-harness-cli`: `runCli(harness)` runs a harness from the terminal. -- Line mode reads one message or `/command` per line, answers approvals and questions, hides a secret answer while the user types, and sends `@path` files. Pass `ui` to show your own screen on a session view instead. -- `-p "prompt"` runs one prompt for scripts and CI, and `--output ndjson` prints every AG-UI event. -- `--serve` serves the session protocol over HTTP behind a bearer token. -- `--acp`, `--mcp`, and `--dashboard` load `@tanstack/ai-acp`, `@tanstack/ai-mcp`, and `@tanstack/ai-dashboard` only when you use the flag. A missing package, or one without harness support, fails that flag with a clear message. -- `host` and `principal` options let the CLI use your durable host and run as your user. +- Line mode reads one message or `/command` per line, hides a secret answer while the user types, and sends `@path` files. In a terminal, `/cancel` and a message typed while a turn runs act at once: the message joins the running turn. When a turn waits, each next line answers one wait: `y` or `n` for an approval (`once`, `always`, or `reject` for an approval of `permissions()`), and an answer read with the schema for a question. Pass `ui` to show your own screen on a session view instead. +- `-p "prompt"` runs one prompt for scripts and CI, and `--output ndjson` prints every AG-UI event. `-p` exits with code 3 (`EXIT.needsAction`) at the first wait of the turn: an approval, a question, or a sign-in. +- `--serve` serves the session protocol over HTTP behind a bearer token. It signs media URLs with its own key, never with the token: `--media-key` or `HARNESS_MEDIA_KEY` (at least 32 bytes), or a new random key at each start. Without a set key, signed media URLs stop working after a restart. On a host without `access`, it also passes `openAccess`: the `user` of `runCli`, or `true` without one. So each request with the token runs as the user of the CLI. +- `--acp` and `--mcp` load `@tanstack/ai-acp` and `@tanstack/ai-mcp` only when you use the flag. A missing package, or one without harness support, fails that flag with a clear message. Any other error while the package loads is reported, not hidden. +- In every mode, the first SIGINT or SIGTERM cancels the running turn. A second one, or one while no turn runs, stops the CLI, and `runCli` resolves to 130 (SIGINT) or 143 (SIGTERM). The CLI closes a host that it made with `recoverable: true`, or releases its session handle on your own host and does not wait for a turn that ignores the cancel. The `signals` option takes another signal source, for example an `EventEmitter` in a test. +- `host` and `user` options let the CLI use your durable host and run as your user. Every mode opens its sessions as that user. On a host with `access`, `--serve` reads the user of each request with `getUser`, and `--mcp` does not start, because stdio has no login. diff --git a/.changeset/ai-harness-coding-tools.md b/.changeset/ai-harness-coding-tools.md index 2ed90ac76a..216f8f9008 100644 --- a/.changeset/ai-harness-coding-tools.md +++ b/.changeset/ai-harness-coding-tools.md @@ -4,10 +4,17 @@ Add coding tools in the new Node entry `@tanstack/ai-harness/plugins/coding`: -- `workspaceTools({ root })` gives `read_file` (text pages, images, PDFs), `write_file`, `edit_file` or `patch` (`editStyle`), `list_files`, `grep`, `bash` (with `background`), `webfetch`, and `websearch` (with a `SearchProvider`). With `permissions()`, edits, `bash`, and `webfetch` ask first, and the rules also apply to the real path of a link. `outside: 'ask'` asks before a path outside `root`. The `cwd` thread setting gives each thread its own working folder. -- A `WorkspaceBackend` runs the tools. `hostBackend` is the default. On Windows, commands do not run a program from the working folder by its name. -- `snapshots()` adds `/undo` and `/redo`, which restore only the files of the turn (git 2.26 or newer), file restore for `session.revert()`, and a diff for each model step. -- `formatter()` formats each written file with the formatter of the project. With `permissions()`, each run asks first. +- `workspaceTools({ root })` gives `read_file` (text pages, images, PDFs), `write_file`, `edit_file` or `patch` (`editStyle`), `list_files`, `grep`, `bash` (with `background`), `webfetch`, and `websearch` (with a `SearchProvider`). With `permissions()`, edits, `bash`, and `webfetch` ask first, and the rules also apply to the real path of a link. `outside: 'ask'` makes a path outside `root` need an approval. `permissions()` decides outside paths and the real paths of links before the call runs, so a tool never asks while it runs. The `cwd` thread setting gives each thread its own working folder. +- `workspaceTools({ workspace: { bind, reattach? } })` gives each thread its own workspace. The root of the workspace is the boundary of every tool. The session log keeps where the workspace is (`harness.workspace.bound`). On another host, `reattach` gives the workspace back (for example a resumed sandbox), or `'pinned'` or `'gone'`. A gone workspace sends `tanstack/workspace-tools:stranded`. With this plugin first, `snapshots()` and `formatter()` use the workspace of each thread, and their `root` is optional. +- `createHarnessHost({ ownership: { hostId } })` gives a host a stable id. Then a thread that uses a folder on that host is pinned to it, and other hosts refuse its tool calls. Without `hostId`, threads are not pinned. A host with `hostId` sends a heartbeat to the log store, and its pins lapse when no heartbeat came for `ownership.expiresAfterMs`: then another host reattaches its threads. `host.unpin({ harness, threadId })` unpins a thread from a host that does not come back, once its ownership expired. +- `workspaceRoot(base, ...parts)` gives a folder in `base` for a tenant and a thread id. It encodes each part, so a thread id from a client cannot leave `base` or reach another thread. +- Commands, `rg`, `git`, and the formatters get only a short list of variables of the server process, plus the `env` of the backend call. Git runs without the hooks and the `fsmonitor` of the project. +- The tools resolve links with `realpath` under the file lock, and fail when a backend cannot resolve a path. A write fails when the file changed after the tool read it in the same chat run, so a lead and its subagents do not lose each other's edits. The file locks are shared by the sessions of a host, by the sandbox (or host) and the real path of the file. +- Background jobs: at most 8 per session (`maxJobs`), the newest 1 MiB of output per job, and the events `tanstack/workspace-tools:job-started`, `job-output`, and `job-ended`. A running command that `session.moveToolCallToBackground()` moves counts as a job, and its process id goes to the log. A job that an earlier host left running is stopped at the next tool call, with a note to the model. When the backend cannot check the job (for example with no `ps`), the log keeps it, and the note says that it can still run. The wake turn of a job runs as the user who started it. `session.jobs()` and `jobs()` of a `host.observe` view list the jobs that did not end (`JobInfo`: the job id, the thread, the command or tool name, who moved it, when it started, and `running` on this host). The `bash` spill file has `[redacted]` in place of each secret of the session. +- `read_file` sends images and PDFs up to 256 KB. `webfetch` gives the model the first 256 KB of a page. +- A `WorkspaceBackend` runs the tools. `hostBackend` is the default. `realpath` is required. `spawn` can give a `jobId`, `exec` can give one with `onStart`, and `killJob`, `binding`, and `close` are optional. `killJob` rejects when it cannot check that the process is the job. On Windows, commands do not run a program from the working folder by its name. +- `snapshots()` adds `/undo` and `/redo`, which restore only the files of the turn (git 2.26 or newer), file restore for `session.revert()`, and a diff for each model step. Each thread of each tenant has its own shadow repository, which never stores `.env` files. A step keeps only the files of its own run. The steps and the undo point are in the session log, so another host with the same repository can undo. A revert uses the last 500 steps. `host.sessions.delete` removes the repository of the thread, and `purgeSnapshots()` does it from your own code. A failed snapshot sends `tanstack/snapshots:failed`. +- `formatter()` formats each written file with the formatter of the project. With `permissions()`, the rules decide the formatter with the call that writes, before that call runs: one approval covers both, and the formatter runs after the write. An `always` answer is tied to a hash of the formatter config files. A write of a formatter config below the root, or under `node_modules`, asks each time, and an `always` answer saves no rule for that formatter run. - `WorkspaceHooks` lets other plugins run code after a read or a write. `@vscode/ripgrep` and `turndown` are optional peer dependencies. diff --git a/.changeset/ai-harness-core.md b/.changeset/ai-harness-core.md index 4519636ee5..f546149ff3 100644 --- a/.changeset/ai-harness-core.md +++ b/.changeset/ai-harness-core.md @@ -5,10 +5,51 @@ Add `@tanstack/ai-harness`: build an agent harness on TanStack AI. A harness keeps one conversation open across many turns. - `defineHarness()` names an agent: its adapter, tools, system prompts, middleware, agents, subagents, routing, plugins, turn hooks, and durability options. -- `createHarnessHost()` opens sessions. A session takes `send` (a prompt, a steer into the running turn, a follow-up, or a message to an agent run with `to`), `continue`, `reset`, `resolve`, `cancel`, and `background` inputs. Each returns an operation with a receipt, and the session streams typed events. `host.isOpen(harness, { threadId })` says whether this host holds that session open. -- `definePlugin()` adds tools, prompts, middleware, agents, commands, config options, state, and extension points to a session or to one turn. -- Agents run from code with `session.agents..run()` and `.start()`. In agent code, `ctx.harness` gives `step.do()` (a finished step returns its stored value when the run runs again) and `agents.start()` (a child that counts against the subagent budget). -- With `stores.log` from `@tanstack/ai-persistence`, a session is durable: inputs, tool results, and steps are in the log, and the next host recovers a turn that a crash stopped. `durableTool()` runs side effects once. `durability.truncatedToolResult` sets the text of a tool call cut at the output limit. Without it, a cut answer ends the turn. +- `createHarnessHost()` opens sessions. Each `host.open` gives its own handle to the live session of the thread. `session.close()`, or `await using`, releases the handle. A dropped handle is released by the garbage collector, with one warning. After `close()`, each call on the handle throws. When the session closes under a handle that is not released (its log failed, or a delete ran), the next call on the handle opens it again. +- `lifecycle` on `createHarnessHost()` sets how sessions live. A session that no handle holds closes after it was idle for `closeIdleSessionsAfterMs` (default 60 000). `maxOpenSessions` (default 500) first closes the least recently used idle sessions. A session is not idle while a background job of this host runs. `retention` (`keepSettledOperations`, `keepSettledAgentRuns`, `maxEventBytesInMemory`) bounds the settled operations, agent runs, receipts, events, and log tail that each session keeps in memory. The log keeps the `MESSAGES_SNAPSHOT` that core sends when a run pauses at an interrupt, so a rejoin from the log rebuilds the paused run. `host.close({ recoverable })` stops the work of every session and refuses later opens. With `recoverable: true`, the host gives each log that has work back with a wake, so a worker takes it at once. +- `host.observe(harness, { threadId, user })` reads a thread with no session: its events, transcript, snapshot, media, usage, and entry. It sets up no plugins, writes nothing, and never takes the log, so a read works while another host owns the thread. Its snapshot follows the same rules as `session.snapshot()`, without what only an open session knows (a running turn, operations, queues). `view.logState()` reads the `durability.logState` fold. +- `host.sessions.delete` closes the sessions of a thread and of its child threads. Then it deletes the log of the session tree (with the settings, config, plugin state, interrupted turn, runs, interrupts, usage, revert data, and the inputs that wait), the index entries, the uploaded media, and the generated artifacts. The first-party plugins also end the thread: `workspaceTools()` closes an owned sandbox with `'closed'`, and `snapshots()` removes the shadow repository. This needs the index entry, and the harness on that host: a host that opened it, or one with it in the `harnesses` option of `createHarnessHost()`. Else the host warns. A child thread is ended in the log of its root. While the delete runs, an open of the thread throws. With `access`, a thread with no index entry throws `AccessDeniedError`. +- A session takes `send` (a prompt, a steer into the running turn, a follow-up, or a message to an agent run with `to`), `continueWithoutMessage`, `reset`, `resolve`, `cancel`, and `moveToolCallToBackground` inputs. Each returns an operation with a receipt, and the session streams typed events. `host.isOpen(harness, { threadId })` says whether this host holds that session open. +- The session methods take one object: `cancelInput({ inputId, user? })`, `revert({ messageId, user? })`, `background({ toolCallId?, user? })`, `getMedia({ id })`, `mediaUrl({ id })`, and `agentTranscript({ subagentRunId })` or `agentTranscript({ runId })`. `AgentRun.subagentRunId` and `AgentRunInfo.subagentRunId` name the child id of an agent run, so the transcript of a run that code or a background call starts is found too. +- `session.appendRecords({ records, inputId?, user? })` returns a `Receipt`. A second append with the same `inputId` writes nothing. Without `inputId`, the append gets a new input id, and the log keeps it as an input. The `harness.revert` record keeps the user of `revert`, and the `harness.job` record keeps the user of `background`. +- `createHarnessHost({ access })` decides who can use each session. `getUser({ request })` reads the user of a request, and `isAllowed({ user, action, session, run })` decides each action. The user type comes from what `getUser` returns. The host keeps each tenant apart, needs a `user` on each call, and `host.open` returns a handle that acts only as its user. A refused action throws `AccessDeniedError`. The operations and the agent runs that a handle returns check access too. A child thread is checked through its root, also on `host.open`, so a stranger does not learn its root. When one user answers the turn of another user, the rest of the turn runs as the user who sent it. Without `access`, one user runs the host, with no checks. With `access`, keyed adapters and provider keys read env vars only when `access.allowServerEnvKeys` is `true`. +- `membersOnly()` gives ready access rules with the roles owner, member, operator, and viewer. `access` is `{ getUser, rules, getGuestUser?, allowServerEnvKeys? }`, and `membersOnly()` returns rules: `{ getUser, rules: membersOnly() }`. Rules are `{ isAllowed, filterSessionList?, members? }`: `filterSessionList` is the store filter of `host.sessions.list`, and `members` turns on `host.sessions.share` and `unshare` (`members.fork` sets the members of a fork). Your own rules are `rules: { isAllowed }`. `host.sessions.share` and `host.sessions.unshare` change the members, and `host.sessions.fork` takes `members: 'self' | 'all'`. The owner is the member with the role `owner`. For apps with their own access tables, `host.revoke` takes the access of a user to a thread, and returns a `Promise`. `host.revoke` and `unshare` cancel the running and queued work of the user on every host, and remove the schedules of the user at once. When that work cannot be stopped, they reject, and the access is gone already. A running `events()` read of a handle or a view ends with `AccessDeniedError`, and later view calls throw. `host.getUserOfRequest` reads the user of a request for your own routes. Each `host.sessions` method takes one object. +- A user with a `tenantId` keeps its sessions in its tenant: one thread id in two tenants is two sessions, with two index entries and two logs. `logIdOf({ tenantId, logId })` gives the id of a log in the store. +- Every run has a run context with its user, and that user's credentials and provider keys: `ctx.context.run` in tools and middleware (type them with `HarnessRunContext`), `ctx.turn.run` in a turn plugin, and `run` in a command. The `ctx.credentials` and `ctx.keys` of a session plugin have no user, so each call throws. `ctx.agents.start(agent, input, { user })` starts an agent for a user. `AgentRun.send` takes `user?`, and the `send` and `cancel` of a run act as the user of the run. `session.agents..run` honors `user`. The log keeps the sender of every input. The run context also has `epoch`, the ownership epoch of the log, and `idempotencyKey(step, scope?)`, a key for one side effect that is the same when the run runs again on any host. +- `definePlugin()` adds tools, prompts, middleware, agents, commands, config options, state, and extension points to a session or to one turn. With `lifetime: 'host'`, the host sets a plugin object up once per harness and shares it with every session that uses the same object. It is cleaned up when the last session that uses it closes, or at `host.close()`. A prompt with `placement: 'context'` goes in one system prompt after all the others, made again at each model call, so text that changes often (the todo list, the goal status, plan mode) leaves the system prompts before it in the provider prompt cache. The default `placement: 'system'` keeps a prompt in the system prompts. `prepareTools` gets `run`. `onTurnStart(turn)` runs once at the start of each turn, before the prompts, so a plugin can refresh the state that its prompts read. A plugin's `ctx.session.send` and `note` take `user?`, and default to the session owner. A thread that removes a plugin cannot run the agents of that plugin by name. +- `routing.router` sends a turn to a root agent. The log keeps the active agent, and the router reads it as `activeAgent`. A router that returns `undefined` gives the turn to the active agent, else to the main model. One agent name makes that agent active, `'main'` clears it, and `{ handoff, note }` is a handoff from the router. `routing.afterAgentsAnswer: 'end-turn' | 'main-answers'` (default `'end-turn'`) says whether the main model answers after the routed agents. +- Handoffs move a conversation to another root agent, and the target answers in the same turn. With `routing`, the main model and each root agent get a `handoff` tool. Code hands off with `ctx.harness.handoff({ to, note })` in a root agent, and with `ctx.context.run.handoff(...)` (`RunContext.handoff`) in a tool of the main model. Each handoff writes a `harness.handoff` record and streams a `harness.handoff` event (`from`, `to`, `note`, `by`, `runId`). A message that an agent writes has `metadata.harness.agent`. A turn hands off 10 times at most. An app tool named `handoff` fails the turn. `routing.giveModelsHandoffTool: false` gives no model the `handoff` tool, and the router and code can still hand off. A handoff stops only the chat that made it. When the part that handed off also stops for interrupts, the resolve turn runs the target (or pauses for a person). The router writes a handoff to the main model once when the active agent is gone. A handoff from the router or from code adds a `handoff` tool call and its result to the transcript before the target answers, so the target sees why it got the conversation. With `routing.giveModelsHandoffTool: false`, it adds one assistant note. +- With `routing.allowHandoffToHuman: true`, a handoff to `{ human: true }` opens a `human` wait with no default expiry, and the turn ends `interrupted`. User messages during the pause go in the transcript, and no model answers them. `session.send(text, { as: 'operator' })` adds an operator reply. It needs the `operator` access action, and it is rejected with `no_human_wait` when no handoff to a person is open. The operator ends the pause with an answer to the `human` wait: `{ resume: true }` or `{ handBack: '' | 'main' }`, also to a root agent of a turn plugin. When the `human` wait ends in any other way (a resolve, a cancel, an expiry), the agent from before the handoff answers again. An operator reply with files keeps its text in the assistant message, and the files in a user message after it with `metadata.harness.operatorAttachment`. A view shows the files with the reply, live and after a reload. +- `host.transfer({ from, to, envelope, user, transferId })` moves a conversation to another thread, also of another harness, in the same tenant. Only the envelope moves: a note, then no message, a summary that the source model writes, or the settled transcript with its media. The user needs `transfer` on the source and `send` on a target that exists. A new target has the user as its only owner. Both logs get a `harness.transferred` record, the index entries link the threads in `metadata`, `host.events()` yields a `transferred` event, and a retry with the same `transferId` is safe. A source thread that does not exist throws. A fork stays in its harness: `host.fork` with a thread of another harness throws. +- Agents run from code with `session.agents..run()` and `.start()`. `agentRun.receipt` resolves. In agent code, `ctx.harness` gives `step.do()` (a finished step returns its stored value when the run runs again), `step.sleep({ name, ms })`, `agents`, `state`, and `handoff` (in a root agent). `ctx.harness` is optional: a run that a plain `chat()` starts has none. +- `ctx.harness.agents` in agent code and `ctx.agents` in a plugin have one type, `AgentActions`: `start(agent, input, options?)`, `run(agent, input, options?)`, and `group({ concurrency, onFailure, key }, (g) => [g.start(...)])`. The input type comes from the input schema of the agent, and the result type from its output. `group` resolves to the results in order. With `onFailure: 'collect'`, each entry is an `AgentSettled` (`{ ok, value }` or `{ ok, error }`). The children of a group over its `concurrency` wait as `queued`. +- Each start takes `key`, `limits: { timeoutMs, maxAttempts }`, `budget: { maxTokens, maxCost }`, and `onModelError`. A run over its budget fails with the code `budget`, after the first model call over it. A run that runs again after a host stop counts the model use of the runs before. A run past `limits.timeoutMs` fails with the code `timeout`. A child never has more budget than its parent has left. `onModelError` is asked when a model call fails before it streams, and `'retry'` calls only that model call again (at most 3 calls). As data, `onModelError: { retries: 1 | 2, transientOnly? }` retries up to `retries` times (by default only a transient error), and the log keeps it, so a run that another host takes over or resumes retries too. A function is not in the log, so a run that another host takes over runs without it. A `ctx.agents` start in a plugin tool that an agent run calls is a child of that run, so it takes no `maxConcurrent` slot and the run can await it. `parent: ctx.context.run` names the run, and `parent: null` starts a root run. A `parent` that names no running agent run of this session throws, so a mistake does not start a root run. A running turn of the session gives a root run. +- The log keeps every run of a session as a `RunNode` of one run tree (`turn`, `agent`, `tool`, or `command`), with its parent, its user, its limits, its budget, and its result. `agentRuns()` reads the tree, and `AgentRunInfo` has `runId`, `kind`, `budget`, `result`, and `error`. Each `harness.usage` record has the `runId` of its run, and an agent run that ends keeps the usage of the run and its children. +- `maxDepth` and `maxCalls` count per tree, and `maxConcurrent` counts the root agent runs of the session. Over `maxConcurrent`, a root start waits as `queued`. The log keeps the counts, so a restart does not reset them. A `queued` start survives a host stop: the next host starts it with the same run id, in order, when a slot is free. +- A child run gets the user, the keys, and the signal of its parent: a cancel, a timeout, or a budget stop of the parent stops the child. A cancel of a turn also cancels the background runs that it started. A background run that a turn starts keeps the turn plugins until it settles. An agent that the model calls as a tool, and a routed agent, get `ctx.harness` too. +- A run with `runAgainAfterCrash: true` that runs again after a host stop gets the children that it started before, with their stored results, and starts only the rest. A background child that the model starts (`background: true`) runs again after a host stop as a run with `runAgainAfterCrash: true`, with the same `subagentRunId`. `durability.resumeBackgroundAgents: false` fails it instead. +- `ctx.harness.state({ key, initial })` gives the agents of a session a shared state by key, with `get` and `update`. The log keeps it, and a fork copies it. `STATE_SNAPSHOT` has `{ plugins, shared }`. +- A routed agent takes steers at its next model call, and late joins in the same turn, and `turn.beforeFinish` and `turn.onModelError` run for it. +- Every host keeps each session tree in one log, `stores.log` from `@tanstack/ai-persistence`. `memoryPersistence()` has a memory log, and a host with no `persistence` uses it. The log holds the inputs, events, transcripts, runs, interrupts, settings, config, plugin state, agent results, schedules, and tool steps, and the host folds them when a thread opens. A root thread and its child threads (agent runs and subagents) write to the log of the root. `session.logState()` reads the `durability.logState` fold of the whole tree. A harness with another `durability.addRecordsToModelContext` or `durability.logState` than the one that folds an open log throws when it opens that log. With a log store that survives a restart, the next host recovers a turn that a crash stopped. +- One host owns a log at a time: the first write takes it, one timer per host renews it, and a host that loses it stops its work on it. A host gives a log back when it has no work. Another host recovers the work of a log only after its ownership expires. A read never takes the log. An input that reaches another host waits in the input lane of the log, and the owner runs it (receipt `queued`) with its JSON overrides (`model`, `modelOptions`, `reasoning`, and `promptCache`). An input to a thread that was deleted is refused: the receipt is `rejected` with the reason `deleted`, and an awaited operation rejects with `InputRejectedError` ("The thread was deleted."). `ownership: { expiresAfterMs, renewEveryMs }` sets the timing. `renewEveryMs` must be less than `expiresAfterMs`, else `createHarnessHost` throws a `RangeError`. With only `expiresAfterMs`, `renewEveryMs` is a third of it. +- When the owner refuses an offered input (busy, or an id that the log has with another payload), `session.settled()` on the host that offered it rejects with `InputRejectedError`. The log gets a `harness.offer.rejected` record. An offer that comes while the owner appends is not missed. +- A host that does not own a thread opens it while the owner runs a background job. Settings, config values, and the interrupted turn follow the changes of other hosts. `updateThreadSettings` merges into the latest stored settings. +- The owner writes a snapshot of the folded log to the log store after `log.saveSnapshotAfter.bytes` of new records (default 1 000 000) or `log.saveSnapshotAfter.ms` (default 300 000), and a new host opens from it. With `lifecycle.retention.keepLogRecords: 'since-snapshot'` (default `'all'`), the owner truncates the records before each snapshot. A reader whose cursor is in the dropped part gets one `harness.gap` event (`EventGap`), then the stream goes on. With `'since-snapshot'`, a thread whose snapshot the host cannot use (a changed `reduce` version or `project`) cannot open, and rejects with `LogTruncatedError`. `session.revert()` works with `'since-snapshot'`: the snapshot keeps the last 500 file steps. A plugin sets which of its records the snapshot keeps with `keepRecords: { [type]: count }` on its definition, and reads them, oldest first, with `ctx.session.keptRecords(type)`. The fold keeps the last 100 ended runs, run nodes, interrupts, and agent results, and drops the turn overrides of a settled input, so a long thread does not grow its fold and its snapshot with each turn. +- `host.submit` stores an input for a thread and runs nothing. `session.schedule({ at, input, scheduleId, user })` runs an input later, and `session.cancelSchedule({ scheduleId })` stops it. `createHarnessWorker({ host, harnesses, concurrency, signal, pollMs })` runs the threads that have work and no live host, and returns `{ stop }`. `host.resumePending` does one sweep. A swept thread that fails to open goes back with a wake in 30 s. A worker skips a thread that fails to open for a time that doubles after each failure in a row, up to 5 minutes, so the other threads still run. `host.submit` throws for a thread of another harness, and a later offer never changes the harness of a log. +- `durableTool()` keeps the result of each step. A crash can cut a step, and then it runs again, so `step.do` passes `{ idempotencyKey, epoch }` to `fn`. Give the key to your system, so it refuses the second side effect. `step.ask({ name, message, schema })` asks the user: the log keeps the question, any host can take the answer, and a run again after a stop returns the logged answer. A `step.do` value must be JSON: a `Date`, a `Map`, or a class instance is a type error. `step.sleep({ name, ms })` waits in the process, and the log keeps the wake time, so a worker continues the call after a stop. A sleep longer than 30 s on a thread that only a worker runs gives the thread back with the wake, so it keeps no worker slot. `durability.truncatedToolResult` sets the text of a tool call cut at the output limit. Without it, a cut answer ends the turn. - `turn.onModelError`, `retryTransientErrors()`, `turn.beforeFinish`, and `turn.canJoin` control how a turn retries, continues, and ends. +- Every approval, question, sign-in, client tool, and human takeover that a run waits for is a `Wait` in the session log (`waitId`, `kind`, `runId`, `schema`, `expiresAt`, `message`). `WaitKind` is `approval`, `question`, `sign-in`, `client-tool`, or `human`. `waitKind`, `isApproval`, `waitFromInterrupt`, and `waitFromQuestion` come from the root and from `./view`. `snapshot().pendingWaits` lists the open waits: each one with its `interrupt` when a chat run stopped for it, and `turn: true` for a wait of the stopped turn. It is the one wait view of a snapshot. and `session.waitForRun({ runId, signal })` resolves with the next one. A wait ends `answered`, `cancelled`, or `expired`. A wait with `expiresAt` expires on time. Only `sign-in` has a default limit: 15 minutes. +- A wait that only the log holds keeps no host busy: the session goes idle, and an answer on any host continues the run. `resolve` answers the waits with an `interrupt`: of a stopped turn, or of one background agent run per call. `answer` answers the waits with no `interrupt` (a `step.ask` question, an MCP question, the `human` wait of a handoff), checks the answer against the JSON Schema of the wait, and keeps a secret out of the log. A `resolve` of such a wait is refused with `use_answer`, and a second answer to a wait that one answer ends is refused. `host.submit` checks an answer against the wait that it answers. +- A run waits for a person only at fixed points: `permissions()` before a tool call, the `question` tool, `step.ask` in a durable tool, a command (`CommandContext.step`), or an agent run, and the live waits of MCP elicitation and OAuth sign-in. `ctx.session.ask` throws anywhere else, with an error that names `step.ask`, the `question` tool, and `PermissionResources`. A command that called `step.ask` runs again from the top after a host stop. `Question` takes `expiresAt` and `signal`. +- A background agent run stops with status `waiting` on a wait, for example an approval of `permissions()`, and continues after the answer, also after a restart. An agent that the model calls as a tool inside a turn waits with the turn: its approvals, questions, sign-ins, app interrupts, client tools, and `ctx.harness.step.ask` questions are waits of the turn in `snapshot().pendingWaits`, with `agent` and `subagentRunId`. After the resolve, also on another host or after a restart, the agent goes on from where it stopped, and its result goes to the model as the tool result. Its `ctx.harness.step.do` values are in the log. With `durability.agentToolWaits: false`, the agent does not wait: a call of it that needs an approval is refused. +- The `operator` access action answers a `human` wait of any run, also with a resolve. A `human` wait takes only `operator`. `membersOnly()` gives it only to the role `operator`, so the user who waits for a person cannot act as that person. `membersOnly({ operator: { roles } })` sets the roles that have it. `isAllowed` gets the `wait` (`waitId`, `kind`) of an `answer` or `operator` action. +- `session.settled({ inputId, signal })` takes one object. `signal` stops the wait for the settlement. - `session.onQuestion`, `session.onAuthRequired`, `session.onMedia`, and `session.onUsage` call a listener for the events that a user answers or sees, with a typed value. -- Sessions also have thread settings, fork and revert, usage totals, media files, the transcript of each child agent (`session.agentTranscript`), and `harnessText()` / `harnessAgent()` to use a harness as a text adapter or an agent. +- Sessions also have thread settings, fork and revert, usage totals, media files, the transcript of each child agent (`session.agentTranscript`), and `harnessText()` / `harnessAgent()` to use a harness as a text adapter or an agent. In a session, a `harnessAgent` child is a child run in the log of the root, with the user, keys, limits, and budget of the run that calls it. Its usage and its waits show in the parent session, on every host and in `host.observe`. An answer on another host goes to the host that runs the child. The base64 cache of media files is shared by all the sessions of a host. +- `harnessText()` uses the user of each call. The inner thread is `/inner/` on every host, and `host.open` refuses that id from app code. When a host with access runs the outer thread too, the inner session is checked through the outer thread, so every user of a shared outer thread can use it. The index entry of the inner thread keeps the link to its outer thread. An abort on a remote harness sends a cancel. The usage of the inner run reaches the outer `RUN_FINISHED`. Its SSE reader takes multi-line `data:` and CRLF. `harnessText()` and `harnessAgent()` forward events on one shared path. +- `createHarnessHost({ observability: { tracer, onOperation } })` traces runs. `tracer` takes an OpenTelemetry `Tracer`. `@opentelemetry/api` is an optional peer dependency: the host loads it only when it has a tracer. The host writes `harness.operation` spans (turns, agent runs, commands), and `harness.recovery`, `harness.wait`, and `harness.transfer` spans. A child run on the same host is a child span: the host passes `trace.setSpan(context.active(), parent)` as the third argument of `startSpan`. A tracer that throws is logged and fails no run. `SendOptions.traceparent` stores a W3C `traceparent` with the input, and the run span on the host that runs it gets a link to it. +- `onOperation` gets a `started` report and a `settled` report (`OperationReport`) for each operation that this host runs, with `status`, `durationMs`, `error`, and `usage`. A listener that throws gets a `harness.plugin.warning` event, and the work goes on. +- More lifecycle events in the session stream: `harness.input.queued`, `harness.ownership.changed` (`reason: 'taken_over'` when the store says that the old owner expired, else `'acquired'`), `harness.stranded`, `harness.transferred`, `harness.wait.opened`, and `harness.wait.settled`. `HarnessEventValues` maps each `harness.*` event name to the type of its value. +- `onEvent`, `onQuestion`, `onUsage`, and `onMedia` also get the events that another host writes to the log of the session. An event fires once. The status of `host.events()` counts that work too, for the sessions that are open on the host. +- The router gets `model`: `model.chat({ adapter, options, label })` in `routing.router` (`HarnessRouterContext.model`) calls a model with the keys of the sender, and a cancel of the turn stops it. +- `session.usage()` has `bySource`: the model calls that are not a turn or an agent run, keyed by `plugin:`, `router`, or `transfer`. Each `UsageCall` and `harness.usage` event of such a call has `source`, and `label` when the call has one. The summary of `host.transfer` with `messages: 'summary'` counts in the source thread under `transfer`, with the label `summary`, and stops after 2 minutes. diff --git a/.changeset/ai-harness-plugins.md b/.changeset/ai-harness-plugins.md index f0812eb769..0c2ca1a70d 100644 --- a/.changeset/ai-harness-plugins.md +++ b/.changeset/ai-harness-plugins.md @@ -2,10 +2,17 @@ '@tanstack/ai-harness': minor --- -Add first-party plugins in `@tanstack/ai-harness/plugins` (a Node entry): +Add first-party plugins. `@tanstack/ai-harness/plugins` loads no Node module. The plugins that read files are in the Node entry `@tanstack/ai-harness/plugins/node`. -- `permissions()`: rules for tools, paths, and commands, the `default`, `plan`, `acceptEdits`, and `bypass` modes, questions answered with once, always, or reject, saved "always" rules with `/permissions` and `/permissions forget `, and `listSavedPermissions` / `deleteSavedPermission` for server code. The rules apply in subagent runs too. -- `agents()`: agent profiles with their own prompt, tools, model, rules, and step limit, in code or in Markdown files, `/agent` to switch the primary agent, and the built-in `build`, `plan`, `general`, and `explore` agents. -- `question()`, `todos()`, `title()`, `modelPicker()`, `projectInstructions()`, `fileCommands()`, `compact()`, `usage()` (with your own prices for calls that have no provider cost), and `boundToolOutput()`. +- `permissions()`: rules for tools, paths, and commands, and the `default`, `plan`, `acceptEdits`, and `bypass` modes. It decides every call of a model message before the calls run, and opens one approval wait for each call that asks, so the turn stops and no process waits. An answer is `true` or `false`, `'once'`, `'always'`, or `'reject'`, or `{ answer, message? }`. It also has saved "always" rules with `/permissions` and `/permissions forget `, and `listSavedPermissions` / `deleteSavedPermission` for server code. The rules apply in subagent runs too. It decides in `onGuardToolCall`, on the final checked arguments. +- `agents()` (in `plugins/node`): agent profiles with their own prompt, tools, model, rules, and step limit, in code or in Markdown files, `/agent` to switch the primary agent, and the built-in `build`, `plan`, `general`, and `explore` agents. A subagent run uses the tools and the model of the turn that started it. A run whose turn is not in the memory of this host fails with a clear error. A run that code starts needs a profile `model`. +- `question()`: a `question` tool for the lead turn. Each question is a wait of the turn, answered with `resolve`. Agent runs do not get the tool. +- `todos()`, `title()`, `modelPicker()`, `compact()`, and `usage()` (with your own prices for calls that have no provider cost). +- `goal()` runs each turn as the user who set the goal. +- `projectInstructions()`, `fileCommands()`, and `boundToolOutput()`, in `plugins/node`. + +A plugin runs agents with `ctx.agents.start`, `ctx.agents.run`, and `ctx.agents.group({ concurrency }, (g) => [g.start(agent, input)])`, with the same types as `ctx.harness.agents` in agent code. `PermissionRules`, `PermissionResources`, `PermissionDecisionCapability`, and `decidePermission` are also exported from the package root. They load no Node module, so an edge-safe plugin can read the permission rules. + +`ctx.model.chat({ adapter, options, label, run, timeoutMs })` calls a model from a plugin, for example a judge or a title. It uses the keys of the user of the run, the budget of the run applies, and it stops when the run stops or after `timeoutMs` (default 2 minutes). Its usage counts in the session under `plugin:` in `session.usage().bySource`. `goal()`, `title()`, and `compact()` call their models this way, with the labels `judge`, `title`, and `compact`. diff --git a/.changeset/ai-harness-provider-keys.md b/.changeset/ai-harness-provider-keys.md index 574005d03f..3539e664a5 100644 --- a/.changeset/ai-harness-provider-keys.md +++ b/.changeset/ai-harness-provider-keys.md @@ -2,4 +2,4 @@ '@tanstack/ai-harness': minor --- -Add the `providerKeys({ providers })` plugin to `@tanstack/ai-harness/plugins`. Users of a shipped harness connect their own model provider in the app, with no `.env` file. It adds `/connect:`, `/disconnect:`, and `/keys`. `/connect:` runs the provider's `signIn` when it has one, else it opens `keyUrl` when set and asks the user to paste the key as a secret question. The key is saved in the credential store of the user, and a `keyedAdapter(...)` for that provider uses it. `/keys` and the plugin state show where each key comes from: a saved key (the last 4 characters only), the env var, or no key. +Add the `providerKeys({ providers })` plugin to `@tanstack/ai-harness/plugins`. Users of a shipped harness connect their own model provider in the app, with no `.env` file. It adds `/connect:`, `/disconnect:`, and `/keys`. `/connect:` runs the provider's `signIn` when it has one, else it opens `keyUrl` when set and asks the user to paste the key with `step.ask` and `secret: true`. The log keeps the question as a wait, so a user on any host can answer it, and the log never keeps the key. The key is saved in the credential store of the user, and a `keyedAdapter(...)` for that provider uses it. `/keys` shows where each key of the user comes from: a saved key (the last 4 characters only), the env var, or no key. The plugin state shows only the env vars that the session reads, as every member of the thread reads it. On a host with access without `allowServerEnvKeys`, no env var counts. diff --git a/.changeset/ai-harness-transport.md b/.changeset/ai-harness-transport.md index 9f0d8ebd40..c0cbab2f9c 100644 --- a/.changeset/ai-harness-transport.md +++ b/.changeset/ai-harness-transport.md @@ -4,8 +4,20 @@ Serve a harness session to clients. -- `createHarnessHandler()` serves a session over HTTP. `POST run` is standard AG-UI, so `useChat` with `fetchServerSentEvents` works on a harness, with approvals, client tools, and a reload that joins the running turn. `POST control`, `GET events`, `GET transcript`, `GET agent-transcript` (the messages of a child agent run), `GET snapshot`, `GET describe`, `sessions`, and the media routes cover the rest. -- `handleHarnessSocket()` serves the same session over a WebSocket. +- `createHarnessHandler()` from `@tanstack/ai-harness/server` serves a session over HTTP, as the user that the host `access` reads from each request. `POST run` is standard AG-UI, so `useChat` with `fetchServerSentEvents` works on a harness, with approvals, client tools, and a reload that joins the running turn. `POST control`, `GET events`, `GET transcript`, `GET agent-transcript` (the messages of a child agent run), `GET snapshot`, and `GET describe` cover one session. `sessions` (with the `share` and `unshare` ops), `GET host-events`, and the media routes cover the rest. A `host.events()` reader listens from the call, and one that falls more than 1 000 changes behind stops, and reads again for the current statuses. A refused action answers 403. A read of a thread that does not exist answers 404 and starts no session. A signed media URL still needs a user: `getUser` reads the user of the request, and that user needs `read` on the thread, in the tenant of the URL. For a request with no login, `access.getGuestUser({ tenantId })` can give a guest, and `isAllowed` decides `read` for it. The guest exists only on the signed media route. `requestUsers` from `@tanstack/ai-harness/server` gives the user of each request with the same rule, for your own server. The read routes read a thread with `host.observe`, so they set up no plugins. `GET describe` opens the session. +- `handleHarnessSocket()`, from the same entry, serves the same session over a WebSocket. It takes the upgrade `request` and reads the user from it. It handles the frames of one connection in order. +- A `command` or an `agent` input over HTTP or the socket runs once for one `inputId`, and the `inputId` of its receipt is the id of the client. - `expose` on `defineHarness()` names the agents, settings, config keys, and commands that a client can use. Any other input of those kinds gets `rejected` with `not_exposed`. -- `createHarnessClient()` from `@tanstack/ai-harness/client` is a typed, browser-safe client. It sends inputs, reads the session, and streams events that resume from a cursor. -- `oauthConnector()` and the OAuth helpers (`loopbackLogin`, `deviceLogin`, `exchangeCode`, `refreshCredential`, and more) keep a credential per sender. `token({ wait: true })` stops the turn with an `auth_required` interrupt until the user signs in. +- `createHarnessClient()` from `@tanstack/ai-harness/client` is a typed, browser-safe client. It sends inputs, reads the session, and streams events that resume from a cursor. Without `from`, `client.events()` starts at the newest event (the cursor of the snapshot), so it does not send the whole history first. This is the default: a reader that needs the history (to rebuild a transcript, for example) must pass `from: '0'` or a saved cursor. When the snapshot answer has no string `cursor`, `client.events()` throws and does not retry. `client.events()` connects again after `idleTimeoutMs` (default 60 000) with no bytes. The `prompt` input of the protocol and `client.send` take `as: 'operator'` for the reply of an operator after a handoff to a person. +- The client methods take one object: `cancelInput`, `revert`, `agentTranscript`, `mediaUrl`, `appendRecords`, and `putMedia`. `client.sessions.list`, `rename`, `delete`, `update`, `fork`, `share`, and `unshare` manage the sessions. `HarnessErrorCode`, the type of `HarnessProtocolError.code`, comes from the main, `/client`, and `/server` entries. +- `oauthConnector()` and the OAuth helpers from `@tanstack/ai-harness/oauth` (`loopbackLogin`, `deviceLogin`, `exchangeCode`, `refreshCredential`, and more) keep a credential per user. A tool calls `token({ run: ctx.context.run })` for the token of the user of its run. `token({ run, wait: true })` stops the turn with an `auth_required` interrupt until the user signs in. `deviceLogin`, `loopbackLogin`, `startLoopbackReceiver`, `exchangeCode`, and `refreshCredential(config, credential, { fetch, signal })` stop at once when their `signal` aborts, so a cancel or a session close stops a running `/connect`. `OAuthConfig.issuer` makes `loopbackLogin` refuse a code from another issuer (RFC 9207). The loopback deadline (`timeoutMs`) does not keep the process alive. +- `run.credentials.refresh(id, update, { signal })` refreshes a saved credential once for all callers on a host, keyed by tenant, user, and id. It saves with `setIf` when the store has `getVersioned` and `setIf`, and uses the other host's token on a conflict. Without that pair, it does a plain `set`. It saves to the scope where it found the credential, so a tenant token stays the tenant's. `update` gets a signal that only `host.close()` aborts. An abort of `signal` rejects only that call. `oauthConnector` refreshes this way, and `refreshCredential` keeps the `client` of the old credential. +- The session stream speaks protocol version 2 on every transport: `GET events`, the socket, and the stdio worker. Each event is an AG-UI wire chunk in a `harness.event` frame, with `cursor`, `threadId`, and `operationId`. The AG-UI `RUN_STARTED` of a child agent run has `parentRunId`, so a client can rebuild the run tree. +- The first frame of a stream is `harness.hello` with the version `v`. A client asks for a version with `?v=` or with `v` in `harness.subscribe`. Another version gets a `harness.error` frame with the code `unsupported_version`, and the stream closes (socket code 4400). +- A `harness.error` frame has a `code` (`unauthorized`, `forbidden`, `invalid_frame`, `unsupported_version`, `slow_consumer`, or `internal`) and a `requestId`. On the socket and the stdio worker, the `requestId` is the id of the frame that failed. Every answer of the handler has an `X-Request-Id` header: the id that the client sent when it is a short safe id, else a new UUID. +- Streams stay open and protect the server. SSE streams send a `: ping` comment and the socket sends a `harness.ping` frame when idle. SSE reads the log only as fast as the client reads, and the socket waits while its `bufferedAmount` is over the limit. A client that does not read in time gets a `slow_consumer` error frame, and the stream closes (socket code 4408). A slow client loses no events: it connects again from its last cursor. `createHarnessClient` does this in `events()` and `hostEvents()`, and ends the read with a `HarnessProtocolError` (with its `code`) for any other error frame. +- `presence: { onConnect, onDisconnect }` on `createHarnessHost` reports each client stream of `GET events` and of the socket (after its subscribe), as a `PresenceEvent` with a new `connectionId`, the user, the thread, the transport, and `resumed`. `onDisconnect` runs once per stream, with the reason `closed`, `revoked`, or `error`. The host does not wait for a hook, and a hook that fails only warns. +- `stream: { heartbeatMs, slowConsumerMs, sseQueueBytes, socketBufferBytes }` on `createHarnessHandler` and `handleHarnessSocket` changes these values. The defaults are 20 s, 30 s, 1 MiB, and 8 MiB. A value that is not a finite number above 0 throws at the start. `HarnessStreamOptions` is the type. +- A reader whose events are gone gets one `harness.gap` event, with `reason: 'truncated'` (a truncate of the log) or `'evicted'` (the memory feed of a session dropped them). The stream then goes on from `resumeAt`. +- A W3C `traceparent` goes with a message: as the `traceparent` header of `POST control` and `POST run`, or as the `traceparent` field of a `harness.input` frame. `ClientSendOptions.traceparent` makes `client.send` send the header. +- A `cancel` input stops a turn from any host that reads the same log, so `client.cancel({ operationId })` works behind a load balancer. A closed connection still never cancels work. To stop a `useChat` turn, call `client.cancel` with the `runId` of `useChat` in the stop handler. diff --git a/.changeset/ai-harness-view.md b/.changeset/ai-harness-view.md index 303679dbe7..0fdf013d0d 100644 --- a/.changeset/ai-harness-view.md +++ b/.changeset/ai-harness-view.md @@ -4,7 +4,11 @@ Add `createSessionView()` in `@tanstack/ai-harness/view`. It turns a session (local, or remote through `createHarnessClient()`) into one live TanStack Store state that works with any UI library. -- The state: messages with their parts, tool calls, approvals, client tools, other interrupts, questions, sign-ins, child agents with their results, media with signed URLs, waiting inputs, usage, models, settings, commands, and plugin state. -- Actions: `send` (a prompt, a steer, or a `/command`), `followUp`, `continue`, `upload`, `cancel`, `cancelInput`, `setDelivery`, `configure`, `reset`, `revert`, and the answers to approvals (with edited arguments), client tools, interrupts, and questions. +- The state: messages with their parts, tool calls, approvals, client tools, other interrupts, questions, sign-ins, `waits` (every open wait, with its kind), child agents with their results and a `status` of `running` or `waiting`, media with signed URLs, waiting inputs, usage, models, settings, commands, and plugin state. +- The waits of background agent runs show in `approvals`, `clientTools`, and `interrupts`. The view answers each one at once: with `resolve`, or with `answer` for a `human` wait that no turn holds. +- Actions: `send` (a prompt, a steer, or a `/command`), `followUp`, `continueWithoutMessage`, `putMedia`, `cancel`, `cancelInput`, `changeInputDelivery`, `updateThreadSettings`, `reset`, `revert`, `loadAgent`, and the answers to approvals (with edited arguments), client tools, interrupts, and questions. `cancelInput`, `revert`, `approve({ id, ...options })`, `reject({ id, payload? })`, `notice({ text })`, and `putMedia` take one object, and so does the `reject({ payload? })` of an approval item. - Typed events: `view.on('approval' | 'interrupt' | 'turnEnd' | ..., handler)` and plugin events. -- A view that opens while a turn runs shows the turn from its start. After a reload, each child agent shows its own messages again. +- In a harness with `routing`, an assistant message has `agent`, the root agent that wrote it. After a handoff, the next agent writes a new message in the same turn. +- `view.send(text, { as: 'operator' })` sends the reply of an operator while a handoff to a person is open. The reply shows in each open view at once. +- A view that opens while a turn runs shows the turn from its start. After a reload, each child agent shows its card, and `view.loadAgent({ id })` fills it with its own messages. +- When the stream sends a `harness.gap` event, the view stops its read, loads the snapshot and the transcript again, then reads on from the new cursor. No turn shows twice. diff --git a/.changeset/ai-mcp-harness.md b/.changeset/ai-mcp-harness.md index e3f5f39956..38e67ecbc9 100644 --- a/.changeset/ai-mcp-harness.md +++ b/.changeset/ai-mcp-harness.md @@ -4,9 +4,11 @@ Use MCP in a TanStack AI harness. -- `@tanstack/ai-mcp/harness` adds `createHarnessMcpServer`. It serves a harness as an MCP server, so Claude Code, Claude Desktop, Cursor, or another agent can use it. The tools are `chat`, `steer`, `cancel`, `approve`, `reject`, `resolve`, `answer`, and `status`, plus `agent_` for each exposed agent and `command_` for each exposed command. Every tool takes an optional `threadId`. With `approvals: 'ask'` (the default), the client asks the user about each approval when it supports elicitation. Otherwise the approvals come back in the result. `approvals: 'auto'` approves every tool call. `resolve` answers every kind of interrupt in one call. -- `@tanstack/ai-mcp/harness` also adds the `mcp()` harness plugin. It connects stdio and HTTP servers by name, gives the model their tools as `_`, and adds a `/mcp` status command. `oauth: true` signs in to a server with `/connect `. -- `@tanstack/ai-mcp/connector` adds `mcpConnector`: one OAuth MCP server as a harness plugin, with `needsApproval`, `prefix`, `scopes`, and `clientName`. The sign-in passes the `iss` of the callback on (RFC 9207). -- `mcp()` and `mcpConnector` answer the `elicitation/create` requests of MCP spec 2025 servers, and the input requests of spec 2026-07-28 servers, as session questions. One tool call can ask up to 5 times. +- `@tanstack/ai-mcp/harness` adds `createHarnessMcpServer`. It serves a harness as an MCP server, so Claude Code, Claude Desktop, Cursor, or another agent can use it. The tools are `chat`, `steer`, `cancel`, `approve`, `reject`, `resolve`, `answer`, and `status`, plus `agent_` for each exposed agent and `command_` for each exposed command. Every tool takes an optional `threadId`. With `approvals: 'ask'` (the default), the client asks the user about each approval when it supports elicitation. Otherwise the approvals come back in the result. `approvals: 'auto'` approves every tool call. `resolve` answers every interrupt of a stopped turn in one call, or the waits of one background agent run. It finds the wait in the log, so it also answers a wait that another server, or a server before a restart, opened. A question takes `answer`. Every result lists the open waits of its work in `waits`, each with a `kind` from `waitKind` (`approval`, `question`, `sign-in`, `client-tool`, or `human`). The `status` tool lists every wait, and each background agent with `status: 'running'` or `'waiting'`. +- On a host with `access`, `createHarnessMcpServer` reads the user of each HTTP request with `getUser`. The host checks each tool call of that user. A host without `access` needs `openAccess: true` (no user) or `openAccess: { user }` (one local user), else the server throws at the start. It uses `requestUsers`, the same rule as the HTTP handler. Each tool call releases its session handle when it ends, so the session closes on idle. The `status` tool and the media reads read the thread with `host.observe`, so they set up no plugins. The server forgets a wait when its operation settles. +- `@tanstack/ai-mcp/harness` also adds the `mcp()` harness plugin. It connects stdio and HTTP servers by name, gives the model their tools as `_`, and adds a `/mcp` status command. A stdio server starts once per host for each `mcp()` object, and every session that uses that object shares it. `oauth: true` signs in to a server with `/connect `, for each user. +- `mcp()` connects again with backoff when a server closes, and lists the tools again on `list_changed`. A server that fails its first connect also tries again with backoff. `mcpConnector` and `oauth: true` servers connect again at the next turn, and list their tools again. Each server takes `needsApproval` and `toolName`. The backoff starts again only after a connection stayed open for 30 s, so a server that crashes right after it connects waits longer each time. When two servers give a tool with the same name, `mcp()` warns once. With `onDuplicateTool: 'error'`, the later server fails instead, with an error that names both servers and the tool. +- `@tanstack/ai-mcp/connector` adds `mcpConnector`: one OAuth MCP server as a harness plugin, with `needsApproval`, `prefix`, `scopes`, and `clientName`. The sign-in passes the `iss` of the callback on (RFC 9207). A cancel or a session close stops a running sign-in at once: the loopback stops listening, and the OAuth requests abort. When the MCP SDK refreshes a token, the connector saves it with `credentials.refresh` of the harness: to the scope of the sign-in, and with compare-and-set when the store has it. Each session still sends its own refresh request. When the server refuses a token, the connector deletes the saved sign-in only if it is the token that this session used. When the user signs out while the SDK refreshes, the connector saves no new token, and the call asks for a sign-in. +- `mcp()` and `mcpConnector` answer the `elicitation/create` requests of MCP spec 2025 servers, and the input requests of spec 2026-07-28 servers, as session questions. One tool call can ask up to 5 times. Each question is a live wait of the run that made the tool call: it ends `cancelled` when that run stops. `mcp()` tracks the live questions for each server, so each question goes to the run that called the tool. A spec 2025 request goes to the user only when all the running calls of that server belong to one run. Else the server gets `decline`. `@tanstack/ai-harness` is an optional peer dependency. Only the `./harness` and `./connector` entries need it. diff --git a/.changeset/ai-sandbox-harness.md b/.changeset/ai-sandbox-harness.md index 10f2443916..a837603ec3 100644 --- a/.changeset/ai-sandbox-harness.md +++ b/.changeset/ai-sandbox-harness.md @@ -5,6 +5,7 @@ Add `@tanstack/ai-sandbox/harness` for `@tanstack/ai-harness`: - `codingAgents({ sandbox, agents, workspace? })` is a harness plugin. The lead model gets one tool per coding agent (Claude Code, Codex, Grok Build, or any ACP agent). Each agent works in the sandbox, keeps its own session per thread (also after a restart), and starts read-only in the `plan` mode of `permissions()`. `/fresh [agent]` starts a new session. `workspace: 'shared'` (default) runs the agents of a thread one at a time in one sandbox. `'per-agent'` gives each agent its own sandbox. -- `sandboxWorkspaceBackend(sandbox)` runs the coding tools of `@tanstack/ai-harness/plugins/coding` (`workspaceTools()`, `snapshots()`, `formatter()`) in a sandbox. +- `codingAgents` runs agents only on the host that owns the log of the thread, so `'shared'` runs one agent at a time also with many hosts. +- `sandboxWorkspaceBackend(sandbox, { owned? })` runs the coding tools of `@tanstack/ai-harness/plugins/coding` (`workspaceTools()`, `snapshots()`, `formatter()`) in a sandbox. It resolves links in the sandbox, reads one page of a file in the sandbox, and stops background jobs by their id (the pid and its start time), also from another host. It never stops another process that got the pid of a job. `binding()` gives the provider and the sandbox id, so another host can resume the sandbox. `close()` stops the jobs of the backend. A session close closes it with `'moved'`, so an owned sandbox stays for the next host. `host.sessions.delete` closes it with `'closed'`, and with `owned: true` that destroys the sandbox. `@tanstack/ai-harness` is an optional peer dependency. diff --git a/.changeset/ai-subagent-binding-interrupts.md b/.changeset/ai-subagent-binding-interrupts.md new file mode 100644 index 0000000000..b7d9a69386 --- /dev/null +++ b/.changeset/ai-subagent-binding-interrupts.md @@ -0,0 +1,16 @@ +--- +'@tanstack/ai': minor +--- + +Add `interrupts` to `SubagentBinding`. + +- A host sets `binding.interrupts` to give interrupt definitions to every `ctx.chat` call of an agent run. Then the host middleware in `chatMiddleware` can stop the child with these interrupts. +- The own `interrupts` of the call win. When a host definition has the same id as one of the call, the host definition is not added. +- A child's `ctx.chat({ subagents })` passes the definitions on to its own children. +- A plain `chat()` call does not get them. + +Add `lastChatMiddleware` to `SubagentBinding`. It runs after the own middleware of each `ctx.chat` call of an agent run, so it sees the final model context and tool results. A child's `ctx.chat({ subagents })` passes it on, after the child's own. + +Add `validateJsonSchema(schema, value)`. It checks a value against a JSON Schema with `@cfworker/json-schema`, as a tool input is checked, but with no repair. The result has the Standard Schema shape: `{ value }` or `{ issues }`. The root also exports the `StandardSchemaV1` and `StandardJSONSchemaV1` types. + +An agent `run` that throws the shape of `MCPInputRequiredError` (`name`, `kind`, `request`) stops the child as `suspended`, as a tool does, and the parent run stops with its interrupt. The interrupt has a generic binding and no tool call. Its id is the `interruptId` of the thrown value, else `mcp_input_`, and its message is the error message. The resume runs the child again, and `ctx.resume` has the answer. diff --git a/.changeset/ai-tool-guard.md b/.changeset/ai-tool-guard.md new file mode 100644 index 0000000000..30087006ea --- /dev/null +++ b/.changeset/ai-tool-guard.md @@ -0,0 +1,11 @@ +--- +'@tanstack/ai': minor +--- + +Add the `onGuardToolCall` middleware hook for checks that must always run, such as permissions. + +- It runs for every tool call, after all `onBeforeToolCall` hooks and after the `inputSchema` check. `hookCtx.args` holds the final checked arguments that the tool gets. +- The guard of every middleware runs, in order, until one returns a decision. An earlier `onBeforeToolCall` decision does not skip it. +- Return `{ type: 'skip', result }` to deny the call. The tool does not run, and the model gets `result`, the same as an `onBeforeToolCall` skip. Return `{ type: 'abort', reason }` to abort the run. +- It runs before a server tool runs and before a client tool goes to the client. A guard cannot change the arguments. +- The new `GuardToolCallDecision` type is exported. diff --git a/.changeset/ai-utils-command-env.md b/.changeset/ai-utils-command-env.md new file mode 100644 index 0000000000..86516cd423 --- /dev/null +++ b/.changeset/ai-utils-command-env.md @@ -0,0 +1,5 @@ +--- +'@tanstack/ai-utils': minor +--- + +Add `commandEnv(env, platform, passEnv?)`, the short env of a command that a process starts (no provider keys or other secrets of the server), and `createOutputTail(maxBytes)`, the newest output of a command. `@tanstack/ai-harness`, `@tanstack/ai-sandbox`, and `@tanstack/ai-sandbox-local-process` use them. diff --git a/.changeset/ai-websocket-buffered-amount.md b/.changeset/ai-websocket-buffered-amount.md new file mode 100644 index 0000000000..5391cc79e7 --- /dev/null +++ b/.changeset/ai-websocket-buffered-amount.md @@ -0,0 +1,5 @@ +--- +'@tanstack/ai': patch +--- + +`WebSocketLike` has an optional `bufferedAmount`. A server can read it to stop sending to a slow socket. A socket without it works as before. diff --git a/.changeset/byok-credential-store.md b/.changeset/byok-credential-store.md index 76c6540880..2b3af2222c 100644 --- a/.changeset/byok-credential-store.md +++ b/.changeset/byok-credential-store.md @@ -9,5 +9,12 @@ BYOK keys can now live on the server. `@tanstack/ai/byok` exports the `Credentia - `tenantScopeOf(scope)`: the tenant scope of a user scope, or `undefined` when the scope has no user or no tenant. - `memoryCredentialStore()`: a store in memory, for development and tests. - `envKeyEntry(provider)`: the first `provider.env` name that is set, with its value. +- `readCredentialEntry(store, scope, id)`: reads a credential like `readCredential`. It also tells you which scope had the credential (the user or the tenant), and gives its revision when the store has `getVersioned`. + +`CredentialStore` has two optional methods for compare-and-set writes, `getVersioned` and `setIf`, with the same shape as the pair on `MetadataStore`. A store without them still works. `memoryCredentialStore` has both. Keep the store in your own database or in a secrets manager, and encrypt the secrets at rest. An `oauth` credential can keep the OAuth `client` it was issued to. `list` never returns secret values. + +`scrubSecrets` from `@tanstack/ai/byok` now also removes the JSON-escaped and URL-encoded forms of each secret. It skips a secret that is shorter than 6 characters, so a short value does not remove common words. + +`Credential` has a new type, `secret_answer` (`{ value, expiresAt }`). The harness keeps a secret answer to a question there for a short time and deletes it once read. Leave this type out of a list of connected accounts. `memoryCredentialStore().list` returns its `expiresAt`. diff --git a/.changeset/chat-resume-should-continue.md b/.changeset/chat-resume-should-continue.md new file mode 100644 index 0000000000..9179d7e862 --- /dev/null +++ b/.changeset/chat-resume-should-continue.md @@ -0,0 +1,5 @@ +--- +'@tanstack/ai': patch +--- + +`chat()` asks the middleware `onShouldContinue` after it runs the pending tool calls of a resume, before the next model call. A middleware that returns `false` there ends the run with no more model calls. The loop strategy is not asked at that point. diff --git a/.changeset/code-mode-harness.md b/.changeset/code-mode-harness.md index 6b2b43211d..c9c649240c 100644 --- a/.changeset/code-mode-harness.md +++ b/.changeset/code-mode-harness.md @@ -2,4 +2,4 @@ '@tanstack/ai-code-mode': minor --- -Add `@tanstack/ai-code-mode/harness` with `codeMode({ driver })`: a harness plugin that gives the model one `execute_typescript` tool. The model writes a program that calls several tools, and the program runs in the isolate of any `@tanstack/ai-isolate-*` driver. By default, only tools that are safe to run without a question move into code mode: server tools with no `needsApproval` that the permission rules allow in plan mode, and never a tool that declares permission resources. `include` picks the tools yourself, a safe tool with `metadata.codeMode: true` always moves, and `lazy: true` lists only the tool names and adds `discover_tools`. Tool names that are not JavaScript identifiers get a safe name inside the program. `@tanstack/ai-harness` is an optional peer dependency. +Add `@tanstack/ai-code-mode/harness` with `codeMode({ driver })`: a harness plugin that gives the model one `execute_typescript` tool. The model writes a program that calls several tools, and the program runs in the isolate of any `@tanstack/ai-isolate-*` driver. By default, only tools that are safe to run without a question move into code mode: server tools with no `needsApproval` that the permission rules allow in plan mode, and never a tool that declares permission resources. `include` picks the tools yourself, but it never moves a tool that needs approval or that the rules deny or ask about, a safe tool with `metadata.codeMode: true` always moves, and `lazy: true` lists only the tool names and adds `discover_tools`. Tool names that are not JavaScript identifiers get a safe name inside the program. The rules decide in the permission mode of the session, again at each turn, also for `include`. `@tanstack/ai-harness` is an optional peer dependency. diff --git a/.changeset/harness-acp-agent.md b/.changeset/harness-acp-agent.md index f860ff5c0a..e8684a7229 100644 --- a/.changeset/harness-acp-agent.md +++ b/.changeset/harness-acp-agent.md @@ -2,4 +2,6 @@ '@tanstack/ai-acp': minor --- -`@tanstack/ai-acp/agent` adds `createAcpAgent` and `serveAcp`, which serve a harness as an ACP v2 agent (experimental, like the draft protocol). Tool approvals become permission requests, cancel cancels the running turn, and image, audio, and embedded file blocks go to the media store of the session. `session/list` lists the sessions of the harness from the session index of the host. `@tanstack/ai-acp` now uses `@agentclientprotocol/sdk` 1.5. The existing ACP client adapters keep working. +`@tanstack/ai-acp/agent` adds `createAcpAgent` and `serveAcp`, which serve a harness as an ACP v2 agent (experimental, like the draft protocol). Each wait of a turn goes to the editor by its kind: an approval becomes a permission request (with `allow_always` only when the answer schema takes `always`), a question or a `human` wait becomes a form elicitation, and a sign-in becomes a URL elicitation, when the editor declares them in `initialize`. A wait that the editor cannot answer is cancelled. Cancel cancels the running turn, and image, audio, and embedded file blocks go to the media store of the session. A prompt takes its place in the queue before its media is saved, and gives the place back when the session is idle. `session/list` lists the sessions of the harness from the session index of the host. The `user` option is the user of the editor: each session opens as that user, and the list shows the sessions of that user. It is required on a host with `access`. The agent holds one session handle per ACP session. `session/close` releases it, and the close of the connection releases them all (a session whose prompt runs keeps its handle until that prompt ends), so the host closes each session on idle. `@tanstack/ai-acp` now uses `@agentclientprotocol/sdk` 1.8, pinned to `~1.8.0` because the agent uses its experimental v2 entry point. The existing ACP client adapters keep working. + +One agent app serves one connection, and reads the editor capabilities from that connection. A second connection gets a clear error. For more connections, make one app for each: `new AcpServer({ createAgent: () => createAcpAgent(options) })`. A session that the agent made or opened in its connection takes prompts after the agent released its handle, also on a host with no `stores.sessions`. diff --git a/.changeset/harness-cli-handler-options.md b/.changeset/harness-cli-handler-options.md deleted file mode 100644 index c2f77f3e99..0000000000 --- a/.changeset/harness-cli-handler-options.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -'@tanstack/ai-harness-cli': patch ---- - -`--serve` passes `canAccess` and `mediaSecret` to the harness handler, which needs both. diff --git a/.changeset/harness-forward-inputs.md b/.changeset/harness-forward-inputs.md deleted file mode 100644 index 27ae95bb0b..0000000000 --- a/.changeset/harness-forward-inputs.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -'@tanstack/ai-harness': patch ---- - -On a durable host with `stores.workClaims`, an input that reaches a host while another host owns the thread no longer takes the log from the running turn. With `stores.inbox`, the input waits in the inbox and the owner applies it (receipt `queued`). Without an inbox, it is refused with `owned_elsewhere`. A question from `ctx.session.ask` keeps its thread claimed, so an answer that reaches another host goes to the host that asked. diff --git a/.changeset/harness-join-any-instance.md b/.changeset/harness-join-any-instance.md index d7089c0ae3..dc47ad6053 100644 --- a/.changeset/harness-join-any-instance.md +++ b/.changeset/harness-join-any-instance.md @@ -2,8 +2,8 @@ '@tanstack/ai-harness': patch --- -On a durable host (`stores.log` and `stores.runs`), any server instance can serve a run. +Any server instance that reads the same log store can serve a run. - `POST run` sends the log cursor of each event as its SSE `id`. A client that is cut off sends the same request with `Last-Event-ID`, and any instance continues the stream from there. The turn does not run again. -- `GET run?runId=` finds a turn that another instance runs. It streams the turn from the shared log, from its start or from `Last-Event-ID`, until the turn ends. +- `GET run?runId=&threadId=` finds a turn that another instance runs. It streams the turn from the log of `threadId`, from its start or from `Last-Event-ID`, until the turn ends. It reads the log from the start of the turn, not from the start of the log. A turn rebuilt after a restart (a retried `POST run`) also streams from its start. Without `threadId`, only the instance that runs the turn finds it, and another instance answers `400`. The join request of a `ChatClient` sends only the run id, so put `threadId` in the URL of the connection. - `GET run?threadId=` shows a turn that another instance runs as `activeRun`, so a reloaded `useChat` joins it. diff --git a/.changeset/harness-log-newer-records.md b/.changeset/harness-log-newer-records.md index f614a2ed2e..671dc6ca55 100644 --- a/.changeset/harness-log-newer-records.md +++ b/.changeset/harness-log-newer-records.md @@ -2,4 +2,4 @@ '@tanstack/ai-harness': patch --- -A host that folds a `harness.*` log record type it does not know (a newer harness wrote it, for example during a rolling deploy) reads the log but refuses to write it, with `LogVersionError`. A fold checkpoint from a newer harness is ignored. +A host that folds a `harness.*` log record type it does not know (a newer harness wrote it, for example during a rolling deploy) reads the log but refuses to write it, with `LogVersionError`. A log snapshot from a newer harness is not used. diff --git a/.changeset/harness-recover-live-owner.md b/.changeset/harness-recover-live-owner.md deleted file mode 100644 index 7cdc56db50..0000000000 --- a/.changeset/harness-recover-live-owner.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -'@tanstack/ai-harness': patch ---- - -A durable host that opens a thread while another host holds its work claim no longer recovers that host's inputs, also an input that has no run lease yet. The crash sweep recovers them after the claim expires. diff --git a/.changeset/harness-redaction.md b/.changeset/harness-redaction.md new file mode 100644 index 0000000000..3db1857a62 --- /dev/null +++ b/.changeset/harness-redaction.md @@ -0,0 +1,12 @@ +--- +'@tanstack/ai-harness': minor +--- + +Keep the secrets of a session out of what it keeps and sends. + +- Each session replaces every secret that it gave out with `[redacted]`. These are the values of `run.keys` (also from env vars) and `run.credentials`, connector credentials and OAuth tokens, and secret answers (each string field of an answer with fields). It scrubs log records, feed events, plugin state (also in a `Map` or a `Set`), notes, the model context, and tool results, also after the own middleware of an agent run. It also scrubs the error frames of the HTTP handler, the WebSocket, and the stdio worker. It finds the raw, JSON-escaped, and URL-encoded forms, and skips values shorter than 6 characters. A secret split across two streaming deltas stays in those two events, and a secret that is JSON-escaped two times is not found. +- `scrubSecrets` from `@tanstack/ai-harness` is the one from `@tanstack/ai/byok`. +- A `step.do` value that holds a secret of the session throws, on the first run and on a replay, so the log never keeps a secret. Read the secret outside `step.do`. +- A retry of an input with a secret in it is a duplicate, not a conflict: the check compares the input without the secrets, as the log keeps it. +- The log keeps the final result of each tool call: the result after every `onAfterToolCall` hook (also the agent's own middleware in an agent run), scrubbed. A replacement from a hook wins, also on an error. Recovery after a crash gives the model that result. +- An answer to a `secret: true` question that reaches a host that does not own the log goes through the `credentials` store of the host, as a one-time reference with the credential type `secret_answer` and an `expiresAt` 24 hours later. Only the reference goes into the input lane, and the owner reads and deletes it. The references that nobody read go when the question ends. A kept answer for a run again goes when the run ends. Without `createHarnessHost({ credentials })`, the answer is rejected with `secret_answer_needs_credentials`. A used, expired, or unknown reference is rejected with `unknown_secret_answer`. `credentials.list` does not show the `secret_answer` type. diff --git a/.changeset/harness-secure-defaults.md b/.changeset/harness-secure-defaults.md index d8f7ec8847..6f187ee8e7 100644 --- a/.changeset/harness-secure-defaults.md +++ b/.changeset/harness-secure-defaults.md @@ -4,6 +4,7 @@ Make the transport defaults safe for a server with many instances. -- `createHarnessHandler` and `handleHarnessSocket` need `canAccess`. To let every principal use every thread, pass `canAccess: () => true`. -- On a durable host (`stores.log`), `createHarnessHandler` throws without `mediaSecret`, so a media URL that one instance signs works on every instance. +- `createHarnessHandler` and `handleHarnessSocket` read the user of each request with the host `access`. On a host without `access`, pass `openAccess: true` (no user) or `openAccess: { user }` (one local user), else they throw at the start. So a server never serves every thread to everyone by mistake. +- `media: { secret }` signs the media URLs of `createHarnessHandler`. Set the same secret on every instance, so a media URL that one instance signs works on every instance. Without it, the handler signs with a random key of its own, and a URL works only on that instance. A host with `ownership` options needs `media`, else the handler throws at the start. On another host without `media`, the first handler logs one warning. A signed URL names its harness and its user, and the handler checks that user again when it serves the file. A URL of a deleted thread answers 404. +- `media: { keys: [newKey, oldKey] }` rotates the key: the first key signs, and every key checks. Each key must be at least 32 bytes, else the handler throws at the start. - `oauthConnector` needs `login`. Use `'device'` on a server, and `'loopback'` only for a harness on the user's own machine, such as a CLI. diff --git a/.changeset/harness-skills.md b/.changeset/harness-skills.md index de7f2585e5..365b90e0c1 100644 --- a/.changeset/harness-skills.md +++ b/.changeset/harness-skills.md @@ -2,4 +2,4 @@ '@tanstack/ai-skills': minor --- -`@tanstack/ai-skills/harness`: the new `skills({ dirs })` plugin gives the harness model the skills of a list of folders, and makes each skill a `/` command that starts a turn with that skill. A skill with the name of another command is `/skill:`, and `/skills` lists them. The plugin watches the folders, so a skill that you add or remove while the session runs changes the commands at once. +`@tanstack/ai-skills/harness`: the new `skills({ dirs })` plugin gives the harness model the skills of a list of folders, and makes each skill a `/` command that starts a turn with that skill. A skill with the name of another command is `/skill:`, and `/skills` lists them. The plugin watches the folders, so a skill that you add or remove while the session runs changes the commands at once. The watcher lives as long as the host, every session that uses the same `skills()` object shares it, and a change makes each session read the folders again. A skill that does not load gets a fallback message. diff --git a/.changeset/harness-tool-ops.md b/.changeset/harness-tool-ops.md index 0e8d098ef3..2ddc4d7494 100644 --- a/.changeset/harness-tool-ops.md +++ b/.changeset/harness-tool-ops.md @@ -6,7 +6,7 @@ Run a harness tool with no model turn, add host records from a client, and add a per-run system preamble. -- `session.tool(name, args?, options?)` runs one harness tool with no model +- `session.runTool({ name, args?, meta?, inputId?, user? })` runs one harness tool with no model turn, for example from a schedule or a webhook. The call goes through the same middleware, input check, and `permissions()` rules as a model call, and the call and its result join the transcript. A tool that asks for approval @@ -15,17 +15,17 @@ Run a harness tool with no model turn, add host records from a client, and add a event. `args` is typed from the input schema of a harness tool. - `expose.tools` lists the harness tools a client may run with a `{ op: 'tool', name, args?, meta? }` input, or with the new - `client.tool(name, args?, options?)`. The list is typed from the harness + `client.runTool({ name, args?, meta?, inputId? })`. The list is typed from the harness `tools`, and `defineHarness` throws for a name that is not in `tools`. Any other tool is refused with `not_exposed`. The capabilities document marks each tool with `exposed`. - `expose.records` lists the host record types a client may add with an - `{ op: 'append', records }` input, or with the new `client.append(records)`. + `{ op: 'append', records }` input, or with the new `client.appendRecords({ records })`. A record of another type, or with a `thread` field, is refused with - `not_exposed`. The host must be durable (with `stores.log`). + `not_exposed`. The session log keeps the records. - `createHarnessMcpServer` adds a `tool_` tool for each tool in `expose.tools`, and an `append` tool when `expose.records` has types. A tool - input with its own `threadId` field now throws when the server is created, + input with its own `threadId` field throws when the server is created, because MCP uses `threadId` to pick the thread. - `session.send(message, { systemPreamble })` prepends per-run, server-only system prompts ahead of the harness's own system prompts (for example diff --git a/.changeset/harness-turn-and-body-limits.md b/.changeset/harness-turn-and-body-limits.md new file mode 100644 index 0000000000..e1c3938cc5 --- /dev/null +++ b/.changeset/harness-turn-and-body-limits.md @@ -0,0 +1,14 @@ +--- +'@tanstack/ai-harness': patch +--- + +How a turn takes steers, settings, and plugin removals, and the size limit of client requests. + +- A steer that arrives during a `session.runTool()` turn waits, and then runs as its own turn. +- A turn keeps the config values that it started with. A config change while it runs, for example `/mode bypass`, applies from the next turn. +- `configure({ settings: { plugins: { remove } } })` removes everything that those plugins give the thread: also their agents, subagents, commands, agent and generation middleware, and the items that they add for other plugins (for example the permission rules of `agents()`). A remove that leaves a plugin (of any lifetime) or a harness middleware without a capability that it needs is rejected. +- When two plugins give a tool with the same name from `discoverTools`, the first one stays, and clients get one `harness.plugin.warning` event that names both plugins. +- `createHarnessHandler` reads each JSON body up to `media.maxBodyBytes` (default 32 MiB). A bigger body answers 413. `handleHarnessSocket` closes the socket with code 1009 on a bigger frame. The WebSocket server reads the whole frame first, so also set its own frame limit (for example `maxPayload` of `ws`). +- Client media is checked part by part before any part is stored, so a refused input stores nothing. A retried input with the same `inputId` stores its media no second time, and gets the first receipt. +- A client input with a provider file handle (`source.type: 'file'`) is refused with `media_url_not_allowed` unless the harness sets `media.allowProviderFiles: true`. Server code can still send file handles. +- `POST run` moves the inline bytes of the last user message to the media store, as `POST control` does, so the transcript keeps a `harness-media:` reference. A refused part answers 413 (`media_too_large`), 415 (`media_not_allowed`), or 400 (`media_invalid`, `media_url_not_allowed`). diff --git a/.changeset/harness-view-experimental.md b/.changeset/harness-view-experimental.md index cdb14f7fa5..700a912b5c 100644 --- a/.changeset/harness-view-experimental.md +++ b/.changeset/harness-view-experimental.md @@ -2,4 +2,4 @@ '@tanstack/ai-harness': patch --- -Mark `createSessionView`, `SessionViewState`, and `SessionViewEvents` as `@experimental`. Document that event cursors are opaque strings: log positions on a durable host, and process-local counters without a log store. +Mark `createSessionView`, `SessionViewState`, and `SessionViewEvents` as `@experimental`. Document that event cursors are opaque strings: positions in the session log. diff --git a/.changeset/persistence-resume-tool-results.md b/.changeset/persistence-resume-tool-results.md index 032f553bfd..41a016b7c7 100644 --- a/.changeset/persistence-resume-tool-results.md +++ b/.changeset/persistence-resume-tool-results.md @@ -3,3 +3,5 @@ --- Store a tool result once when a run resumes from a client-tool interrupt. The client posts earlier tool results under interrupt snapshot ids the store never saw, so `withPersistence` appended a second copy and the next provider request failed. + +`withPersistence` lets a subagent that a tool call started check the answers to its own interrupts. The parent run does not check them against its own tools and batch, so a child client tool, and two children that wait at once, resume. diff --git a/.changeset/persistence-routed-host-after-tool-call.md b/.changeset/persistence-routed-host-after-tool-call.md new file mode 100644 index 0000000000..b176911c9b --- /dev/null +++ b/.changeset/persistence-routed-host-after-tool-call.md @@ -0,0 +1,5 @@ +--- +'@tanstack/ai-persistence': patch +--- + +Keep the answer of a routed agent in its own message when the turn resumes after an interrupt and a finished tool call (for example a handoff) comes before that answer. Before, the stored answer moved onto the message of the tool call. diff --git a/.changeset/persistence-session-stores.md b/.changeset/persistence-session-stores.md index 47f18372ea..470f8f1c16 100644 --- a/.changeset/persistence-session-stores.md +++ b/.changeset/persistence-session-stores.md @@ -4,14 +4,11 @@ Add opt-in session stores. The chat and generation middleware work as before without them. -- `inbox` (`InboxStore`): the inputs a host accepted, with an idempotent `append` by `inputId`, `listPending`, `markApplied`, `markRejected`, and `get`. -- `log` (`LogStore`): an append-only log per thread. `append` writes a batch at a position, all or nothing, and rejects with `LogConflictError` when another writer took that position. `read` and `subscribe` complete it. `memoryLogStore()` is the reference store. It is not part of `memoryPersistence()`. -- `leases` (`LeaseStore`, `TurnLease`, `TurnLeaseKey`): turn leases from a system that already has them. `defineLeaseStore()` types a custom one, `memoryLeaseStore()` is the reference store, and the conformance testkit covers it. It is not part of `memoryPersistence()`. -- `workClaims` (`WorkClaimStore`): claims on threads that have pending work, with `claim`, `release`, and `listExpired`. -- `sessions` (`SessionIndexStore`): one index entry per thread, with `upsert`, `get`, `list`, and `delete`. `list` filters by parent thread, owner, title `search`, `harness` name, and exact `metadata` values, and pages with a cursor. The optional `getVersioned` and `upsertIf` give compare-and-set writes. +- `log` (`LogStore`): an append-only log for each session tree, with 15 methods, each with one object argument. Records: `append`, `read`, and `subscribe`. Ownership: `acquireOwnership`, `renewOwnership`, `releaseOwnership`, `pinToWorkers`, `markHostAlive`, and `listLogsWithoutOwner`. A pin (`pinToWorkers`) counts only while its host is alive: `acquireOwnership` and `listLogsWithoutOwner` ignore the affinity of a host with no live `heartbeat({ hostId, ttlMs })`, so the logs of a host that stopped go to other hosts. A host that is alive keeps its pins, also on the logs that it released. The input lane: `offerInput` and `listOfferedInputs`, consumed in the same write with `append({ consumes })`. The owner or the first offer names the harness of a log for `listLogsWithoutOwner`: a later offer does not change it. Snapshots and lifecycle: `getSnapshot`, `putSnapshot`, `truncate`, and `deleteLog`. `acquireOwnership` returns `expired: true` when it takes the log from another owner whose ownership expired with no release. Each fenced write carries the epoch from `acquireOwnership`, and a write with an old epoch rejects with `OwnershipLostError`. A writer that never calls `acquireOwnership` appends with `NO_OWNER_EPOCH`. `pinToWorkers`, `putSnapshot`, and `truncate` need the owner, so they refuse `NO_OWNER_EPOCH`. After `deleteLog`, every fenced write fails until the next `acquireOwnership`, and `offerInput` returns the new `OfferResult` `'deleted'` and stores nothing, so a late write or input cannot bring a deleted log back. `append` rejects with `LogConflictError` when another writer took the position, and `read` rejects with `LogTruncatedError` before the truncation point. `memoryLogStore()` is the reference store. `memoryLogStore({ state })` with one `memoryLogState()` lets two store instances share one log. The conformance kit needs `twin`, a second store set with a new log store instance on the same backend, for the `log` cases across instances. +- `sessions` (`SessionIndexStore`): one index entry per tenant and thread. The key is `{ tenantId?, threadId }` (`SessionIndexKey`), so one thread id in two tenants is two entries. An entry has `members` (`SessionMember`, with the role `owner`, `member`, `viewer`, or `operator`). The owner is the member with the role `owner`. The store has `get(key)`, `getVersioned(key)`, `upsert(entry)`, `upsertIf({ entry, revision })` for compare-and-set writes, `list`, and `delete(key)`, a hard delete. `list` reads one tenant, filters by member, parent thread, title `search`, `harness` name, and exact `metadata` values, and pages with a cursor. -`defineInboxStore`, `defineLogStore`, `defineWorkClaimStore`, and `defineSessionIndexStore` type an adapter. `memoryPersistence()` now also has `inbox`, `workClaims`, and `sessions`. `runPersistenceConformance` runs the cases of each session store when the store is present, so an existing backend needs no `skip` entry. +`defineLogStore` and `defineSessionIndexStore` type an adapter. `memoryPersistence()` now also has `log` and `sessions`. `runPersistenceConformance` runs the cases of each session store when the store is present, so an existing backend needs no `skip` entry. -With `stores.sessions`, `withPersistence` writes one index entry for each subagent child (`subagent:`), with the parent thread, the tool call, and the owner and `harness` name of the parent entry. +With `stores.sessions`, `withPersistence` writes one index entry for each subagent child (`subagent:`), in the tenant of its `tenantId` option. The entry has the parent thread, the tool call, and the `harness` name of the parent entry. It has no members: access to a child goes through its top-level thread. `loadChildTranscript(messages, subagentRunId, threadId)` reads the stored messages of a child agent run for a UI. It returns `undefined` for a child of another thread. diff --git a/.changeset/persistence-store-shapes.md b/.changeset/persistence-store-shapes.md deleted file mode 100644 index c448f320e4..0000000000 --- a/.changeset/persistence-store-shapes.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -'@tanstack/ai-persistence': patch ---- - -`InboxStore.get`, `markApplied`, and `markRejected` take the thread id first, so a store can key an entry by thread and input id. `InboxStore.delete` removes an entry: a harness host deletes a forwarded answer after it applies it, so a secret answer does not stay in the inbox. `WorkClaimStore.get(threadId)` returns the owner of a claim and when it expires. The session store docs allow a content-hash revision (an ETag of S3 or Netlify Blobs) that comes back for the same entry. diff --git a/.changeset/sandbox-local-process-env.md b/.changeset/sandbox-local-process-env.md new file mode 100644 index 0000000000..ca17bde66c --- /dev/null +++ b/.changeset/sandbox-local-process-env.md @@ -0,0 +1,5 @@ +--- +'@tanstack/ai-sandbox-local-process': minor +--- + +A command in `localProcessSandbox` gets only a few variables of the host process: `PATH`, `HOME`, `LANG`, `LC_ALL`, `TMPDIR`, `TEMP`, `TMP`, and on Windows `SystemRoot`, `ComSpec`, `PATHEXT`, `WINDIR`, and `USERPROFILE`. The workspace secrets and the env of the call still reach it. Provider keys and other secrets of the server do not. If a command needs another variable of the host, set it as a workspace secret or in the env of the call. `passEnv` names more variables of the host for each command, for example `HTTPS_PROXY` or `CLAUDE_CONFIG_DIR`. `scrubEnv` still removes a variable from the env of each command. diff --git a/.changeset/subagent-context-extend.md b/.changeset/subagent-context-extend.md index ac193c83aa..34239c85b3 100644 --- a/.changeset/subagent-context-extend.md +++ b/.changeset/subagent-context-extend.md @@ -5,4 +5,4 @@ A host can add fields to the `ctx` an agent's `run` gets. - `subagents.binding.extend(ctx)` returns the fields to add. A child's `ctx.chat({ subagents })` passes it down, so the fields reach nested children too. -- `SubagentContextExtension` is an empty interface that `SubagentRunContext` includes. A package augments it to type its fields, for example `ctx.harness` from `@tanstack/ai-harness`. +- `SubagentContextExtension` is an empty interface that `SubagentRunContext` includes. A package augments it to type its fields, for example `ctx.harness` from `@tanstack/ai-harness`. Make each field optional, because a run that a plain `chat()` starts has no host. diff --git a/docs/advanced/middleware.md b/docs/advanced/middleware.md index c52d5e90ea..6a843a8a9f 100644 --- a/docs/advanced/middleware.md +++ b/docs/advanced/middleware.md @@ -734,6 +734,38 @@ The `hookCtx` provides: | `toolName` | `string` | Tool name | | `toolCallId` | `string` | Tool call ID | +### onGuardToolCall + +A permission check in `onBeforeToolCall` can miss calls. An earlier middleware can return a decision first, and then your hook does not run. Put checks that must always run in `onGuardToolCall`: + +```typescript +import { type ChatMiddleware } from "@tanstack/ai"; + +const noDeletes: ChatMiddleware = { + name: "no-deletes", + onGuardToolCall: (ctx, hookCtx) => { + if (hookCtx.toolName === "deleteFile") { + return { type: "skip", result: { error: "Deleting files is not allowed." } }; + } + }, +}; +``` + +When the guard runs: + +- It runs for every tool call, after all `onBeforeToolCall` hooks and after the `inputSchema` check. +- `hookCtx.args` holds the final checked arguments. The tool gets these same arguments. +- The guard of every middleware runs, in array order, until one returns a decision. +- It runs just before a server tool runs, and before a client tool goes to the client. A call that waits for approval does not run it. + +| Decision | Effect | +|----------|--------| +| `void` / `undefined` | Allow the call. The next guard runs. | +| `{ type: 'skip', result }` | Deny the call. The tool does not run, and the model gets `result`, the same as an `onBeforeToolCall` skip. | +| `{ type: 'abort', reason? }` | Abort the entire chat run | + +A guard cannot change the arguments. To change them, use `onBeforeToolCall`. + ### onAfterToolCall Called after each tool execution (or skip). Every middleware runs, in array order. @@ -1025,6 +1057,7 @@ const stream = chat({ | `onStart` | Sequential | All run in order | | `onChunk` | **Piped** — chunks flow through each middleware | If first drops a chunk, later middleware never see it | | `onBeforeToolCall` | **First-win** — first non-void decision wins | Earlier middleware has priority | +| `onGuardToolCall` | Sequential. Every guard runs until one returns a decision | An earlier `onBeforeToolCall` decision never skips a guard | | `onShouldContinue` | **AND** — any explicit `false` stops the loop | Order only affects which middleware runs first when short-circuiting | | `onModelError` | Sequential. If any middleware calls `ctx.retry()` and the call has retries left, the model is called again | All run in order, so each one sees the error | | `onAfterToolCall` | **Piped**: each gets the result of the previous `replaceResult` | The last replacement wins | diff --git a/docs/advanced/server-keys.md b/docs/advanced/server-keys.md index d8e71f0f40..5918826a34 100644 --- a/docs/advanced/server-keys.md +++ b/docs/advanced/server-keys.md @@ -146,7 +146,7 @@ Your users now save a key once, and every device and job uses it. ## Write a store for production -A `CredentialStore` has four methods. You decide where the secrets live: +A `CredentialStore` has four methods, and two optional ones for [compare-and-set](#add-compare-and-set). You decide where the secrets live: - A `credentials` table in the database that you already use, next to your chat tables. - A secrets manager, for example AWS Secrets Manager or Vault. @@ -156,6 +156,7 @@ import type { CredentialScope } from "@tanstack/ai/byok"; type Credential = | { type: "api_key"; value: string } + | { type: "secret_answer"; value: string; expiresAt: number } // epoch ms | { type: "oauth"; accessToken: string; @@ -181,6 +182,17 @@ interface CredentialStore { list( scope: CredentialScope, ): Promise>; + // Optional: compare-and-set. + getVersioned?(key: { + scope: CredentialScope; + id: string; + }): Promise<{ credential: Credential; revision: string } | null>; + setIf?(entry: { + scope: CredentialScope; + id: string; + credential: Credential; + expectedRevision: string | null; + }): Promise<{ ok: true; revision: string } | { ok: false; reason: "conflict" }>; } ``` @@ -192,12 +204,51 @@ Rules for the keys: - A user credential is not visible at the tenant scope. A tenant credential is not visible at the user scope. - A scope with no tenant and no user is one bucket that every caller shares. Get the scope from your server-side auth, never from the request body. - `client` is the OAuth client that the tokens belong to. A token refresh needs it. +- A `secret_answer` row is not a user credential. The harness keeps a secret answer there for a short time. Leave this type out of a list of connected accounts. You can delete a row after its `expiresAt`. Rules for the secrets (`value`, `accessToken`, `refreshToken`, `client.clientSecret`): - Encrypt them at rest. Keep the encryption key in a KMS or a server-only env var, not in the same database. - `list` returns ids and types only. It never returns a secret, not even masked. - `delete` removes the row. Do not keep a soft-deleted copy of the secret. -- Do not write a secret to a log, an error message, or a trace. +- Do not write a secret to a log, an error message, or a trace. To remove known secrets from a text, use `scrubSecrets(text, secrets)` from `@tanstack/ai/byok`. The [harness](../harness/provider-keys) uses the same store: pass it to `createHarnessHost({ credentials })`. + +### Add compare-and-set + +Two servers can refresh the same OAuth token at the same time. If the service rotates refresh tokens, the second save then keeps a token that no longer works. Add `getVersioned` and `setIf`, so the second save does not remove the first: + +- `getVersioned` returns the credential and its revision, or `null` when the key is absent. +- `setIf` writes only when the stored revision equals `expectedRevision`. Pass `null` to write only when the key is absent. Success returns `{ ok: true, revision }`. A different revision returns `{ ok: false, reason: "conflict" }` and does not write. +- Every write gives the key a new revision. A plain `set` does this too. After `delete`, the deleted revision is never used again. +- Add both methods or neither. `memoryCredentialStore()` has both. + +To refresh a token in your own route, read it with `readCredentialEntry`. It finds the credential like `readCredential` (the user first, then the tenant), and also returns the scope where it found it and its `revision`: + +```typescript +import { readCredentialEntry } from "@tanstack/ai/byok/server"; +import type { Credential, CredentialStore } from "@tanstack/ai/byok"; + +export async function refreshGithub( + store: CredentialStore, + scope: { tenantId: string; userId: string }, + refresh: (current: Credential) => Promise, +) { + const entry = await readCredentialEntry(store, scope, "github"); + if (!entry) return null; + const next = await refresh(entry.credential); + // Save to the scope where the credential was, also when it is the tenant's. + const key = { scope: entry.scope, id: "github", credential: next }; + if (!store.setIf || entry.revision === undefined) { + await store.set(key); + return next; + } + const saved = await store.setIf({ ...key, expectedRevision: entry.revision }); + if (saved.ok) return next; + // Another server saved first. Use its token, and do not refresh again. + return (await readCredentialEntry(store, scope, "github"))?.credential ?? null; +} +``` + +In a harness, `run.credentials.refresh` does this for you. See [Refresh a token](../harness/auth#refresh-a-token). diff --git a/docs/api/ai.md b/docs/api/ai.md index 4e125f4ee4..965bf8cd1b 100644 --- a/docs/api/ai.md +++ b/docs/api/ai.md @@ -567,9 +567,12 @@ import { maskKey } from "@tanstack/ai/byok"; maskKey("sk-abcdefghij"); // "ghij" ``` -## `scrubSecrets(input, secrets)` +## `scrubSecrets(text, secrets)` -Replace each listed secret in `input` with `[redacted]`. Use this before you log an error string. +Replace each listed secret in `text` with `[redacted]`. Use this before you log an error string. + +- It also replaces the JSON-escaped and the URL-encoded form of each secret. +- It skips a secret shorter than 6 characters, so a short value does not remove common words. ```typescript import { scrubSecrets } from "@tanstack/ai/byok"; diff --git a/docs/chat/subagents.md b/docs/chat/subagents.md index 15f9c24634..88f74a25aa 100644 --- a/docs/chat/subagents.md +++ b/docs/chat/subagents.md @@ -330,7 +330,7 @@ Every agent tool takes the agent's own fields, plus two more: - The fields of `inputSchema`, for an agent with one. The tool checks them against the schema. - `prompt`: the task as text, for an agent without `inputSchema`. The child gets it as the last user message. - `sessionId`: the `subagentRunId` of an earlier result. The call continues that child. The tool has this field only when `withPersistence` is on the parent `chat()`. -- `background`: `true` starts the child and returns its `subagentRunId` at once. The tool has this field only when the host sets `subagents.binding.start`. The host returns `done`. `maxConcurrent` counts that child until `done` settles. +- `background`: `true` starts the child and returns its `subagentRunId` at once. The tool has this field only when a host that runs agents for you, such as a [harness](../harness/subagents), turns it on. `maxConcurrent` counts that child until it ends. When the run uses `sessionId` or `background`, that name is reserved. If an `inputSchema` has a field with that name, `chat()` throws before the run starts. diff --git a/docs/config.json b/docs/config.json index 5eded52077..79225eab5e 100644 --- a/docs/config.json +++ b/docs/config.json @@ -160,7 +160,7 @@ "label": "Subagents", "to": "chat/subagents", "addedAt": "2026-09-21", - "updatedAt": "2026-10-14" + "updatedAt": "2026-10-19" }, { "label": "Agentic Cycle", @@ -353,7 +353,7 @@ "label": "Middleware", "to": "advanced/middleware", "addedAt": "2026-04-15", - "updatedAt": "2026-10-14" + "updatedAt": "2026-10-17" }, { "label": "Built-in Middleware", @@ -712,7 +712,7 @@ "label": "Build Your Own Adapter", "to": "persistence/build-your-own-adapter", "addedAt": "2026-08-04", - "updatedAt": "2026-10-16" + "updatedAt": "2026-10-17" }, { "label": "Migrations", @@ -761,7 +761,7 @@ "label": "Store Reference", "to": "persistence/store-reference", "addedAt": "2026-08-04", - "updatedAt": "2026-10-16" + "updatedAt": "2026-10-20" }, { "label": "How Persistence Works", @@ -927,35 +927,54 @@ "label": "Build your first harness", "to": "harness/overview", "addedAt": "2026-10-13", - "updatedAt": "2026-10-16" + "updatedAt": "2026-10-20" + }, + { + "label": "Host options", + "to": "harness/host-options", + "addedAt": "2026-10-20" }, { "label": "Send and show media", "to": "harness/media", "addedAt": "2026-10-13", - "updatedAt": "2026-10-16" + "updatedAt": "2026-10-20" }, { "label": "Connect clients", "to": "harness/connect", "addedAt": "2026-10-13", - "updatedAt": "2026-10-16" + "updatedAt": "2026-10-20" + }, + { + "label": "Write a client in any language", + "to": "harness/protocol", + "addedAt": "2026-10-17", + "updatedAt": "2026-10-19" + }, + { + "label": "Access and tenants", + "to": "harness/access", + "addedAt": "2026-10-17", + "updatedAt": "2026-10-20" }, { "label": "Run in the terminal", "to": "harness/cli", - "addedAt": "2026-10-13" + "addedAt": "2026-10-13", + "updatedAt": "2026-10-19" }, { "label": "Deploy a harness", "to": "harness/deploy", - "addedAt": "2026-10-13" + "addedAt": "2026-10-13", + "updatedAt": "2026-10-20" }, { "label": "Send inputs safely", "to": "harness/inputs", "addedAt": "2026-10-13", - "updatedAt": "2026-10-16" + "updatedAt": "2026-10-20" }, { "label": "Run a tool without a model", @@ -966,166 +985,205 @@ "label": "Durable sessions", "to": "harness/durable-sessions", "addedAt": "2026-10-13", - "updatedAt": "2026-10-15" + "updatedAt": "2026-10-20" + }, + { + "label": "Run background work on workers", + "to": "harness/workers", + "addedAt": "2026-10-17", + "updatedAt": "2026-10-19" }, { "label": "List, rename, and fork sessions", "to": "harness/sessions", "addedAt": "2026-10-13", - "updatedAt": "2026-10-16" + "updatedAt": "2026-10-20" + }, + { + "label": "Lifecycle and retention", + "to": "harness/lifecycle", + "addedAt": "2026-10-17", + "updatedAt": "2026-10-20" }, { "label": "Run side effects once", "to": "harness/durable-tools", - "addedAt": "2026-10-13" + "addedAt": "2026-10-13", + "updatedAt": "2026-10-20" + }, + { + "label": "Wait for a person", + "to": "harness/waits", + "addedAt": "2026-10-18", + "updatedAt": "2026-10-20" }, { "label": "Build on the session log", "to": "harness/session-log", "addedAt": "2026-10-13", - "updatedAt": "2026-10-16" + "updatedAt": "2026-10-19" }, { "label": "Control how a turn ends", "to": "harness/turn-control", "addedAt": "2026-10-13", - "updatedAt": "2026-10-16" + "updatedAt": "2026-10-17" }, { "label": "Store settings per thread", "to": "harness/thread-settings", - "addedAt": "2026-10-13" + "addedAt": "2026-10-13", + "updatedAt": "2026-10-18" }, { "label": "Compact a long session", "to": "harness/compaction", - "addedAt": "2026-10-13" + "addedAt": "2026-10-13", + "updatedAt": "2026-10-17" }, { "label": "Fork and reset a thread", "to": "harness/fork-and-reset", "addedAt": "2026-10-13", - "updatedAt": "2026-10-16" + "updatedAt": "2026-10-18" }, { - "label": "Share one log between sessions", + "label": "One log for a session tree", "to": "harness/shared-logs", - "addedAt": "2026-10-13" + "addedAt": "2026-10-13", + "updatedAt": "2026-10-20" }, { "label": "Write a plugin", "to": "harness/plugins", "addedAt": "2026-10-13", - "updatedAt": "2026-10-16" + "updatedAt": "2026-10-20" }, { "label": "Track usage and cost", "to": "harness/usage", "addedAt": "2026-10-13", - "updatedAt": "2026-10-16" + "updatedAt": "2026-10-18" }, { "label": "Ask before risky tool calls", "to": "harness/permissions", "addedAt": "2026-10-13", - "updatedAt": "2026-10-16" + "updatedAt": "2026-10-20" }, { "label": "Give your agent coding tools", "to": "harness/coding-tools", "addedAt": "2026-10-13", - "updatedAt": "2026-10-14" + "updatedAt": "2026-10-18" }, { "label": "Sandbox and extend the coding tools", "to": "harness/coding-tools-backends", "addedAt": "2026-10-13", - "updatedAt": "2026-10-16" + "updatedAt": "2026-10-19" + }, + { + "label": "Run the coding tools for many threads", + "to": "harness/coding-workspaces", + "addedAt": "2026-10-18", + "updatedAt": "2026-10-20" }, { "label": "Undo a turn", "to": "harness/snapshots", "addedAt": "2026-10-13", - "updatedAt": "2026-10-13" + "updatedAt": "2026-10-19" }, { "label": "Build a coding agent", "to": "harness/coding-agent", "addedAt": "2026-10-13", - "updatedAt": "2026-10-14" + "updatedAt": "2026-10-18" }, { "label": "Delegate to coding agents", "to": "harness/coding-agents", - "addedAt": "2026-10-13" + "addedAt": "2026-10-13", + "updatedAt": "2026-10-18" }, { "label": "Switch between agent profiles", "to": "harness/agents", "addedAt": "2026-10-13", - "updatedAt": "2026-10-16" + "updatedAt": "2026-10-18" }, { "label": "Auth and connectors", "to": "harness/auth", "addedAt": "2026-10-13", - "updatedAt": "2026-10-16" + "updatedAt": "2026-10-19" }, { "label": "Share a thread between people", "to": "harness/shared-threads", "addedAt": "2026-10-13", - "updatedAt": "2026-10-16" + "updatedAt": "2026-10-20" }, { "label": "Connect model providers", "to": "harness/provider-keys", "addedAt": "2026-10-13", - "updatedAt": "2026-10-16" + "updatedAt": "2026-10-20" }, { "label": "Build your own UI", "to": "harness/custom-ui", "addedAt": "2026-10-13", - "updatedAt": "2026-10-16" + "updatedAt": "2026-10-19" }, { "label": "Code mode in a harness", "to": "harness/code-mode", - "addedAt": "2026-10-13" + "addedAt": "2026-10-13", + "updatedAt": "2026-10-18" }, { "label": "Work until a goal is met", "to": "harness/goal", - "addedAt": "2026-10-13" + "addedAt": "2026-10-13", + "updatedAt": "2026-10-18" }, { "label": "Run agents from a harness", "to": "harness/subagents", "addedAt": "2026-10-13", - "updatedAt": "2026-10-16" + "updatedAt": "2026-10-20" + }, + { + "label": "Hand off a conversation", + "to": "harness/handoffs", + "addedAt": "2026-10-18", + "updatedAt": "2026-10-20" }, { "label": "Add skills", "to": "harness/skills", - "addedAt": "2026-10-13" + "addedAt": "2026-10-13", + "updatedAt": "2026-10-20" }, { "label": "Use MCP servers", "to": "harness/mcp", "addedAt": "2026-10-13", - "updatedAt": "2026-10-16" + "updatedAt": "2026-10-20" }, { "label": "Use from any MCP client", "to": "harness/mcp-server", "addedAt": "2026-10-13", - "updatedAt": "2026-10-16" + "updatedAt": "2026-10-18" }, { "label": "Use a harness from an editor (ACP)", "to": "harness/acp", - "addedAt": "2026-10-13" + "addedAt": "2026-10-13", + "updatedAt": "2026-10-18" } ] }, @@ -1149,7 +1207,7 @@ "label": "Providers", "to": "sandbox/providers", "addedAt": "2026-06-29", - "updatedAt": "2026-10-07" + "updatedAt": "2026-10-19" }, { "label": "Harnesses", @@ -1160,7 +1218,8 @@ { "label": "Harness Auth", "to": "sandbox/auth", - "addedAt": "2026-08-18" + "addedAt": "2026-08-18", + "updatedAt": "2026-10-19" }, { "label": "Workspace", @@ -1344,7 +1403,8 @@ { "label": "Keep Keys on the Server", "to": "advanced/server-keys", - "addedAt": "2026-10-16" + "addedAt": "2026-10-16", + "updatedAt": "2026-10-19" }, { "label": "Debug Logging", @@ -1694,7 +1754,7 @@ "label": "@tanstack/ai", "to": "api/ai", "addedAt": "2026-04-15", - "updatedAt": "2026-10-13" + "updatedAt": "2026-10-17" }, { "label": "@tanstack/ai-client", diff --git a/docs/harness/access.md b/docs/harness/access.md new file mode 100644 index 0000000000..4b67c2ce18 --- /dev/null +++ b/docs/harness/access.md @@ -0,0 +1,294 @@ +--- +title: Access and tenants +id: harness-access +order: 3 +description: "Decide who can use each session of a harness. Read the user of each request, check each action, and keep the threads of each organization apart." +keywords: + - tanstack ai + - harness + - access + - getUser + - isAllowed + - tenantId + - membersOnly +--- + +Your harness serves many users. Ada must not read the threads of Bob, and the company Acme must never see a thread of the company Globex. Give the host `access`. Then the host reads the user of each request, asks you about each action, and keeps each organization apart. + +A user here is `{ id, name?, tenantId? }`: who a call acts for. Its type is `HarnessUser`. + +## 1. Turn on access + +Give `createHarnessHost` an `access` object and a `sessions` store. The store keeps the owner and the members of each session: + +```ts group=harness-access +import { createHarnessHost, defineHarness, membersOnly } from '@tanstack/ai-harness' +import { createHarnessHandler } from '@tanstack/ai-harness/server' +import { memoryPersistence } from '@tanstack/ai-persistence' +import { openaiText } from '@tanstack/ai-openai' + +export const assistant = defineHarness({ + name: 'acme/assistant', + adapter: openaiText('gpt-6.1-sol'), +}) + +// Your own login check: a session token to a user. +const users = new Map([ + ['Bearer token-ada', { id: 'ada', tenantId: 'acme' }], + ['Bearer token-gus', { id: 'gus', tenantId: 'globex' }], +]) + +export const host = createHarnessHost({ + persistence: memoryPersistence(), + access: { + getUser: ({ request }) => users.get(request.headers.get('authorization') ?? '') ?? null, + rules: membersOnly(), + }, +}) + +export const handler = createHarnessHandler({ host, harness: assistant }) +``` + +- `getUser` reads the user of a request with your own login check. Return `null` to refuse the request with `401`. +- `rules` decides each action. `membersOnly()` gives ready rules: only the members of a session can use it. See [Share a thread between people](./shared-threads). +- `memoryPersistence()` has a `sessions` store. Without `stores.sessions`, `createHarnessHost` throws. + +The handler gets the user from `getUser`, so it needs no login option of its own. A new thread belongs to the user who opens it first. + +## 2. Write your own rule + +Give `access` your own `rules` with an `isAllowed` function. `isAllowed` gets the user, the action, and the session. Return `false` to refuse. The user has the type that `getUser` returns, so your own fields are typed too: + +```ts group=harness-access-rule +import { createHarnessHost } from '@tanstack/ai-harness' +import { memoryPersistence } from '@tanstack/ai-persistence' + +interface User { + id: string + tenantId: string + isAdmin: boolean +} + +const users = new Map([ + ['Bearer token-ada', { id: 'ada', tenantId: 'acme', isAdmin: false }], + ['Bearer token-eve', { id: 'eve', tenantId: 'acme', isAdmin: true }], +]) + +export const host = createHarnessHost({ + persistence: memoryPersistence(), + access: { + getUser: ({ request }) => users.get(request.headers.get('authorization') ?? '') ?? null, + rules: { isAllowed: ({ user, action, session }) => { + const isOwner = session.members.some( + (member) => member.userId === user.id && member.role === 'owner', + ) + if (isOwner || user.isAdmin) return true + // Everyone in the organization can read every thread of it. + return action === 'read' || action === 'list' + } }, + }, +}) +``` + +Each action covers these calls: + +| Action | What it covers | +| --- | --- | +| `read` | Open the session, and read its stream, snapshot, transcript, media, and child runs. | +| `list` | See the session in `host.sessions.list`. | +| `send` | Prompt, steer, follow up, run a command or a tool, start an agent, and upload media. | +| `answer` | Answer a question or an approval. | +| `operator` | Reply as an operator with `send(text, { as: 'operator' })`, and answer a `human` wait of any run. See [Hand off to a person](./handoffs#4-hand-off-to-a-person). | +| `cancel` | Stop a run, or drop an input that waits. | +| `configure` | Change the thread settings, the plugin config, the title, and the metadata. | +| `share` | Add or remove members. | +| `delete` | Remove the session from the index. | +| `fork` | Copy the session into a new session. | +| `transfer` | Move the conversation to another thread, also of another harness, with `host.transfer`. See [Move a conversation to another harness](./handoffs#5-move-a-conversation-to-another-harness). | + +- For `answer` and `cancel`, `isAllowed` also gets `run`: the `runId` and the `userId` of the user who started that run. +- For `answer` and `operator`, `isAllowed` also gets `wait`: the `waitId` and the `kind` of the [wait](./waits) that the input answers, when the host knows it. A `human` wait takes only `operator`, also for a resolve that ends it. +- For `cancelSchedule`, `run` is the schedule: the `runId` is the schedule id, and the `userId` is the user who made it. +- `host.submit` and `schedule` check an answer or a cancel when you send it, with `run` from the log. A run that the log does not have is refused with `unknown_run`: `submit` returns a `rejected` receipt, and `schedule` throws `InputRejectedError`. +- A handle checks `read` once, when it opens. Each action that changes the session asks `isAllowed` again. This includes `cancel` and `send` on an operation or an agent run that the handle returns. + +The rules have two more keys. `membersOnly()` sets both. Your own rules can leave them out: + +- `filterSessionList`: the filter that `host.sessions.list` gives the store, for example `{ memberId }`. Without it, the store reads every session of the tenant, and `isAllowed` drops the others, so a page can have fewer entries than `limit`. +- `members`: turns on `host.sessions.share` and `unshare`, and `members.fork` sets the members of a fork. Without it, `share` and `unshare` throw. + +## What the host checks for you + +`isAllowed` decides only the rule of your app. The host does these checks before it calls it: + +- The tenant. A user reaches only the sessions of their own tenant. `isAllowed` never gets a session of another tenant. +- A user on every call. With access, `host.open`, `host.fork`, `host.transfer`, and each `host.sessions` method need a `user`. A call without one throws. +- One user per handle. `host.open` returns a session that acts only as its `user`. A call on it that passes another `user` throws. +- Child threads. The host checks a subagent child as its root thread, so `session` is the root session. A user who cannot read the root gets `403` for the child, and does not learn its root. +- Missing threads. A read of a thread with no entry answers `404` and starts no session. `GET run?threadId=` answers an empty hydration instead, because `useChat` reads it before the first message. + +A refused action throws `AccessDeniedError`, with `code: 'forbidden'`, the `action`, and the `threadId`. The handler answers it with `403`. + +The host keeps only the `id` and the `tenantId` of a user in the log. So `isAllowed` can get a user with only these two fields. A rule that reads another field, such as `isAdmin` above, must refuse when the field is missing. + +A signed media URL is no login. The handler runs `getUser` for it too, and checks `read` for that user, in the tenant of the URL. A request with no user gets `401`. So load the file with a request that sends the login, for example an `` on the same site, which sends its cookies. + +### Serve media to a request with no login + +An `` on another site sends no cookies, so `getUser` finds no user. Give `access` a `getGuestUser` function. It returns a guest for that request, and `isAllowed` decides `read` for the guest: + +```ts group=harness-access-guest +import { createHarnessHost, membersOnly } from '@tanstack/ai-harness' +import { memoryPersistence } from '@tanstack/ai-persistence' + +interface User { + id: string + tenantId?: string + isGuest?: boolean +} + +const members = membersOnly() + +export const host = createHarnessHost({ + persistence: memoryPersistence(), + access: { + getUser: () => null, + // `tenantId` is the tenant of the signed URL. + getGuestUser: ({ tenantId }) => ({ id: 'guest', tenantId, isGuest: true }), + rules: { + isAllowed: (input) => + input.user.isGuest === true ? input.action === 'read' : members.isAllowed(input), + filterSessionList: members.filterSessionList, + members: members.members, + }, + }, +}) +``` + +- The handler calls `getGuestUser` only for a signed media URL, after the signature checks out, and only when `getUser` finds no user. +- The guest gets nothing else. Every other route still needs a user from `getUser`. +- Without `getGuestUser`, or when it returns `null`, the request gets `401`. +- The guest needs the tenant of the URL, else the request gets `403`. + +## Keep tenants apart + +Give each user a `tenantId` in `getUser`. The tenant keeps each organization apart: + +- One `threadId` in two tenants is two sessions, with two index entries and two logs. +- `host.sessions.list` reads one tenant: the tenant of `user`. A user with no `tenantId` is in the default tenant. +- The log store keeps each session tree under `logIdOf({ tenantId, logId })`, for example `acme:thread-1`. The default tenant gives `:thread-1`. Use `logIdOf` when you read a log yourself. +- The credentials of a user are kept per tenant. See [Keep organizations apart](./shared-threads#7-keep-organizations-apart). + +## Call the host from code with no request + +A cron job, an admin tool, or a queue worker has no request, so `getUser` cannot run. Pass the user yourself. The host checks the action for that user, the same as for a request: + +```ts group=harness-access +// Your database knows the owner of each thread. +export async function nightlyReport(threadId: string, owner: { id: string; tenantId: string }) { + const session = await host.open(assistant, { threadId, user: owner }) + await session.send('Write the report of today.') +} +``` + +Your own routes and an MCP server have a request. Read its user with `host.getUserOfRequest`: + +```ts group=harness-access +export async function myThreads(request: Request) { + const user = await host.getUserOfRequest({ request }) + if (!user) return new Response('Unauthorized', { status: 401 }) + const page = await host.sessions.list({ user, limit: 20 }) + return Response.json(page.entries.map((entry) => entry.title ?? 'New chat')) +} +``` + +`host.getUserOfRequest` throws on a host with no `access`. + +## Use your own access tables + +Your app already keeps who can use each thread, for example a table of teams. Read it in `isAllowed`. When you take a user out of the table, call `host.revoke`: + +```ts group=harness-access-own-tables +import { createHarnessHost } from '@tanstack/ai-harness' +import { memoryPersistence } from '@tanstack/ai-persistence' + +const users = new Map([ + ['Bearer token-ada', { id: 'ada', tenantId: 'acme' }], + ['Bearer token-bob', { id: 'bob', tenantId: 'acme' }], +]) + +// Your own table: the team of each thread. +const teams = new Map([['plan-q3', new Set(['ada', 'bob'])]]) + +export const host = createHarnessHost({ + persistence: memoryPersistence(), + access: { + getUser: ({ request }) => users.get(request.headers.get('authorization') ?? '') ?? null, + rules: { isAllowed: ({ user, session }) => + session.members.some((member) => member.userId === user.id) || + (teams.get(session.threadId)?.has(user.id) ?? false) }, + }, +}) + +export async function removeFromTeam(threadId: string, user: { id: string; tenantId: string }) { + teams.get(threadId)?.delete(user.id) + await host.revoke({ threadId, user }) +} +``` + +When the promise of `host.revoke` resolves: + +- The running and queued work of the user on the thread is cancelled, on every host. The schedules of the user on the thread are removed. +- The streams of the user on the thread close. A socket gets a `forbidden` error frame. A running `events()` read of a handle or a view ends with `AccessDeniedError`. +- The open handles and views of the user on the thread throw `AccessDeniedError`. A new `host.open` asks `isAllowed` again. +- The signed media URLs stop working for the user. The handler checks the user of each request again when it serves the file. +- `host.events()` yields a `revoked` event. + +When the cancels cannot be sent (for example, the log store fails), the promise rejects. The access is gone already. Call `host.revoke` again to stop the work. `host.sessions.unshare` rejects the same way, after it removed the member. + +`host.sessions.share` and `unshare` need `members` in the rules, which `membersOnly()` has. With your own tables, change the tables, then call `host.revoke`. + +## Serve with no login + +A demo, or a tool on your own machine, has no login. Give the handler `openAccess` on a host with no `access`: + +```ts group=harness-access-open +import { createHarnessHost, defineHarness } from '@tanstack/ai-harness' +import { createHarnessHandler } from '@tanstack/ai-harness/server' +import { memoryPersistence } from '@tanstack/ai-persistence' +import { openaiText } from '@tanstack/ai-openai' + +const assistant = defineHarness({ + name: 'acme/assistant', + adapter: openaiText('gpt-6.1-sol'), +}) + +const host = createHarnessHost({ persistence: memoryPersistence() }) + +// One user on this machine: every request runs as Ada. +export const handler = createHarnessHandler({ + host, + harness: assistant, + openAccess: { user: { id: 'ada' } }, +}) +``` + +- `openAccess: true`: every request runs with no user, in the default tenant. +- `openAccess: { user }`: every request runs as that one user. Their credentials and keys do the work. +- Without host `access` and without `openAccess`, `createHarnessHandler` throws at the start. So a server never serves every thread to everyone by mistake. +- On a host with `access`, `openAccess` throws. The host reads the user itself. +- `handleHarnessSocket` takes the same option. + +CAUTION: Do not use `openAccess` on a server that the internet can reach. Every caller can then use every thread. + +## Use the provider keys of the server + +With access, a keyed adapter and `run.keys` read only the keys that each user saved. They do not read the env vars of the server, so a user never spends the key of the server. To let users fall back to the server key, set `allowServerEnvKeys: true` in `access`, next to `getUser` and `policy`. See [Connect model providers](./provider-keys). + +## What you have now + +- A host that reads the user of each request, and checks each action. +- A rule of your own, typed from what `getUser` returns. +- Organizations that never see each other's threads. +- Cron jobs and admin tools that act as a user they name. +- A way to take access away from a user at once. diff --git a/docs/harness/acp.md b/docs/harness/acp.md index e87958c313..58d1afd50e 100644 --- a/docs/harness/acp.md +++ b/docs/harness/acp.md @@ -49,7 +49,9 @@ serveAcp({ host, harness: assistant, version: '1.0.0' }) ``` - Each ACP session is one harness session. The ACP session id is the thread id. -- The prompts of a session run one after another. The harness `busy` setting does not apply. +- `user` is the user of the editor, for example `serveAcp({ host, harness: assistant, user: { id: 'ada' } })`. Every session opens as that user, so `/connect` saves keys for them. On a host with [`access`](./access), `user` is required. +- The prompts of a session run one after another, in the order that the editor sent them. A prompt with a large file keeps its place while the file uploads. The harness `busy` setting does not apply. +- The agent holds a session only while a prompt runs or waits. An idle session closes after the host `lifecycle.closeIdleSessionsAfterMs`, and the next prompt opens it again. - `version` is the version that `initialize` reports. Default: `'0.0.0'`. - `cwd` is the working directory that the session list shows for each session. Default: `process.cwd()`. - Use a persistent store instead of `memoryPersistence()` if a session must stay after the process stops. @@ -62,22 +64,37 @@ Give the editor the command that runs the script, for example `node agent.js`. T - The text and the reasoning of the main agent. Text from child agents stays out of the main message. - Each tool call, of the main agent and of child agents, as a tool call update. -- Tool approvals as permission requests. The user picks **Allow** or **Reject**, and the turn continues. -- Cancel stops the running turn, also while the editor asks for an approval. A turn that fails ends with the `error` stop reason and the message. +- Each [wait](./waits) of a turn, by its kind (see [Waits in the editor](#waits-in-the-editor)). +- Cancel stops the running turn, also while the editor asks for an answer. A turn that fails ends with the `error` stop reason and the message. - Image blocks, audio blocks, and embedded files of a prompt go to the media store of the session. A file that the harness refuses fails the prompt with an invalid params error. See [Send and show media](./media). +## Waits in the editor + +The agent sends each wait to the editor by its kind: + +| Kind | What the editor shows | +| --- | --- | +| `approval` | A permission request with the id of the real tool call. **Allow always** shows only for an approval of `permissions()`, because its answer schema takes `always`. | +| `question`, `human` | A form elicitation of the answer schema. | +| `sign-in` | A URL elicitation with the sign-in page. | +| `client-tool` | Nothing. The editor cannot run it, so the agent cancels it. | + +- The editor must declare form and URL elicitation in `initialize`. If it does not, the agent cancels that wait, and the model continues without the answer. +- A question that a tool asks while it runs (`step.ask`, an MCP server) is a form elicitation too. An answer that does not fit the schema asks again. +- If the editor cancels the prompt while it asks, the turn stops. Its waits stay open until the next prompt. + ## Open an old session The editor lists the sessions of the harness, newest first. The user picks one, and the next prompt continues that thread. -- The list shows the top-level sessions of the harness. Sessions of child agents stay out. +- The list shows the top-level sessions of the harness that `user` can list. Sessions of child agents stay out. - The list comes from the session index of the host. Use a persistent store, so that the list keeps the sessions after the process stops. -- A session id that the session index does not know fails with a not found error. +- A session id that the session index does not know, or that `user` cannot read, fails with a not found error. ## Known limits - ACP v2 is still a draft, so this API is experimental. -- The editor shows tool approvals only. The agent cancels a sign-in, an input form, or a client tool call, because the editor cannot answer it. The model then continues without the answer. A question from `session.ask` waits until the user cancels the turn. +- A client tool call, and a wait that the editor did not declare it can show, are cancelled. The model then continues without the answer. - Each session in the list shows the same working directory: the `cwd` of the agent. If the editor asks for the sessions of another directory, the list is empty. - An old session opens without its earlier messages. The editor shows only the new turns. The model still gets the full thread. - A resource link in a prompt is only named. The harness does not read it. The model can open it with its own tools. @@ -92,7 +109,9 @@ import { createAcpAgent } from '@tanstack/ai-acp/agent' export const app = createAcpAgent({ host, harness: assistant }) ``` +One agent app serves one connection. It reads the editor capabilities from that connection, and a second connection to it fails with an error. To serve more editors, for example over HTTP, make one app for each connection. With the `AcpServer` of `@agentclientprotocol/sdk/experimental/server`, pass `createAgent: () => createAcpAgent({ host, harness: assistant })`. + ## What you have now - Your harness as an agent in any ACP v2 editor. -- Tool approvals, cancel, media, and old sessions from the editor. +- Approvals, questions, and sign-ins in the editor, and cancel, media, and old sessions. diff --git a/docs/harness/agents.md b/docs/harness/agents.md index 315b6b7355..2afb1e488c 100644 --- a/docs/harness/agents.md +++ b/docs/harness/agents.md @@ -21,7 +21,8 @@ import { readFile, writeFile } from 'node:fs/promises' import { resolve } from 'node:path' import { toolDefinition } from '@tanstack/ai' import { createHarnessHost, defineHarness } from '@tanstack/ai-harness' -import { agents, permissions } from '@tanstack/ai-harness/plugins' +import { permissions } from '@tanstack/ai-harness/plugins' +import { agents } from '@tanstack/ai-harness/plugins/node' import { anthropicText } from '@tanstack/ai-anthropic' import { memoryPersistence } from '@tanstack/ai-persistence' import { z } from 'zod' @@ -63,18 +64,18 @@ const coder = defineHarness({ const host = createHarnessHost({ persistence: memoryPersistence() }) const session = await host.open(coder, { threadId: 'dark-mode' }) -await session.command('agent', 'plan') +await session.runCommand({ name: 'agent', input: 'plan' }) const plan = await session.send('How do I add a dark mode?') console.log(plan.text) -await session.command('agent', 'build') +await session.runCommand({ name: 'agent', input: 'build' }) await session.send('Do step 1.') ``` 1. The `plan` agent reads the code and gives numbered steps. It has no tool that changes a file. 2. The `build` agent changes the files, after your answer to each [permission question](./permissions). -- `session.command('agent')` with no input shows the current agent and the list: `Agent: build. Agents: build, plan.` +- `session.runCommand({ name: 'agent' })` with no input shows the current agent and the list: `Agent: build. Agents: build, plan.` - A switch applies at the next turn. The same switch is the `agent` setting. `session.describe().config` lists its options for a menu: @@ -85,7 +86,7 @@ if (setting?.option.type === 'select') { console.log(setting.value, setting.option.options) // 'build' ['build', 'plan'] } -await session.setConfig({ key: 'agent', value: 'plan' }) +await session.setPluginSetting({ key: 'agent', value: 'plan' }) ``` A client can change the setting only when the harness exposes it. Add `expose: { config: ['agent'] }` to `defineHarness`. @@ -115,7 +116,7 @@ Give `agents()` a profile. This one reviews the changes with a stronger model: ```ts group=harness-agents import type { AnyTextAdapter } from '@tanstack/ai' -import type { AgentProfile } from '@tanstack/ai-harness/plugins' +import type { AgentProfile } from '@tanstack/ai-harness/plugins/node' const review: AgentProfile = { name: 'review', @@ -142,10 +143,10 @@ export const withReview = agents({ | Field | What it does | | --- | --- | -| `model` | A model id. `adapter` turns it into an adapter. Without it, a primary agent keeps the harness model, and a subagent uses the model of the main turn. | +| `model` | A model id. `adapter` turns it into an adapter. Without it, a primary agent keeps the harness model, and a subagent uses the model of the turn that started it. | | `system` | Text for the system prompt of each turn of this agent. | | `tools` | The tools that this agent can call, by name. A `*` matches any text, for example `todo_*`. Without it, the agent gets every tool. | -| `permissions` | [Permission rules](./permissions#write-rules) that count while this agent is the primary agent. | +| `permissions` | [Permission rules](./permissions#write-rules) that count while this agent is the primary agent. At that time, they also apply in each subagent run. | | `steps` | The step limit. See [Limit the steps](#limit-the-steps). | | `hidden` | `true` leaves the agent out of the `agent` command and setting. The model can still start it as a subagent. | @@ -183,7 +184,7 @@ export const fromFiles = agents({ - The body is the system prompt. - The frontmatter sets `description`, `mode`, `model`, `tools`, `steps`, and `hidden: true`. The default `mode` is `all`. - An unknown `mode`, or a `steps` that is not a whole number, stops `host.open` with an error, for example `Agent file review.md: "mode" is not valid.` -- The plugin reads the folder when a session opens, and again at each `session.reload()`. A folder that it cannot read gives no agents. +- The plugin reads the folder when a session opens, and again at each `session.reloadPlugins()`. A folder that it cannot read gives no agents. ## Limit the steps @@ -207,19 +208,20 @@ A profile with the mode `subagent` or `all` is also a subagent. The main model g A subagent run gets these parts of its profile: -- The tools of the last main turn, filtered by its `tools`. +- The tools of the turn that started it, filtered by its `tools`. - Its `system` prompt. -- Its `model`, or the model of the main turn. +- Its `model`, or the model of the turn that started it. - Its step limit. -A subagent run stops after 50 model calls, or when an answer is cut at the output limit. The harness also limits the depth and the number of children. See [Stay within limits](./subagents#stay-within-limits). +A subagent run stops after 50 model calls, or when an answer is cut at the output limit. The harness also limits the depth and the number of children. See [Stay within limits](./subagents#stay-within-limits). To give one run from code its own budget and time limit, see [Give one run a budget and a time limit](./subagents#give-one-run-a-budget-and-a-time-limit). ## Known limits -- The `permissions` of a profile count only while it is the primary agent. When the profile runs as a subagent, its own rules do not apply. Put rules that must hold in every run in `permissions({ rules })`. See [Rules in subagents](./permissions#rules-in-subagents). +- A subagent run uses the `permissions` of the primary agent, not the rules of its own profile. For example, when `build` is the primary agent and starts `review`, the `build` rules apply and the `review` rules do not. Put rules that must hold in every run in `permissions({ rules })`. See [Rules in subagents](./permissions#rules-in-subagents). - A Markdown agent cannot set `permissions`. Put those rules in a profile in code. - On Amazon Bedrock, the model can still call a tool on the last call. The tool does not run, but the turn can end with no text answer. -- A subagent that starts before the first turn of the session has no tools. It also needs its own `model`. +- A subagent run gets the tools and the model of its turn from the memory of the host that ran that turn. On another host, or after a restart, the run fails with `Agent cannot go on here`. Start it again from a turn. +- A subagent run that your code starts, not a turn, has no tools of a turn. Give its profile a `model`. ## What you have now diff --git a/docs/harness/auth.md b/docs/harness/auth.md index e631942bb7..7ffbce667a 100644 --- a/docs/harness/auth.md +++ b/docs/harness/auth.md @@ -19,6 +19,7 @@ Your agent needs to open a pull request, so it needs the user's GitHub token. Th import { toolDefinition } from '@tanstack/ai' import { defineHarness, oauthConnector } from '@tanstack/ai-harness' import { openaiText } from '@tanstack/ai-openai' +import type { HarnessRunContext } from '@tanstack/ai-harness' const github = oauthConnector({ id: 'github', @@ -33,10 +34,10 @@ const github = oauthConnector({ scopes: ['repo'], }, tools: (token) => [ - toolDefinition({ name: 'list_issues', description: 'List my open issues' }).server( - async () => { + toolDefinition({ name: 'list_issues', description: 'List my open issues' }).server( + async (_args, ctx) => { const response = await fetch('https://api.github.com/issues', { - headers: { authorization: `Bearer ${await token()}` }, + headers: { authorization: `Bearer ${await token({ run: ctx.context.run })}` }, }) return response.json() }, @@ -51,13 +52,15 @@ export const assistant = defineHarness({ }) ``` -`token()` returns a fresh access token. It refreshes an expired token when the service gave a refresh token. +`token({ run })` returns a fresh access token of the user of that run. If the token is expired and the service gave a refresh token, `token` refreshes it first, one time for all the calls that need it. See [Refresh a token](#refresh-a-token). Pass the `run` of the tool call, `ctx.context.run`, so each user gets their own token. See [Act as the user of a run](./plugins#act-as-the-user-of-a-run). ## 2. Sign in - The user runs `/connect github`. - With `login: 'loopback'`, the CLI opens the browser. The harness listens on `127.0.0.1` on a random port for one callback, with PKCE and a random `state`. +- With `oauth.issuer` set, the sign-in refuses a code that the service sends with another `iss` (RFC 9207). - The token goes into the credential store. `/disconnect github` deletes it. +- A cancel of the command, or a close of the session, stops a sign-in that waits. The listener on `127.0.0.1` closes at once. `login` is required. Use `login: 'device'` for a harness on a server, and for SSH sessions, containers, and CI. The user then gets a code to enter on the service's page. Device login needs `oauth.deviceUrl`. @@ -77,12 +80,13 @@ export function showSignIns(session: HarnessSession) { ## Wait for the sign-in -A turn from a background agent or an email runs while nobody watches it. That turn can need a sign-in. Without a sign-in, the tool fails, and the work of that turn stops. Pass `wait: true` to `token()`. Then the turn waits for the sign-in and continues after it. +A turn from a background agent or an email runs while nobody watches it. That turn can need a sign-in. Without a sign-in, the tool fails, and the work of that turn stops. Pass `wait: true` to `token`. Then the turn waits for the sign-in and continues after it. ```ts group=harness-auth-wait import { toolDefinition } from '@tanstack/ai' import { defineHarness, oauthConnector } from '@tanstack/ai-harness' import { openaiText } from '@tanstack/ai-openai' +import type { HarnessRunContext } from '@tanstack/ai-harness' const github = oauthConnector({ id: 'github', @@ -97,10 +101,10 @@ const github = oauthConnector({ scopes: ['repo'], }, tools: (token) => [ - toolDefinition({ name: 'list_issues', description: 'List my open issues' }).server( - async () => { + toolDefinition({ name: 'list_issues', description: 'List my open issues' }).server( + async (_args, ctx) => { const response = await fetch('https://api.github.com/issues', { - headers: { authorization: `Bearer ${await token({ wait: true })}` }, + headers: { authorization: `Bearer ${await token({ run: ctx.context.run, wait: true })}` }, }) return response.json() }, @@ -117,17 +121,17 @@ export const assistant = defineHarness({ }) ``` -In your own plugin, use `ctx.credentials.require('github', { wait: true })`. +In a tool of your own plugin, use `ctx.context.run.credentials.require('github', { wait: true })`. When the credential is missing, the turn stops with an interrupt: - The interrupt has `reason: 'auth_required'`. Its `metadata['tanstack:interruptPayload'].request.connector` is `'github'`. -- The sign-in belongs to the sender of the stopped turn. When that person saves the credential, the same turn continues as that person, and the tool runs again. The save can be `/connect github`, the `connect:github` command, or `ctx.credentials.set`. +- The sign-in belongs to the sender of the stopped turn. When that person saves the credential, the same turn continues as that person, and the tool runs again. The save can be `/connect github`, the `connect:github` command, or `run.credentials.set` in a command of your own. - A credential that another person saves never continues the turn. - To stop the wait, resolve the interrupt with `status: 'cancelled'`. Then the tool fails, the same as without `wait`. -- On a durable host, the wait continues after a restart. +- On a [durable host](./durable-sessions), the wait continues after a restart. - `wait` is for a tool of the running chat turn. Do not pass it in a command, a background agent, or a hook. The credentials of a command never wait. -- Provider keys (`ctx.keys.require`) do not wait. +- Provider keys (`run.keys.require`) do not wait. In a web app, the wait shows in `state.signIns` of the session view, also after a reload. When the user presses your connect button, run the connector command. Each sign-in then has the `url` and the `userCode` to show. A client can run only the commands in `expose.commands`, so the harness above exposes `connect:github` (see [Choose what clients can change](./connect#choose-what-clients-can-change)): @@ -143,14 +147,14 @@ view.on('signIn', (signIn) => { console.log(`Connect ${signIn.connector} to continue the turn.`) }) -export const connectGithub = () => view.command('connect:github') +export const connectGithub = () => view.runCommand({ name: 'connect:github' }) ``` After the sign-in, the turn finishes its work with no new message. ## Keep credentials -Pass a credential store to the host. The host reads credentials from it, keyed by the sender of the running turn. Without one, credentials live in memory until the process stops. When several people write in one thread, see [Share a thread between people](./shared-threads). +Pass a credential store to the host. The host reads credentials from it, keyed by the user of each run. Without one, credentials live in memory until the process stops. When several people write in one thread, see [Share a thread between people](./shared-threads). ```ts group=harness-auth import { defineCredentialStore } from '@tanstack/ai/byok' @@ -184,14 +188,145 @@ The store can be a table in your own database or a secrets manager. See the rule - Encrypt tokens at rest in a real store. - `list` returns ids and types only, never the secret values. +- The harness keeps a secret answer in the store for a short time, with the type `secret_answer`. Leave that type out of a list of connected accounts. See [Secret answers on another host](./durable-sessions#secret-answers-on-another-host). - A credential saved without a `userId` belongs to the whole tenant. A user who has no credential of their own uses it. See [Share one credential with the whole organization](./shared-threads#share-one-credential-with-the-whole-organization). +- If more than one host uses the store, add `getVersioned` and `setIf`. Then two hosts do not both save a refreshed token. See [Add compare-and-set](../advanced/server-keys#add-compare-and-set). ## Read credentials in your own plugin -`ctx.credentials.require('github')` returns the credential, or stops with `auth_required` when the user has not signed in. Plugins read only the credentials of the sender of the running turn. Outside a turn, they read the credentials of the person who opened the session. +Your plugin reads the credentials of the user of each run: + +- In a tool or a middleware hook, `ctx.context.run.credentials.require('github')` returns the credential. Before the user signs in, it stops with `auth_required`. +- In a command, use `run.credentials` of the command context. +- The `ctx.credentials` of a session plugin have no user, so each call throws. + +See [Act as the user of a run](./plugins#act-as-the-user-of-a-run). + +## Keep tokens out of the log and the model + +A tool can show a token by mistake: an error message with the request URL, or a debug field in its result. The session replaces each secret that it gave out with `[redacted]` before it keeps or sends anything. + +The secrets that the session knows: + +- The provider keys of `run.keys`, also a key from an env var. +- The credentials of `run.credentials` and of the connectors: API keys, OAuth tokens, and the OAuth client secret. +- The answer to a question with `secret: true`. For an answer with fields, each string field. + +Where the session replaces them: + +- The session log: your records, the transcript, plugin state (also in a `Map` or a `Set`), notes, and tool results. +- The events that clients get. +- The model context, so the model never sees a secret. +- The error frames of the HTTP handler, the WebSocket, and the stdio worker. + +A [durable step](./durable-tools) value with a secret throws, so the log never keeps it. Read the secret outside `step.do`. + +The session finds the raw value, its JSON-escaped form, and its URL-encoded form. It does not find: + +- A value shorter than 6 characters. So a short value does not remove common words. +- A secret that a stream splits across two deltas, in those two events. The message in the transcript does not have the secret. +- A secret that is JSON-escaped two times. +- A secret in an instance of your own class, a `Date`, or a typed array in plugin state. + +To scrub a text of your own, use `scrubSecrets(text, secrets)` from `@tanstack/ai/byok`. It uses the same rules. + +## Write your own sign-in command + +`oauthConnector` covers most services. For a sign-in of your own, use the helpers from `@tanstack/ai-harness/oauth`. Pass the `signal` of the command, so a cancel stops the sign-in at once: + +```ts group=harness-auth-own +import { defineCommand, definePlugin } from '@tanstack/ai-harness' +import { deviceLogin, isExpired, refreshCredential } from '@tanstack/ai-harness/oauth' +import type { OAuthConfig } from '@tanstack/ai-harness/oauth' + +const gitlab: OAuthConfig = { + authorizationUrl: 'https://gitlab.com/oauth/authorize', + tokenUrl: 'https://gitlab.com/oauth/token', + deviceUrl: 'https://gitlab.com/oauth/authorize_device', + clientId: 'your-gitlab-app-client-id', + scopes: ['read_api'], +} + +export const gitlabLogin = definePlugin({ + name: 'acme/gitlab-login', + setup: (ctx) => ({ + commands: { + 'connect:gitlab': defineCommand({ + description: 'Sign in to GitLab', + run: async (_input, { credentials, signal }) => { + const credential = await deviceLogin(gitlab, { + onCode: ({ userCode, verificationUri }) => + ctx.session.authRequired({ connector: 'gitlab', url: verificationUri, userCode }), + signal, + }) + await credentials.set({ id: 'gitlab', credential }) + return 'Connected to GitLab.' + }, + }), + }, + }), +}) +``` + +- `deviceLogin`, `loopbackLogin`, `exchangeCode`, and `refreshCredential` take a `signal`. When it aborts, the call stops at once and rejects with the reason of the signal. +- The `signal` of a command aborts on a cancel of the command and on a close of the session. +- `loopbackLogin` takes `timeoutMs`. The default is 10 minutes. This timer does not keep the process alive. +- When another library runs the OAuth flow, `startLoopbackReceiver({ timeoutMs, signal })` only waits for the callback. `waitForCode(state)` gives the code and the `iss` of the callback. + +### Refresh a token + +An access token expires, and several tool calls can need it at the same time. Some services rotate the refresh token: after one refresh, the old refresh token stops working. Then a second refresh at the same time fails, and the user must sign in again. + +`run.credentials.refresh(id, update, { signal })` runs one refresh, and every caller gets its result: + +```ts group=harness-auth-own +import { toolDefinition } from '@tanstack/ai' +import type { HarnessRunContext } from '@tanstack/ai-harness' + +export const gitlabProjects = toolDefinition({ + name: 'gitlab_projects', + description: 'List my GitLab projects', +}).server(async (_args, ctx) => { + const { credentials, signal } = ctx.context.run + let credential = await credentials.require('gitlab') + if (isExpired(credential)) { + const fresh = await credentials.refresh( + 'gitlab', + // Another call can refresh first. Then keep its credential. + (current, hostSignal) => + isExpired(current) + ? refreshCredential(gitlab, current, { signal: hostSignal }) + : Promise.resolve(current), + { signal }, + ) + // The user disconnected meanwhile: ask for a sign-in. + credential = fresh ?? (await credentials.require('gitlab')) + } + const token = credential.type === 'oauth' ? credential.accessToken : credential.value + const response = await fetch('https://gitlab.com/api/v4/projects?membership=true', { + headers: { authorization: `Bearer ${token}` }, + signal, + }) + return response.json() +}) +``` + +`update` gets the saved credential and returns the new one. Return `current` to keep it. `refresh` resolves with the credential to use, or `null` when none is saved. `oauthConnector` refreshes with `refresh` too. + +What `refresh` does: + +- One host runs one `update` at a time for each saved credential. A call that comes while one runs waits for it, and gets the same credential. +- It saves the new credential to the scope where it found it. A tenant token stays the tenant's, so every user of the tenant gets the new token. +- If the store has `getVersioned` and `setIf`, the save is a compare-and-set. When another host saved first, the call uses that token and does not refresh again. +- Without that pair, the one-at-a-time rule is per host only. Two hosts can refresh at the same time, and the last save wins. +- `update` gets a signal of the host, which only `host.close()` aborts. An abort of your `signal` rejects only your call. The refresh goes on for the other calls. + +`refreshCredential` keeps the OAuth `client` of the old credential, so the next refresh has it too. ## What you have now -- Browser and device-code sign-in for any OAuth service. +- Browser and device-code sign-in for any OAuth service, and sign-ins of your own that stop at once on a cancel. - Tools that get a fresh token without the model seeing it. +- One refresh for each expired token, also on many hosts. +- A log, events, and a model context with `[redacted]` in place of each secret. - Turns that wait for a sign-in and then finish their work. diff --git a/docs/harness/cli.md b/docs/harness/cli.md index 92eb7a56c6..d67ae84b6c 100644 --- a/docs/harness/cli.md +++ b/docs/harness/cli.md @@ -59,7 +59,13 @@ To use the harness from another program: `--mcp` and `--serve` run only the commands in `expose.commands` of the harness. `--serve` also sets only the config keys in `expose.config`. Add the names to `expose` in `defineHarness`, for example `expose: { commands: ['connect:github'] }`. Line mode and `-p` run every command. See [Choose what clients can change](./connect#choose-what-clients-can-change). -Line mode reads one message or command per line and waits for each turn. It works the same in a terminal and with piped input. In a terminal, it also opens sign-in links in the browser. +Line mode reads one message or command per line. In a terminal, you can type while a turn runs: + +- `/cancel` cancels the running turn at once. +- A message joins the running turn as a steer. The model reads it at its next call. +- A message that you type when no turn runs starts a new turn. + +With piped input, each line waits for the turn that the line before it started. A script runs in order. In a terminal, line mode also opens sign-in links in the browser. ## 3. Use it in CI @@ -69,26 +75,60 @@ Line mode reads one message or command per line and waits for each turn. It work |---|---| | 0 | The turn finished. | | 1 | The turn failed. | -| 2 | The turn waits for an approval. | -| 130 | The turn was cancelled. | +| 3 | The turn waits for a person: an approval, an answer, or a sign-in. | +| 130 | The turn was cancelled, or SIGINT (Ctrl+C) stopped the CLI. | +| 143 | SIGTERM cancelled the turn or stopped the CLI. | + +`EXIT` from `@tanstack/ai-harness-cli` has these codes, for example `EXIT.needsAction` (3). `-p` exits 3 at the first [wait](./waits) of the turn, because nobody can answer in this mode. It writes what the turn waits for on stderr. On a durable host, the wait stays in the log, so a later run, or another server, can answer it. + +The codes 130 and 143 apply in every mode. See [Stop with Ctrl+C](#stop-with-ctrlc). + +## Stop with Ctrl+C + +Ctrl+C works the same in every mode: + +- The first Ctrl+C (SIGINT) or SIGTERM cancels the running turn. +- A second one, or one while no turn runs, stops the CLI. `runCli` resolves to 130 for SIGINT, and to 143 for SIGTERM. +- When the CLI made the host, it closes the host with `recoverable: true`. Another host then runs the cut work again. +- When you gave the CLI your own `host`, the CLI releases only its session handle. Your host stays open. + +The CLI hears the signals on `process`. To send them yourself, for example in a test, pass a `signals` source. An `EventEmitter` works: + +```ts group=harness-cli-signals +import { EventEmitter } from 'node:events' +import { defineHarness } from '@tanstack/ai-harness' +import { runCli } from '@tanstack/ai-harness-cli' +import { openaiText } from '@tanstack/ai-openai' + +const assistant = defineHarness({ + name: 'acme/assistant', + adapter: openaiText('gpt-6.1-sol'), +}) + +const signals = new EventEmitter() +const exit = runCli(assistant, { argv: ['-p', 'Write a long story.'], signals }) +// No turn runs yet, so this stops the CLI. +signals.emit('SIGTERM') +console.log(await exit) // 143 +``` ## Serve over HTTP safely `--serve` is the quick way to put a harness on HTTP, so it makes the trust decisions for you: one bearer token, one user, every thread. Anyone with the token acts as that one user. They can open or drive any thread on the host, including the tools it runs (shell commands, file edits) on the machine the CLI runs on. Treat the token like a shell login. - It binds to `127.0.0.1:8787`, reachable only from the same machine. Change the bind with `--host` and `--port`. `--host 0.0.0.0` puts it on the network, so do that only behind TLS and a proxy you trust. -- The CLI signs media links with the token, so rotating the token stops any link a turn already handed out. +- The CLI signs media links with its own key, never with the token. By default it makes a new random key at each start, so media links stop working after a restart. To keep them working, pass `--media-key` or set `HARNESS_MEDIA_KEY`, with a key of at least 32 bytes. Change the key to stop every link a turn already handed out. -Need per-user access, your own sign-in check, or a rule for which threads each user can open? Use [`createHarnessHandler`](./connect) instead. It requires `authorize` and `canAccess`, so no thread is open by accident. +Need per-user access, your own sign-in check, or a rule for which threads each user can open? Use [`createHarnessHandler`](./connect) on a host with [`access`](./access) instead. The host reads the user of each request and checks each action, so no thread is open by accident. ## Use your own host and user -By default, the CLI builds its own host in memory and opens sessions with no user. Your server already has a durable host with leases, recovery, and stores, and each person has their own credentials. Give the CLI that host and the user: +By default, the CLI builds its own host in memory and opens sessions with no user. Your server already has a durable host with its own stores, and each person has their own credentials. Give the CLI that host and the user: ```ts group=harness-cli-host import { createHarnessHost, defineHarness } from '@tanstack/ai-harness' import { runCli } from '@tanstack/ai-harness-cli' -import { memoryLogStore, memoryPersistence } from '@tanstack/ai-persistence' +import { memoryPersistence } from '@tanstack/ai-persistence' import { memoryCredentialStore } from '@tanstack/ai/byok/server' import { openaiText } from '@tanstack/ai-openai' @@ -97,21 +137,20 @@ const assistant = defineHarness({ adapter: openaiText('gpt-6.1-sol'), }) -const { runs, metadata } = memoryPersistence().stores const host = createHarnessHost({ - persistence: { stores: { log: memoryLogStore(), runs, metadata } }, + persistence: memoryPersistence(), credentials: memoryCredentialStore(), }) -process.exitCode = await runCli(assistant, { host, principal: { id: 'ada' } }) +process.exitCode = await runCli(assistant, { host, user: { id: 'ada' } }) await host.close() ``` - You own the host, so the CLI does not close it. Close it yourself when `runCli` returns. Without `host`, the CLI builds a host and closes it. -- Give `host` or `persistence`, not both. With both, `runCli` writes `Give runCli a host or persistence, not both.` and returns code 1. -- Line mode, `-p`, and your own `ui` open the session as `principal`, so `/connect` saves keys for that user. -- `--serve` gives the same principal to each request with the token. -- `--acp`, `--mcp`, and `--dashboard` open their sessions without the principal. +- Give `host`, or `persistence` and `credentials`, not both. With both, `runCli` writes `Give runCli a host, or persistence and credentials, not both.` and returns code 1. +- Line mode, `-p`, your own `ui`, `--acp`, and `--mcp` open their sessions as `user`, so `/connect` saves keys for that user. +- `--serve` checks the token first. On a host without `access`, each request then runs as `user`. On a host with [`access`](./access), `getUser` reads the user of each request. +- On a host with `access`, give `user` too. `--mcp` does not start on such a host, because stdio has no login. ## Commands in line mode @@ -128,7 +167,14 @@ For the running work: - `/status`: show what runs and what waits. - `/exit`: quit. -Plugin commands (for example `/model` or `/todos`) show up in `/help`. When a turn stops for an approval or a plugin asks a question, type your answer. For yes-or-no questions, `y` approves and `n` refuses. +Plugin commands (for example `/model` or `/todos`) show up in `/help`. + +When a turn waits for you, each next line answers one wait, in order. The turn continues after the last one: + +- An approval: `y` approves and `n` refuses. An approval of `permissions()` also takes `once`, `always`, or `reject`. +- A question: type the answer. The CLI reads it with the schema of the question. +- A sign-in: sign in, then type `y` to go on, or `n` to skip it. +- A client tool: the terminal cannot run it. Any line skips it. ## Send files and save media @@ -175,8 +221,9 @@ process.exitCode = await runCli(assistant, { }) ``` -- `ui` runs only in an interactive terminal. Piped input uses line mode. `-p`, `--acp`, `--mcp`, `--serve`, and `--dashboard` do not use `ui`. +- `ui` runs only in an interactive terminal. Piped input uses line mode. `-p`, `--acp`, `--mcp`, and `--serve` do not use `ui`. - When `ui` resolves, the CLI disposes the view and `runCli` returns. +- In a reopened thread, a saved child agent shows only its card. To show what it wrote, call `view.loadAgent({ id })`, for example when the screen draws the card. See [Show subagents](./custom-ui#show-subagents). - To write `Screen`, read [Build your own UI](./custom-ui). For an Ink screen with approvals, questions, sign-ins, and child agents, copy [`examples/harness-cli/src/tui.tsx`](https://github.com/TanStack/ai/blob/main/examples/harness-cli/src/tui.tsx). @@ -185,6 +232,8 @@ For an Ink screen with approvals, questions, sign-ins, and child agents, copy [` - One entry file that runs your harness as a terminal app, a script step, an editor agent, an MCP server, or an HTTP server. - A CLI that uses your durable host and runs as your user. +- A Ctrl+C that cancels the turn first, then stops with exit code 130. +- Exit code 3 from `-p` when a turn waits for a person, and one line for each answer in line mode. - Your own terminal screen on the same session, with any TUI library. - Files that you send with `@path`, and a folder with the media that your agents make. diff --git a/docs/harness/code-mode.md b/docs/harness/code-mode.md index 266c022390..b6a66b42ed 100644 --- a/docs/harness/code-mode.md +++ b/docs/harness/code-mode.md @@ -99,7 +99,8 @@ export const onlyIssues = codeMode({ }) ``` -- A tool that `include` picks moves even when it needs approval, and its calls inside the program do not ask. Pick only tools that are safe to run without a question. +- `include` never moves a tool with `needsApproval`, or a tool that an `ask` or `deny` rule matches. That tool stays a normal tool call with its approval, and code mode warns once. The rules decide in the permission mode of the session, again at each turn. So in `plan` mode, a tool that edits stays a normal tool call. +- Any other tool that `include` picks moves, also when a rule asks for it. Its calls inside the program do not ask, so pick only tools that are safe to run without a question. - With `include`, a call of `execute_typescript` gets the permission rules like any tool, for example a question with `default: 'ask'`. - A safe tool with `metadata.codeMode: true` moves even when `include` does not pick it. diff --git a/docs/harness/coding-agent.md b/docs/harness/coding-agent.md index a8bffe5f11..81ffed8f91 100644 --- a/docs/harness/coding-agent.md +++ b/docs/harness/coding-agent.md @@ -13,29 +13,26 @@ keywords: - undo --- -You want your own coding agent in the terminal. It reads and edits files, and runs commands after you approve them. When a turn goes wrong, you undo it. The first-party plugins in `@tanstack/ai-harness/plugins` and `@tanstack/ai-harness/plugins/coding` give you those parts. You pick the model and the rules. +You want your own coding agent in the terminal. It reads and edits files, and runs commands after you approve them. When a turn goes wrong, you undo it. The first-party plugins in `@tanstack/ai-harness/plugins`, `@tanstack/ai-harness/plugins/node`, and `@tanstack/ai-harness/plugins/coding` give you those parts. You pick the model and the rules. ## 1. Define the agent ```ts group=harness-coding-agent +import { randomUUID } from 'node:crypto' import { homedir } from 'node:os' import { join } from 'node:path' import { createInterface } from 'node:readline/promises' -import { - HARNESS_EVENTS, - createHarnessHost, - defineHarness, -} from '@tanstack/ai-harness' +import { createHarnessHost, defineHarness } from '@tanstack/ai-harness' +import type { RunAgentResumeItem } from '@tanstack/ai' import { compact, - fileCommands, modelPicker, permissions, - projectInstructions, title, todos, usage, } from '@tanstack/ai-harness/plugins' +import { fileCommands, projectInstructions } from '@tanstack/ai-harness/plugins/node' import { snapshots, workspaceTools } from '@tanstack/ai-harness/plugins/coding' import { openaiText } from '@tanstack/ai-openai' import { memoryPersistence } from '@tanstack/ai-persistence' @@ -70,36 +67,39 @@ const terminal = createInterface({ output: process.stdout, }) -// A risky call waits for an answer: once, always, or reject. -// Calls can run in parallel. Ask one question at a time, each once. -const asked = new Set() -let asking = Promise.resolve() -const stopAsking = session.onEvent('CUSTOM', ({ event }) => { - if (event.name !== HARNESS_EVENTS.question) return - asking = asking.then(async () => { - for (const question of session.snapshot().pendingQuestions) { - if (asked.has(question.questionId)) continue - asked.add(question.questionId) - const answer = await terminal.question(`\n${question.message} `) - await session.answer({ questionId: question.questionId, value: answer }) - } - }) +// Print the text of every turn, also of a turn that continues after an answer. +const stopPrinting = session.onEvent('TEXT_MESSAGE_CONTENT', ({ event }) => { + process.stdout.write(event.delta) }) +// A risky call stops the turn before it runs. Ask about each approval: +// once, always, or reject. Then send all the answers in one resolve. +async function answerApprovals() { + let waits = session.snapshot().pendingWaits + while (waits.length > 0) { + const resume: Array = [] + for (const wait of waits) { + const answer = await terminal.question(`\n${wait.message ?? wait.kind} (once/always/reject) `) + resume.push({ interruptId: wait.waitId, status: 'resolved', payload: answer }) + } + const inputId = randomUUID() + await session.resolve({ resume, inputId }) + await session.settled({ inputId }) + waits = session.snapshot().pendingWaits + } +} + // Type `exit` to stop. let line = await terminal.question('\n> ') while (line !== 'exit') { if (line.startsWith('/')) { // `/undo`, `/model fast`, `/mode plan`, and the other plugin commands. const [name = '', ...args] = line.slice(1).split(' ') - console.log(await session.command(name, args.join(' ') || undefined)) + console.log(await session.runCommand({ name, input: args.join(' ') || undefined })) } else { - const turn = session.send(line) - turn.onEvent('TEXT_MESSAGE_CONTENT', ({ event }) => { - process.stdout.write(event.delta) - }) try { - await turn + await session.send(line) + await answerApprovals() } catch (error) { // A failed turn rejects. Show the error and keep the loop. console.error(error) @@ -107,12 +107,12 @@ while (line !== 'exit') { } line = await terminal.question('\n> ') } -stopAsking() +stopPrinting() terminal.close() await host.close() ``` -Both plugin entries run only in Node. The coding tools use the file system and the shell of the machine. A line that starts with `/` runs a command of a plugin. Any other line is a prompt. +The `plugins/node` and `plugins/coding` entries run only in Node. The coding tools use the file system and the shell of the machine. A line that starts with `/` runs a command of a plugin. Any other line is a prompt. ## 2. Run it diff --git a/docs/harness/coding-agents.md b/docs/harness/coding-agents.md index 125577e14c..e264d7f401 100644 --- a/docs/harness/coding-agents.md +++ b/docs/harness/coding-agents.md @@ -108,6 +108,8 @@ To start over, run `/fresh claude_code`, or `/fresh` for every agent. Use `'shared'` when the agents build on each other's changes. Use `'per-agent'` when they work on separate copies and you merge the results. +Only the host that owns the log of the thread runs its agents. So with many servers, `'shared'` still runs one agent at a time. + ## Plan mode When the `permissions()` plugin is in [`plan` mode](./permissions#pick-a-mode), the agents start read-only: diff --git a/docs/harness/coding-tools-backends.md b/docs/harness/coding-tools-backends.md index 1641fe93e3..7b8f43e3de 100644 --- a/docs/harness/coding-tools-backends.md +++ b/docs/harness/coding-tools-backends.md @@ -62,12 +62,12 @@ export const sandboxed = defineHarness({ 3. When you are done, stop the sandbox with `await sandbox.destroy()`. -Every provider works. Get the handle from `create()`, or from `resume()` for a sandbox that runs already. See [Providers](../sandbox/providers). +Every provider works. Get the handle from `create()`, or from `resume()` for a sandbox that runs already. See [Providers](../sandbox/providers). To create a sandbox for each thread, and resume it on another server, see [Run the coding tools for many threads](./coding-workspaces). In a sandbox, some things are different: - Paths are POSIX paths in the sandbox, like `/workspace/src/a.ts`, also on a Windows host. -- The tools do not check where links lead. The sandbox backend has no `realpath`. +- The tools check where links lead with `lstat` and `readlink` in the sandbox. A sandbox without `lstat` cannot find links, so the tools refuse every path. - `background: true` needs a sandbox with `capabilities.backgroundProcesses`. - A sandbox with `capabilities.killableProcesses` off cannot stop a command after the time limit. The call returns, and the command continues in the sandbox. - `list_files` and `grep` use `rg` in the sandbox. The `@vscode/ripgrep` binary is a host file, so the sandbox does not use it. @@ -82,11 +82,15 @@ The backend must have: | --- | --- | | `readFile(path)` | Gives the bytes of a file. Throws when there is no file. | | `writeFile(path, data)` | Creates or replaces a file. Creates the missing parent folders. | -| `stat(path)` | Gives `{ type, size, mtimeMs }`, or `undefined` when nothing is at `path`. | +| `stat(path)` | Gives `{ type, size, mtimeMs }`, or `undefined` when nothing is at `path`. It does not read the file. | +| `realpath(path)` | Gives the path with every link resolved, or `undefined` when nothing is at `path`. Throws when it cannot resolve a path. | | `readdir(path)` | Gives the entries of a folder. A link has the type `'link'`. | | `exec(command, options)` | Runs a shell command and waits. Gives `{ exitCode, stdout, stderr }`. | -`exec` resolves with the exit code of a command that fails. It does not throw. After `options.timeoutMs`, it stops the command with every process that it started, and gives exit code 124. +- `exec` resolves with the exit code of a command that fails. It does not throw. +- After `options.timeoutMs`, `exec` stops the command with every process that it started, and gives exit code 124. `options.signal` stops it too. +- `options.env` holds the variables to add. Give a command only these and the few variables that it needs to start. Do not give it the variables of your server. +- When `realpath` throws, the tool call fails. So a path with a link that the backend cannot resolve is never used. The backend can also have: @@ -95,8 +99,18 @@ The backend can also have: | `shell` | `'sh'` or `'cmd'`. Sets the path style and how arguments are quoted. | `sh` quotes, and the path style of this machine. | | `remove(path)` | Removes one file. | `patch` cannot delete or move files. | | `readLines(path, { offset, limit })` | Gives one page of a file: `{ head, size, lines, total }`. `head` is the first 8 KiB. | `read_file` reads the whole file for each page. | -| `realpath(path)` | Gives the path with every link resolved. | The tools do not check links. | -| `spawn(command, options)` | Starts a command in the background. | `background: true` gives an error. | + +For background jobs and for threads that move to another server: + +| Member | What it does | Without it | +| --- | --- | --- | +| `spawn(command, options)` | Starts a command in the background. Gives `{ wait, kill, output, jobId? }`. `jobId` resolves with the id of the process, for `killJob`. | `background: true` gives an error. | +| `killJob(jobId)` | Stops a job that `spawn` started, also on an earlier process or machine. Resolves `false` when no such job runs. Rejects when it cannot check that the process is the job: then the log keeps the job. | A job that a crash left running is not stopped. | +| `exec` option `onStart(jobId)` | Gives the id of a foreground command when it starts, so a command that moves to the background can be stopped by a later host too. | A moved command has no id in the log. | +| `binding()` | Gives `{ kind: 'sandbox', provider, sandboxId }` for a sandbox that another machine can resume. | The workspace is a folder on this host. | +| `close({ reason })` | Releases the backend. `'closed'`: the session closed. `'moved'`: the session continues on another machine, so keep what it can resume. | Nothing to release. | + +`close` must never destroy a sandbox that the backend did not create. The fastest start is to wrap `hostBackend`. This backend logs each command before it runs: @@ -212,8 +226,9 @@ export const withToml = formatter({ - `builtins: false` turns the built-in formatters off. - `timeoutMs` limits each run. The default is 20 seconds. - In a sandbox or on your own backend, give `formatter()` the same `backend` as `workspaceTools()`. +- With [`workspaceTools({ workspace: { bind } })`](./coding-workspaces) before it, `formatter()` runs in the workspace of each thread. Leave out `root` and `backend`. -A formatter that fails, stops at the time limit, or is not installed does not fail the tool call. The file stays as the tool wrote it, and the plugin sends `FormatFailed`: +A formatter that fails, stops at the time limit, or is not installed does not fail the tool call. The file stays as the tool wrote it, and the plugin sends `FormatFailed`. A formatter that changes the file sends `FormatterRewrote` with `{ path, formatter }`: ```ts group=harness-coding-tools-backends import { FormatFailed } from '@tanstack/ai-harness/plugins/coding' @@ -228,16 +243,19 @@ export const formatAlerts = definePlugin({ }) ``` -Clients get the same event as a `harness.plugin.event` with the name `tanstack/formatter:failed`. +Clients get the same events as a `harness.plugin.event` with the name `tanstack/formatter:failed` or `tanstack/formatter:rewrote`. + +### One approval covers the write and its formatter -### Each run asks first +A formatter runs code from the project, like `prettier.config.js`. The model can change that code with an edit. So with `permissions()`, the rules decide the formatter as a part of the call that writes. The rule names the run `formatter:prettier`, `formatter:biome`, and so on. -A formatter runs code from the project, like `prettier.config.js`. The model can change that code with an edit. So with `permissions()`, the formatter asks before each run, also in `acceptEdits` mode. The question names the run `formatter:prettier`, `formatter:biome`, and so on. +The rules decide before the call runs, and the formatter runs after the write. The call never stops in the middle to ask. -- Answer `always` once, and the formatter runs without a question in this project. -- A `reject` keeps the write. The tool result says that the formatter did not run. -- A rule can allow a formatter: `{ tool: 'formatter:prettier', decision: 'allow' }`. -- Without `permissions()`, the formatter runs without a question. +- When the formatter asks, the call asks one time for the write and the formatter, also in `acceptEdits` mode. Answer that [approval](./permissions#answer-an-approval) with `resolve`. +- A `reject` stops the call. Nothing is written, and the formatter does not run. +- Answer `always` once, and the formatter runs without a question in this project. The answer is tied to a hash of the formatter config files, like `prettier.config.js` and `package.json`. When they change, the next write asks again. +- A rule that denies the formatter keeps the write. The tool result says that the formatter did not run. +- A rule can allow a formatter: `{ tool: 'formatter:prettier', decision: 'allow' }`. Without `permissions()`, the formatter runs without a question. [Formatters](./permissions#formatters) shows what each mode does. @@ -246,7 +264,7 @@ A formatter runs code from the project, like `prettier.config.js`. The model can One tool result with a full log or a big JSON file fills the context of the model. `boundToolOutput()` cuts every tool result that is too long: ```ts group=harness-coding-tools-backends -import { boundToolOutput } from '@tanstack/ai-harness/plugins' +import { boundToolOutput } from '@tanstack/ai-harness/plugins/node' export const bounded = defineHarness({ name: 'acme/bounded-coder', diff --git a/docs/harness/coding-tools.md b/docs/harness/coding-tools.md index 577d2a0a99..7af94ef80b 100644 --- a/docs/harness/coding-tools.md +++ b/docs/harness/coding-tools.md @@ -50,12 +50,10 @@ vanilla: @vscode/ripgrep turndown ## 2. Add the tools ```ts group=harness-coding-tools +import { randomUUID } from 'node:crypto' import { createInterface } from 'node:readline/promises' -import { - HARNESS_EVENTS, - createHarnessHost, - defineHarness, -} from '@tanstack/ai-harness' +import { createHarnessHost, defineHarness } from '@tanstack/ai-harness' +import type { RunAgentResumeItem } from '@tanstack/ai' import { permissions } from '@tanstack/ai-harness/plugins' import { workspaceTools } from '@tanstack/ai-harness/plugins/coding' import { openaiText } from '@tanstack/ai-openai' @@ -76,38 +74,41 @@ const terminal = createInterface({ output: process.stdout, }) -// Ask the user each question that a risky call waits on. Calls can run -// in parallel, so ask one question at a time, each once. -const asked = new Set() -let asking = Promise.resolve() -const stopAsking = session.onEvent('CUSTOM', ({ event }) => { - if (event.name !== HARNESS_EVENTS.question) return - asking = asking.then(async () => { - for (const question of session.snapshot().pendingQuestions) { - if (asked.has(question.questionId)) continue - asked.add(question.questionId) - const answer = await terminal.question(`\n${question.message} `) - await session.answer({ questionId: question.questionId, value: answer }) - } - }) +// Print the text of every turn, also of a turn that continues after an answer. +const stopPrinting = session.onEvent('TEXT_MESSAGE_CONTENT', ({ event }) => { + process.stdout.write(event.delta) }) -// Send each line as a prompt. Print the answer. Type `exit` to stop. +// A risky call stops the turn before it runs. Ask about each approval of +// the turn, then send all the answers in one resolve. +async function answerApprovals() { + let waits = session.snapshot().pendingWaits + while (waits.length > 0) { + const resume: Array = [] + for (const wait of waits) { + const answer = await terminal.question(`\n${wait.message ?? wait.kind} (once/always/reject) `) + resume.push({ interruptId: wait.waitId, status: 'resolved', payload: answer }) + } + const inputId = randomUUID() + await session.resolve({ resume, inputId }) + await session.settled({ inputId }) + waits = session.snapshot().pendingWaits + } +} + +// Send each line as a prompt. Type `exit` to stop. let line = await terminal.question('\n> ') while (line !== 'exit') { - const turn = session.send(line) - turn.onEvent('TEXT_MESSAGE_CONTENT', ({ event }) => { - process.stdout.write(event.delta) - }) try { - await turn + await session.send(line) + await answerApprovals() } catch (error) { // A failed turn rejects. Show the error and keep the loop. console.error(error) } line = await terminal.question('\n> ') } -stopAsking() +stopPrinting() terminal.close() await host.close() ``` @@ -118,7 +119,7 @@ await host.close() 1. Run `npx tsx coder.ts` in your project. 2. Ask: `find the failing test and fix it`. -3. When the agent wants to change a file or run a command, it asks in the terminal. Answer `once`, `always`, or `reject`. +3. When the agent wants to change a file or run a command, the turn stops, and the script asks in the terminal. Answer `once`, `always`, or `reject`. The agent reads and searches without a question. For a full coding agent with undo, a todo list, and a model picker, see [Build a coding agent](./coding-agent). @@ -134,7 +135,7 @@ These tools run without a question. | `list_files` | Lists the files in a folder. An optional glob, like `src/**/*.ts`, filters them. | | `grep` | Searches the file contents with a regular expression. Gives `file:line: text`. | -- `read_file` gives images (PNG, JPEG, GIF, WebP) and PDFs up to 5 MB to the model as media. +- `read_file` gives images (PNG, JPEG, GIF, WebP) and PDFs up to 256 KB to the model as media. For a bigger file, the model gets an error that tells it to make a smaller copy. - `read_file` refuses a binary file with a short note. For a missing file, it names up to 3 files with a close name. - `list_files` and `grep` skip the files that `.gitignore` names, the `.git` and `node_modules` folders, and links. - `list_files` shows at most 1000 files. `grep` shows at most 200 matches. @@ -156,6 +157,8 @@ A model gets `edit_file` or `patch`, not both. Every model gets `write_file`. Th `edit_file` keeps the line endings and the byte order mark of the file. It also finds text that is different only in Unicode form or in the spaces at the end of a line. +When a file changed after the model read it, for example by another agent, the three tools fail. The model reads the file again, then edits it. + To let the user undo the changes of a turn, add [snapshots](./snapshots). ### Commands and the web @@ -170,10 +173,15 @@ For `bash`: - The model gets the exit code and the last 1000 lines (20 KiB) of the output. - A command stops after 2 minutes, with exit code 124. Every process that it started stops too. Set another limit with `bashTimeoutMs`. The model can ask for up to 10 minutes. -- With `background: true`, the call returns at once with a job id. When the job ends, the model gets a note with the output, and a new turn starts. The time limit does not apply to a background job. -- With `spillDir`, long output goes to a file in that folder, and the model gets the path. `spillDir` is relative to `root`, or absolute. +- With `spillDir`, long output goes to a file in that folder, and the model gets the path. `spillDir` is relative to `root`, or absolute. The file has `[redacted]` in place of each secret of the session, the same as the tool result. See [Keep tokens out of the log and the model](./auth#keep-tokens-out-of-the-log-and-the-model). +- A command gets only a few variables of your process, like `PATH` and `HOME`. Your provider keys are not in its env. On the host this is not isolation: a command can still read the files of your server. See [Keep server secrets out of commands](./coding-workspaces#2-keep-server-secrets-out-of-commands). + +With `background: true`, the call returns at once with a job id. When the job ends, the model gets a note with the output, and a new turn starts. That turn runs as the user whose run started the job. + +- The time limit does not apply to a background job. +- At most 8 background jobs run at a time in a session. Set another limit with `maxJobs`. Each job keeps the newest 1 MiB of its output. - When the session closes, the background jobs stop. -- On a durable host (one with `stores.log`), the log keeps each background job until it ends. When a crash stops a job, the next host adds the note "Background job bash-1 stopped when the host restarted." Without `stores.log`, a crash stops the job, and no note comes. +- The log keeps each background job until it ends. When a crash stops a job, the next host adds the note "Background job bash-1 stopped when the host restarted." On Windows, the default backend runs commands in `cmd.exe`. A program name like `rg` or `gradlew` comes from the `PATH` only, not from the working folder. So a cloned repo cannot put its own `rg.cmd` in place of `rg`. To run a script in the working folder, give its path: `.\gradlew build`. @@ -182,7 +190,8 @@ For `webfetch`: - It reads `http` and `https` URLs only, and follows at most 5 redirects. - It refuses `localhost`, private network addresses, and cloud metadata addresses. It checks the DNS answer of each connection. - HTML comes back as Markdown, or as text without `turndown`. The model can ask for `format: 'html'`. Text and JSON come back as they are. -- It reads at most 5 MB. It stops after 30 seconds, and the model can ask for up to 120 seconds. +- The model gets the first 256 KB of a page, with a note when there is more. It reads at most 5 MB. +- It stops after 30 seconds, and the model can ask for up to 120 seconds. Change the web tools with the `web` option: @@ -247,6 +256,7 @@ export const trustTests = permissions({ - Paths match from `root`. Give `permissions()` and `workspaceTools()` the same `root`. - A rule also matches the real path of a link, and the stricter decision wins. So a link `notes` to `.env` asks, and a `deny` for `secrets/**` also stops a link to `secrets`. +- The rules decide every path before the call runs. A tool never asks while it runs. - `bash` splits a command at `&&`, `||`, `;`, `|`, and new lines. Every part must be allowed. - A command with `$(`, a backtick, or a heredoc never gets an allow. @@ -256,17 +266,22 @@ export const trustTests = permissions({ ### A path outside `root` -The tools refuse a path outside `root`. To ask the user instead, set `outside: 'ask'`: +The tools refuse a path outside `root`. To let the user approve it, set `outside: 'ask'`: ```ts group=harness-coding-tools export const askOutside = workspaceTools({ root, outside: 'ask' }) ``` -- A yes allows that folder, and the folders in it, until the session ends. -- `bypass` mode allows every path without a question. -- A link in `root` that leads out of `root` counts as outside. +`permissions()` decides the outside path before the call runs. The path is a part of the one [approval](./permissions#answer-an-approval) of the call: the turn stops, and you answer with `resolve`. The tool never stops in the middle of a call to ask. + +- `once` allows this call. `always` also saves an allow rule for the path. +- `reject` stops the call. The call does not run, so nothing is read or written. +- A link in `root` that leads out of `root` counts as outside. The rules decide its real path. +- `bypass` mode allows every path without a question. Without `permissions()`, nobody can approve the path, so the tool refuses it. - With `permissions()`, `grep` skips the files outside `root`, because `read_file` asks for them. A `read_file` rule for that folder, for example `{ tool: 'read_file', resource: '/shared/**', decision: 'allow' }`, lets `grep` show them. +With a [workspace for each thread](./coding-workspaces), the root is not known before the call, so no approval can cover an outside path. Then only a rule that allows the path, or `bypass` mode, lets the call use it. + ### A working folder for each thread Each thread can work in its own folder in `root`. Set the `cwd` thread setting: @@ -285,13 +300,13 @@ const repos = defineHarness({ const host = createHarnessHost({ persistence: memoryPersistence() }) const session = await host.open(repos, { threadId: 'fix-login' }) -await session.configure({ cwd: 'web-app' }) +await session.updateThreadSettings({ settings: { cwd: 'web-app' } }) ``` - Relative paths start in `/srv/repos/web-app`, and `bash` runs there. - The system prompt tells the model the working folder. - Permission rules still match paths from `root`, for example `web-app/src/**`. -- `cwd` is relative to `root`. A `cwd` outside `root` counts as an outside path. +- `cwd` is relative to `root`. The tools refuse a `cwd` outside `root`, also with `outside: 'ask'`. [Store settings per thread](./thread-settings) shows how a client sets `cwd`. @@ -301,4 +316,4 @@ await session.configure({ cwd: 'web-app' }) - A question before each edit, command, and web fetch. - A working folder for each thread, and a question before a path outside `root`. -Next: run the same tools in a sandbox, or format each file after a write. See [Sandbox and extend the coding tools](./coding-tools-backends). +Next: run the same tools in a sandbox, or format each file after a write. See [Sandbox and extend the coding tools](./coding-tools-backends). To give each thread its own folder on a server, see [Run the coding tools for many threads](./coding-workspaces). diff --git a/docs/harness/coding-workspaces.md b/docs/harness/coding-workspaces.md new file mode 100644 index 0000000000..6e258dfd0c --- /dev/null +++ b/docs/harness/coding-workspaces.md @@ -0,0 +1,278 @@ +--- +title: Run the coding tools for many threads +id: harness-coding-workspaces +order: 7 +description: "Give each thread of a coding agent its own folder or sandbox, keep server secrets out of commands, let agents share files safely, watch background jobs, and move a thread to another server." +keywords: + - tanstack ai + - harness + - coding agent + - workspace tools + - sandbox + - multi-tenant + - background jobs +--- + +Your server runs coding agents for many users. Each thread must work in its own folder or sandbox. A command must not see the API keys of your server. Two agents in one folder must not lose each other's edits. And a thread must continue when another server takes it. + +`workspace: { bind }` gives each thread its own workspace. The session log keeps where that workspace is, so the next server finds it again. + +## 1. Give each thread its own workspace + +Pass `workspace` to `workspaceTools()` in place of `root`. `bind` gets the run, and gives a backend with its `root`: + +```ts group=harness-coding-workspaces +import { mkdir } from 'node:fs/promises' +import { defineHarness } from '@tanstack/ai-harness' +import { permissions } from '@tanstack/ai-harness/plugins' +import { + hostBackend, + workspaceRoot, + workspaceTools, +} from '@tanstack/ai-harness/plugins/coding' +import { openaiText } from '@tanstack/ai-openai' + +export const coder = defineHarness({ + name: 'acme/coder', + adapter: openaiText('gpt-6.1-sol'), + plugins: () => [ + permissions(), + workspaceTools({ + workspace: { + bind: async ({ run }) => { + const root = workspaceRoot('/srv/workspaces', run.tenantId ?? 'default', run.threadId) + await mkdir(root, { recursive: true }) + return { ...hostBackend, root } + }, + }, + }), + ], +}) +``` + +- The tools call `bind` once for each thread, before the first turn or at the first tool call. If `bind` throws, the next turn calls it again. +- A client sends the thread id. Do not put it in a path with `join` or a template string: an id like `../other-tenant/t-1` leads into another folder. `workspaceRoot` encodes each part, so each id gets its own folder in `/srv/workspaces`, and it throws for a path that leaves it. +- The `root` that `bind` gives is the boundary of the file tools. A path outside it is refused. With `outside: 'ask'`, a rule that allows the path, or `bypass` mode, lets a call use it. No approval can cover it, because the root is not known before the call. The `cwd` thread setting must stay inside it. +- The tools resolve every link with `realpath` before they read or write, while they hold the lock on the file. So a link in one thread cannot lead into the folder of another thread. +- The session log keeps a `harness.workspace.bound` record with the place of the workspace. [Move a thread to another server](#5-move-a-thread-to-another-server) uses it. +- Put `workspaceTools()` before [`snapshots()`](./snapshots) and [`formatter()`](./coding-tools-backends#format-each-file-after-a-write). Then they work in the workspace of each thread, and their `root` is optional. + +A folder on the server is not isolation: `bash` runs as the user of your server (see [Keep server secrets out of commands](#2-keep-server-secrets-out-of-commands)). When many users share the server, give each thread a sandbox. Create it in `bind`, and give the backend `owned: true`. Then the thread owns the sandbox: + +```ts group=harness-coding-workspaces +import { sandboxWorkspaceBackend } from '@tanstack/ai-sandbox/harness' +import { dockerSandbox } from '@tanstack/ai-sandbox-docker' + +const docker = dockerSandbox({ image: 'node:22' }) + +export const sandboxed = workspaceTools({ + workspace: { + bind: async () => ({ + ...sandboxWorkspaceBackend(await docker.create({}), { owned: true }), + root: '/workspace', + }), + }, +}) +``` + +- A session close (an idle close, a deploy, or a move to another server) closes the backend with `'moved'`. The sandbox stays, so the next server can [reattach](#a-sandbox) it. +- [`host.sessions.delete`](./lifecycle#delete-a-thread-for-good) ends the thread. The host reattaches the sandbox and closes it with `'closed'`, so it is destroyed. The host that deletes must know the harness: it opened the harness, or it has the harness in its [`harnesses`](./lifecycle#delete-a-thread-for-good) option. Else it warns, and the sandbox stays. + +## 2. Keep server secrets out of commands + +A command gets only a few variables of the server process: + +- `PATH`, `HOME`, `LANG`, `LC_ALL`, `TMPDIR`, `TEMP`, and `TMP`. +- On Windows also `SystemRoot`, `ComSpec`, `PATHEXT`, `WINDIR`, and `USERPROFILE`. +- `AGENT=1`, so a tool can see that an agent runs it. + +Provider keys, `DATABASE_URL`, and the other variables of the server are not in the env of a command. The same rule applies to `bash`, `rg`, `git`, and the formatters. Git also runs without the hooks and the `fsmonitor` of the project. + +With `hostBackend`, and with the local-process sandbox, this short env and the `root` are defense in depth. They are not isolation: + +- A command runs as the user of your server. `bash` can read every file that this user can read, for example the `.env` file of the server, or the folder of another thread. +- `bash` can also read the env of each process of that user, for example `/proc//environ` of the server. + +For users that you do not trust, use [a sandbox for each thread](#1-give-each-thread-its-own-workspace). + +To give commands a variable, add it to the `env` of the backend calls: + +```ts group=harness-coding-workspaces +import type { WorkspaceBackend } from '@tanstack/ai-harness/plugins/coding' + +const extra = { NPM_TOKEN: 'npm_read_only_token' } +const { spawn } = hostBackend + +const withToken: WorkspaceBackend = { + ...hostBackend, + exec: (command, options) => + hostBackend.exec(command, { ...options, env: { ...options?.env, ...extra } }), + spawn: + spawn && + ((command, options) => + spawn(command, { ...options, env: { ...options?.env, ...extra } })), +} + +export const withTokenTools = workspaceTools({ root: '/srv/repo', backend: withToken }) +``` + +`exec` runs the foreground commands. `spawn` runs the background commands. + +## 3. Let agents share a folder + +Two agents, or two sessions, can work in the same folder. No edit is lost: + +- A write holds a lock on the file. The lock is shared by every session on this server, and it is keyed by the sandbox (or this server) and the real path of the file. Two roots that lead to one folder share the lock. Two sandboxes with the same root do not. +- `write_file`, `edit_file`, and `patch` fail when the file changed after the tool read it in the same agent run. A subagent and its lead each check their own read. The model gets `src/app.ts changed since it was read. Read it again before you edit it.`, reads the file again, and tries again. +- The check compares the size and the time of the last change. A sandbox gives no change time, so there only a change of size is seen. + +## 4. Watch background jobs + +With `background: true`, `bash` returns at once with a job id like `bash-1`. Two limits keep jobs small: + +- At most 8 background jobs run at a time in a session. One more gives the model a tool error. Set another limit with `maxJobs`. A running command that `session.moveToolCallToBackground()` moves counts too, and at the limit it does not move. +- Each job keeps the newest 1 MiB of its output. + +When a job ends, the model gets a note, and the wake turn runs as the user whose run started the job. + +To show the jobs that run now, for example in a status bar, call `session.jobs()`. A [view](./lifecycle#read-a-thread-without-a-session) of `host.observe` has `jobs()` too: + +```ts group=harness-coding-workspaces +import type { HarnessSession } from '@tanstack/ai-harness' + +export function listJobs(session: HarnessSession) { + for (const job of session.jobs()) { + console.log(`${job.jobId}: ${job.label ?? 'job'}, started by ${job.user?.id ?? 'the agent'}`) + } +} +``` + +Each entry has: + +- `jobId` and `threadId`: the job, and the thread of the workspace that runs it. +- `label`: the command, or the name of the tool that `session.moveToolCallToBackground()` moved. +- `user` and `startedAt`: who moved the call to the background, and when the job started. +- `running`: `true` when this host runs the job. A job of another host, or one that a crash stopped, has `false`. +- `backendId`: the id of the process in the workspace backend, when it has one. + +`session.jobs()` reads the log, so it also lists the jobs of other hosts. After a restart, it lists a job until the host notes that the job stopped. + +Each job sends events. Listen for them in a plugin: + +```ts group=harness-coding-workspaces +import { definePlugin } from '@tanstack/ai-harness' +import { JobEnded, JobOutput, JobStarted } from '@tanstack/ai-harness/plugins/coding' + +export const jobLog = definePlugin({ + name: 'acme/job-log', + setup: (ctx) => { + ctx.on(JobStarted, (job) => console.log(`${job.jobId} started: ${job.command}`)) + ctx.on(JobOutput, (job) => process.stdout.write(job.output)) + ctx.on(JobEnded, (job) => console.log(`${job.jobId} ended: ${job.exitCode}`)) + }, +}) +``` + +A client gets each event as a `CUSTOM` event with the name `harness.plugin.event`. Its value is `{ plugin, name, value }`: + +```ts group=harness-coding-workspaces-client +import { createHarnessClient } from '@tanstack/ai-harness/client' +import type { coder } from './harness' + +const client = createHarnessClient({ + url: '/api/harness', + threadId: 'fix-login', +}) + +for await (const { event } of client.events()) { + if (event.type !== 'CUSTOM' || event.name !== 'harness.plugin.event') continue + const { value } = event + if (typeof value !== 'object' || value === null) continue + if (!('name' in value) || value.name !== 'tanstack/workspace-tools:job-output') continue + if ('value' in value) console.log(value.value) +} +``` + +The coding plugins send these events: + +| Name | When | +| --- | --- | +| `tanstack/workspace-tools:job-started` | A background job started. | +| `tanstack/workspace-tools:job-output` | New output of a job. At most one event each 250 ms, with at most the newest 64 KiB. | +| `tanstack/workspace-tools:job-ended` | A job ended, after its last output event. | +| `tanstack/workspace-tools:stranded` | The workspace of the thread is gone. See below. | +| `tanstack/snapshots:failed` | A snapshot failed. The turn goes on. | + +The formatter also sends `tanstack/formatter:rewrote` and `tanstack/formatter:failed`. See [Format each file after a write](./coding-tools-backends#format-each-file-after-a-write). + +## 5. Move a thread to another server + +One server owns a thread at a time. When it stops, another server takes the thread (see [One host drives a thread](./durable-sessions#one-host-drives-a-thread)). What happens to the workspace depends on where it is. + +### A sandbox + +The log keeps the provider and the id of the sandbox. The next server calls `reattach` in place of `bind`. Resume the sandbox there: + +```ts group=harness-coding-workspaces +export const movable = workspaceTools({ + workspace: { + bind: async () => ({ + ...sandboxWorkspaceBackend(await docker.create({}), { owned: true }), + root: '/workspace', + }), + reattach: async ({ binding }) => { + if (binding.kind === 'host') return 'pinned' + if (binding.provider !== 'docker') return 'gone' + const handle = await docker.resume({ id: binding.sandboxId }) + if (!handle) return 'gone' + return { ...sandboxWorkspaceBackend(handle, { owned: true }), root: binding.root } + }, + }, +}) +``` + +- Give the workspace back to continue in the same files. +- Give `'gone'` when the sandbox is gone. Then each tool call of the thread fails with `The workspace of this thread, /workspace in the docker sandbox , is gone, so the tools cannot work on it.`, and the tools send `tanstack/workspace-tools:stranded`. The session also sends a `harness.stranded` event, for clients that do not know the plugin. See the [event catalog](./protocol#event-catalog). +- Without `reattach`, a sandbox counts as `'gone'`. + +### A folder on one server + +A folder on the disk of one server cannot move. Give each server a stable id: + +```ts group=harness-coding-workspaces +import { createHarnessHost } from '@tanstack/ai-harness' +import { memoryPersistence } from '@tanstack/ai-persistence' + +const host = createHarnessHost({ + persistence: memoryPersistence(), + ownership: { hostId: process.env.POD_NAME ?? 'web-1' }, +}) +``` + +- With `hostId`, a thread that uses a local folder is pinned to this server. Workers on other servers do not take it, also while the thread is idle. +- Another server that opens the thread refuses its tool calls with `The workspace of this thread is on the host web-1. Only that host can continue the thread.` +- Keep the id the same after a restart. On Kubernetes, use the pod name of a StatefulSet. Do not use `HOSTNAME` of a Deployment pod: it changes at each restart, and the threads of the old pod stay pinned to it. +- Without `hostId`, threads are not pinned, and every server calls `bind` again. Use this when every server sees the same folder, for example on a shared disk. + +The server sends a heartbeat to the log store each `ownership.renewEveryMs`. When no heartbeat came for `ownership.expiresAfterMs` (default 30 seconds), its pins do not count. Then the next server that gets work for the thread calls `reattach` with the old binding. Give a new workspace there, for example a fresh clone, to move the thread. Keep a restart shorter than `ttlMs`, or raise `ttlMs`. + +To move the threads of a server at once, unpin them with `host.unpin`. Its ownership must have expired first: + +```ts group=harness-coding-workspaces +await host.unpin({ harness: coder, threadId: 'fix-login' }) +``` + +### After a move + +- Background jobs do not move. At the next tool call of the thread, the new server stops each job that the old server left running. The model gets the note `Background job bash-1 was stopped: it ran on a host before the session moved to this one.` +- A job that the backend cannot check stays in the log, and the model gets `Background job bash-1 may still run: it ran on a host before this one, and this host cannot stop it.` For example, `hostBackend` needs `ps` to check a job. +- `/undo`, `/redo`, and a revert work on the new server when it reaches the same snapshot repository. See [Undo a turn](./snapshots#on-more-than-one-server). +- [Coding agents](./coding-agents) run only on the server that owns the log. So two servers never run the agents of one shared workspace at the same time. + +## What you have now + +- A folder or a sandbox for each thread, with its root as the boundary of every tool. +- Commands that get only a short env, and a sandbox for users that you do not trust. +- Agents that share a folder and never lose an edit. +- Events for every background job, and a limit on how many run. +- Threads that move to another server with their sandbox, or stay on the server that has their folder. diff --git a/docs/harness/compaction.md b/docs/harness/compaction.md index aca25bbc1c..c6266a78ff 100644 --- a/docs/harness/compaction.md +++ b/docs/harness/compaction.md @@ -21,11 +21,11 @@ Set `durable: true`. Compaction then writes a `tanstack.compaction` record into 1. Create the compaction middleware with `durable: true`. 2. Add it to `middleware` in `defineHarness`. -3. Give the host `projectCompaction` in its `project` option. +3. Give the harness `projectCompaction` in its `durability.addRecordsToModelContext` option. ```ts group=harness-compaction import { createHarnessHost, defineHarness } from '@tanstack/ai-harness' -import { memoryLogStore, memoryPersistence } from '@tanstack/ai-persistence' +import { memoryPersistence } from '@tanstack/ai-persistence' import { openaiText } from '@tanstack/ai-openai' import { conversationSummarizer, @@ -50,18 +50,14 @@ const assistant = defineHarness({ name: 'acme/assistant', adapter: openaiText('gpt-6.1-sol'), middleware: [compaction], + durability: { addRecordsToModelContext: { version: 'v1', record: projectCompaction } }, }) -const { runs, metadata } = memoryPersistence().stores -const host = createHarnessHost({ - persistence: { stores: { log: memoryLogStore(), runs, metadata } }, - project: { version: 'v1', record: projectCompaction }, -}) +const host = createHarnessHost({ persistence: memoryPersistence() }) ``` - Compaction runs before a model call when the count passes `maxTokens`. If the usage of the last model call of a turn passed `maxTokens` or `contextWindow`, compaction also runs right after that call. This check needs `countTokens: 'usage'`. - The log only grows, so the original messages stay in the store. The fold that the model gets starts with the summary. -- A host without a log has no place for the record. Compaction then keeps its result in the `metadata` store, the same as in a plain `chat()` call. It also skips the check after the turn. ## Prepare the summary in the background @@ -84,7 +80,7 @@ const earlyCompaction = withCompaction({ - `atTokens` must be below `maxTokens`. If it is not, `withCompaction` throws. - When the count at a model call passes `atTokens`, the summary starts beside the turn. The turn does not wait for it. -- The summary applies at the first model call of the next run, never in the middle of a run. On a durable host, it goes into the log as a `tanstack.compaction` record with `reason: 'background'`. +- The summary applies at the first model call of the next run, never in the middle of a run. It goes into the log as a `tanstack.compaction` record with `reason: 'background'`. - A model call over `maxTokens` while the summary runs waits for it and applies it. It compacts inline only when the messages are still over `maxTokens`. - If you cancel the run during that wait, the wait stops. The summary continues and applies at the next run. @@ -95,7 +91,7 @@ Sometimes a summary does not apply: ## Fold your own records too -If your host already folds records of its own, call `projectCompaction` first: +If your harness already folds records of its own, call `projectCompaction` first: ```ts group=harness-compaction import type { ProjectOptions } from '@tanstack/ai-harness' @@ -114,11 +110,11 @@ const project: ProjectOptions = { } ``` -Change `version` when you change the function. A fold checkpoint with another version is ignored. +Change `version` when you change the function. A log snapshot with another version is not used, so the host folds the log again from the first record. ## Retry after a context overflow -A model call can still fail because the context is too long. `withCompaction` handles it for you. When a call fails with `CONTEXT_OVERFLOW_CODE` (`'context_overflow'`), the middleware compacts and calls the model again, inside the `chat()` call of the turn. On a durable host, that compaction also writes its record to the log. +A model call can still fail because the context is too long. `withCompaction` handles it for you. When a call fails with `CONTEXT_OVERFLOW_CODE` (`'context_overflow'`), the middleware compacts and calls the model again, inside the `chat()` call of the turn. That compaction also writes its record to the log. - It retries once per model call. This also works with `auto: false`. - `turn.onModelError` sees the overflow only when the compacted call is still too long. Then `error.code` is `'context_overflow'`, and the turn fails unless your hook returns `'retry'`. diff --git a/docs/harness/connect.md b/docs/harness/connect.md index 8d6b23afe1..9f12042a7e 100644 --- a/docs/harness/connect.md +++ b/docs/harness/connect.md @@ -15,10 +15,11 @@ Your harness runs on a server, but the people who use it are in a browser or ano ## Serve the session over HTTP -Mount one fetch handler on a route. `authorize` is required, so no endpoint is open by accident. +Mount one fetch handler on a route. The host reads the user of each request with its `access`, so no endpoint is open by accident. ```ts group=harness-connect -import { defineHarness, createHarnessHandler, createHarnessHost } from '@tanstack/ai-harness' +import { createHarnessHost, defineHarness, membersOnly } from '@tanstack/ai-harness' +import { createHarnessHandler } from '@tanstack/ai-harness/server' import { memoryPersistence } from '@tanstack/ai-persistence' import { openaiText } from '@tanstack/ai-openai' @@ -27,42 +28,104 @@ export const assistant = defineHarness({ adapter: openaiText('gpt-6.1-sol'), }) -const host = createHarnessHost({ persistence: memoryPersistence() }) +const host = createHarnessHost({ + persistence: memoryPersistence(), + access: { + getUser: ({ request }) => + request.headers.get('authorization') === `Bearer ${process.env.HARNESS_TOKEN}` + ? { id: 'user-1' } + : null, + rules: membersOnly(), + }, +}) + +export const handler = createHarnessHandler({ host, harness: assistant }) +``` + +- `getUser` reads the user of each request. `null` answers `401`. +- `isAllowed` decides each action. A refused action answers `403`. `membersOnly()` lets a user use the threads they opened and the threads shared with them. +- A host with no `access` needs `openAccess` on the handler, else `createHarnessHandler` throws at the start. See [Serve with no login](./access#serve-with-no-login). + +[Access and tenants](./access) has the full rules. -export const handler = createHarnessHandler({ +When more than one instance serves the same URL, also set `media: { secret }`. The secret signs media URLs, so every instance must use the same value. Without it, the handler signs with a random key of its own, so a signed URL works only on this instance, until it restarts. + +On a host with `ownership` options, `createHarnessHandler` throws at the start when `media` is not set. On another host without `media`, the first handler logs one warning. + +```ts group=harness-connect +const mediaKey = process.env.MEDIA_KEY +const oldMediaKey = process.env.OLD_MEDIA_KEY +if (!mediaKey || !oldMediaKey) { + throw new Error('Set MEDIA_KEY and OLD_MEDIA_KEY.') +} + +export const rotatingHandler = createHarnessHandler({ host, harness: assistant, - authorize: (request) => - request.headers.get('authorization') === `Bearer ${process.env.HARNESS_TOKEN}` - ? { id: 'user-1' } - : null, - canAccess: (principal, threadId) => threadId.startsWith(`${principal.id}-`), + // The first key signs. Every key checks the URLs. + media: { keys: [mediaKey, oldMediaKey] }, }) ``` -`canAccess` is required. It decides which threads each principal can use. To let every principal use every thread, pass `canAccess: () => true`. +- Each key must be at least 32 bytes. A shorter key throws at the start. +- To change the key, put the new key first in `keys` and keep the old key after it. A URL that the old key signed works until it expires (1 hour). Then remove the old key. +- A signed URL works only for this harness, in the tenant of the user who got it. Each request still needs a user from `getUser` that can read the thread, so send the login with it. For a request with no login, see [Serve media to a request with no login](./access#serve-media-to-a-request-with-no-login). When the thread is deleted, its URLs answer `404`. -On a durable host (`stores.log`), also set `mediaSecret`. It signs media URLs, so every instance must use the same value. Without a log store, the default is a random key, and signed URLs stop working after a restart. +A request body over `media.maxBodyBytes` (default 32 MiB) answers `413`. See [Files inside a client message](./media#files-inside-a-client-message). -The handler answers these paths under your route: +The handler answers these paths under your route. For chat and events: - `GET capabilities`: the AG-UI capabilities, with the agents in `expose.agents`. - `POST run`: standard AG-UI. One request runs one prompt and streams it as SSE. The prompt keeps every content part of the last user message, for example an image. `useChat` works with it. See [Use it from useChat](#use-it-from-usechat). - `GET run?threadId=`: the saved messages, the running turn, and the waiting approvals, for a `useChat` that loads the thread after a reload. -- `GET run?runId=`: the turn with that run id as SSE, from its first event. A reloaded `useChat` joins a running turn with it. +- `GET run?runId=&threadId=`: the turn with that run id as SSE, from its first event. A reloaded `useChat` joins a running turn with it. Another instance finds the turn only with `threadId`. See [Join a turn after a reload](#join-a-turn-after-a-reload). - `GET events?threadId=`: every event of the session as SSE. Each event id is a cursor, so a reconnect with `Last-Event-ID` continues where it stopped. + +For one session: + - `POST control`: send `{ threadId, input }`, for example `{ op: 'prompt', message }`. You get a receipt back. Agents, settings, config keys, and commands need `expose`. See [Choose what clients can change](#choose-what-clients-can-change). - `GET snapshot?threadId=`: the status, running operations, and waiting approvals. +- `GET transcript?threadId=` and `GET describe?threadId=`: the saved messages, and the commands and settings that a client can use. - `GET agent-transcript?threadId=&subagentRunId=`: the saved messages of one child agent of the thread. See [Show what a child wrote after a reload](./subagents#show-what-a-child-wrote-after-a-reload). +- `POST media`, `GET media-url`, and `GET media`: upload a file, get a signed URL, and read the bytes. A signed URL works for 1 hour, and only for a user who can read the thread. See [Send and show media](./media). + +For a list of sessions: + +- `GET sessions`: one page of the sessions that this user can list, newest first. +- `POST sessions`: change a session with `{ op, threadId }`. The ops are `rename`, `update`, `delete`, `fork`, `share`, and `unshare`. `client.sessions` has one method for each op. A thread that is not there throws `404`. See [List, rename, and fork sessions](./sessions) and [Share the thread](./shared-threads#2-share-the-thread). +- `GET host-events`: the status changes and index changes of the sessions that this user can read, as SSE. + +A read of a thread that does not exist answers `404`. Only `GET run?threadId=` answers an empty thread, because `useChat` reads it before the first message. + +The read paths read the thread without a session, so they set up no plugins and claim nothing. `GET describe` opens the session, because the plugins give the commands and the settings. For a read route of your own, use `host.observe`. See [Read a thread without a session](./lifecycle#read-a-thread-without-a-session). ## Cursors Each event id of `GET events` and `GET run`, and the `cursor` of each `harness.event` frame, is an opaque string. Send it back as `Last-Event-ID` or `from`, as it is. Do not parse it. -- On a durable host (`stores.log`), a cursor is a position in the log of the thread. It works on every instance and after a restart. -- Without a log store, a cursor is a counter in the memory of one process. Another instance or a restart does not know it. +A cursor is a position in the log of the thread. It works on every instance that reads the same log store, and after a restart when the store keeps the log. With the memory log of `memoryPersistence()`, only this process knows the log. + +To run more than one instance behind one URL, give every instance the same log store. -To run more than one instance behind one URL, use `stores.log`. +Sometimes the events after a cursor are gone. A host with `keepLogRecords: 'since-snapshot'` drops old records (see [Drop the records before a snapshot](./lifecycle#drop-the-records-before-a-snapshot)), and before its log opens, a session keeps only its newest events in memory. A reader whose cursor is in the lost part gets one `CUSTOM` event first: + +- Its name is `harness.gap` (`HARNESS_EVENTS.gap`). +- Its value is an `EventGap`: `{ from, resumeAt, reason }`. `reason` is `truncated` or `evicted`. Its cursor is `resumeAt`. +- Then the stream goes on from `resumeAt`. + +The session view loads the snapshot and the transcript again when it sees this event. A custom client must do the same. See [When events are missing](./protocol#when-events-are-missing). + +## Keep the stream open + +Every transport sends the same frames: each event is an AG-UI wire chunk in a `harness.event` frame. A long stream stays usable through proxies and slow networks: + +- The first frame of `GET events` and of the socket is `harness.hello`, with the protocol version `v`. A client can ask for a version with `?v=2` or `v: 2`. +- When the stream is idle, the server sends a heartbeat every 20 s. Set the idle timeout of your proxy to more than that. +- A client that does not read for 30 s gets a `slow_consumer` error frame, and the stream closes. Connect again from the last cursor. +- Every answer of the handler has an `X-Request-Id` header, and every error frame has a `code` and a `requestId`. Log the id, to find the request in the server logs. +- To know who has a stream open, pass `presence: { onConnect, onDisconnect }` to `createHarnessHost`. See [Show who is here](./shared-threads#8-show-who-is-here). + +To change the heartbeat or the limits, pass `stream` to `createHarnessHandler` and `handleHarnessSocket`. [The harness wire protocol](./protocol) lists every frame, the error codes, and the `stream` values, for a client in any language. ## Choose what clients can change @@ -81,15 +144,15 @@ export const team = defineHarness({ | Field | What a client can do | | --- | --- | | `agents` | Run these agents, for example with `client.agents..start(input)`. | -| `settings` | Change these [thread settings](./thread-settings) with `client.configure(settings)`. | -| `config` | Set these plugin config keys with `client.setConfig({ key, value })`. | -| `commands` | Run these commands with `client.command(name)`, or with `/name` in a [session view](./custom-ui) on a client. | -| `tools` | Run these harness tools with no model turn, with `client.tool(name, args)`. See [Run a tool without a model](./tool-runs). | -| `records` | Add records of these types to the thread with `client.append(records)`. See [Let a client add records](./session-log#let-a-client-add-records). | +| `settings` | Change these [thread settings](./thread-settings) with `client.updateThreadSettings({ settings })`. | +| `config` | Set these plugin config keys with `client.setPluginSetting({ key, value })`. | +| `commands` | Run these commands with `client.runCommand({ name })`, or with `/name` in a [session view](./custom-ui) on a client. | +| `tools` | Run these harness tools with no model turn, with `client.runTool({ name, args })`. See [Run a tool without a model](./tool-runs). | +| `records` | Add records of these types to the thread with `client.appendRecords({ records })`. See [Let a client add records](./session-log#let-a-client-add-records). | - Any other input of these kinds gets `{ status: 'rejected', reason: 'not_exposed' }`. - `GET describe` and `client.describe()` list only the exposed commands and config keys. A UI built from them shows only what a client can use. -- Server code that calls the session is not limited. `session.setConfig()` and `session.command()` work for every key and command, and `session.describe()` lists all of them. +- Server code that calls the session is not limited. `session.setPluginSetting()` and `session.runCommand()` work for every key and command, and `session.describe()` lists all of them. - `defineHarness` checks the names in `agents` and `tools`, and TypeScript suggests them. Plugins add their config keys and commands when a session opens, so it does not check those names. ## Talk to it from a web app @@ -106,16 +169,22 @@ const client = createHarnessClient({ headers: { authorization: 'Bearer my-token' }, }) +const { cursor } = await client.snapshot() await client.send('Summarize my open tickets.') -for await (const entry of client.events()) { +for await (const entry of client.events({ from: cursor })) { if (entry.event.type === 'TEXT_MESSAGE_CONTENT') { console.log(entry.event.delta) } } ``` -`events()` reconnects after a network error and continues from the last cursor. A second tab, a phone, or a reload all see the same session. +`events()` reconnects after a network error and continues from the last cursor. It also reconnects when no bytes come for 60 s, so a dead connection (a laptop that slept) does not stay `open`. To change that time, pass `idleTimeoutMs` to `createHarnessClient`. Keep it larger than the [heartbeat](./protocol#stream-values) of the server, or the client reconnects again and again. A second tab, a phone, or a reload all see the same session. + +Without `from`, `events()` starts at the newest event, so it sends no old events. That is why the example reads the snapshot cursor before it sends. + +- To read every event from the start of the session, pass `from: '0'`. Do this when you rebuild a transcript from the events. +- If the snapshot answer has no `cursor` (the URL is not a harness handler), `events()` throws and does not try again. To upload files and show the media that your agents make, read [Send and show media](./media). @@ -127,7 +196,8 @@ A background agent runs for minutes, and the user wants to correct it from the p ```ts group=harness-connect-agents import { defineAgent } from '@tanstack/ai' -import { createHarnessHandler, createHarnessHost, defineHarness } from '@tanstack/ai-harness' +import { createHarnessHost, defineHarness, membersOnly } from '@tanstack/ai-harness' +import { createHarnessHandler } from '@tanstack/ai-harness/server' import { memoryPersistence } from '@tanstack/ai-persistence' import { openaiText } from '@tanstack/ai-openai' @@ -144,15 +214,18 @@ export const studio = defineHarness({ expose: { agents: ['drafter'] }, }) -export const handler = createHarnessHandler({ - host: createHarnessHost({ persistence: memoryPersistence() }), - harness: studio, - authorize: (request) => - request.headers.get('authorization') === `Bearer ${process.env.HARNESS_TOKEN}` - ? { id: 'user-1' } - : null, - canAccess: (principal, threadId) => threadId.startsWith(`${principal.id}-`), +const host = createHarnessHost({ + persistence: memoryPersistence(), + access: { + getUser: ({ request }) => + request.headers.get('authorization') === `Bearer ${process.env.HARNESS_TOKEN}` + ? { id: 'user-1' } + : null, + rules: membersOnly(), + }, }) + +export const handler = createHarnessHandler({ host, harness: studio }) ``` 2. In the browser, send the message to the id of the run: @@ -186,7 +259,7 @@ The id of the run is the `operationId` in the receipt of `client.agents.drafter. - `mode: 'steer'` (the default): the message joins the next model call of the run. - `mode: 'followUp'`: the agent runs again after the run ends. `receipt.operationId` is the id of the new run. -`send` with `to` posts an `agentMessage` input to `POST control`. It runs as the user that `authorize` returned. The input looks like this: +`send` with `to` posts an `agentMessage` input to `POST control`. It runs as the user that `getUser` returned. The input looks like this: ```json { "op": "agentMessage", "operationId": "op-agent-m1x2-3", "message": "Add a title.", "mode": "followUp" } @@ -277,19 +350,21 @@ The approval reaches the harness, and the tool runs on the server. What else `PO - Each turn runs as the `runId` of its request. A retry with the same `runId` runs once. The same `runId` with another message gets `409`. - A tool with only a `.client()` implementation runs in the browser, and the same turn gets its result. -- A question from `ctx.session.ask` arrives on the stream of the turn that waits for it. +- A question of `step.ask` or of an MCP server arrives on the stream of the turn that waits for it. See [Wait for a person](./waits). - The `forwardedProps` of the request are the input's [context](./inputs#send-context-with-a-message). - With `persistence: true`, `useChat` loads the thread from `GET run?threadId=`. - If a turn still runs after a reload, `useChat` joins it with `GET run?runId=`. The turn shows from its first event, and its answer continues to stream. -- On a durable host (`stores.log` and `stores.runs`), each event of `POST run` has its log cursor as its id. If the connection drops, `useChat` sends the request again with `Last-Event-ID`. Any server continues the stream from there, and the turn does not run again. +- Each event of `POST run` has its log cursor as its id. If the connection drops, `useChat` sends the request again with `Last-Event-ID`. Any server continues the stream from there, and the turn does not run again. -To keep the thread after a reload, set `persistence: true` in `useChat`: +### Join a turn after a reload + +To keep the thread after a reload, set `persistence: true` in `useChat`. Put the `threadId` in the URL of the connection too: ```tsx group=harness-connect-tools-client export function ReloadableChat() { const { messages } = useChat({ threadId: 'user-1-thread', - connection: fetchServerSentEvents('/api/harness/run'), + connection: fetchServerSentEvents('/api/harness/run?threadId=user-1-thread'), tools: [deploy.client()] as const, persistence: true, }) @@ -299,31 +374,107 @@ export function ReloadableChat() { The join has these rules: -- `canAccess` decides who can join, the same as for every other path. -- On a durable host, any server can answer the join. It reads the turn from the shared log. -- Without a log store, the turn must run on the host that answers the request. On another host, the join gets `404`. +- The host checks that the user can read the thread, the same as for every other path. +- The join request of `useChat` sends only the run id. With `threadId` in the URL, any instance that reads the same log store can answer the join. It reads the turn from the log of that thread. +- Without `threadId`, only the instance that runs the turn finds it. Another instance answers `400`. - A turn that ended gives the events that the session still keeps, and then the stream ends. +### Stop a turn from useChat + +The `stop()` of `useChat` closes the request, but the turn continues on the server. A closed connection never cancels work, because another tab or a reload can still read the turn. To stop the turn too, send a `cancel` input for its run. + +1. On the server, serve the harness with one handler. `POST run` streams the turn to `useChat`, and `POST control` takes the cancel: + +```ts group=harness-connect-stop-server +import { createHarnessHost, defineHarness, membersOnly } from '@tanstack/ai-harness' +import { createHarnessHandler } from '@tanstack/ai-harness/server' +import { memoryPersistence } from '@tanstack/ai-persistence' +import { openaiText } from '@tanstack/ai-openai' + +export const writer = defineHarness({ + name: 'acme/writer', + adapter: openaiText('gpt-6.1-sol'), +}) + +const host = createHarnessHost({ + persistence: memoryPersistence(), + access: { + getUser: ({ request }) => + request.headers.get('authorization') === `Bearer ${process.env.HARNESS_TOKEN}` + ? { id: 'user-1' } + : null, + rules: membersOnly(), + }, +}) + +// Mount it on `/api/harness/*`. +export const handler = createHarnessHandler({ host, harness: writer }) +``` + +2. In the browser, call `client.cancel` with the `runId` of `useChat` in your stop handler. The turn of `POST run` runs as that id: + +```tsx group=harness-connect-stop-client +import { fetchServerSentEvents, useChat } from '@tanstack/ai-react' +import { createHarnessClient } from '@tanstack/ai-harness/client' +import type { writer } from './harness' + +const headers = { authorization: 'Bearer my-token' } + +const client = createHarnessClient({ + url: '/api/harness', + threadId: 'user-1-thread', + headers, +}) + +export function StoppableChat() { + const { messages, runId, isLoading, stop, sendMessage } = useChat({ + threadId: 'user-1-thread', + connection: fetchServerSentEvents('/api/harness/run', { headers }), + }) + + const stopTurn = async () => { + stop() + if (!runId) return + const receipt = await client.cancel({ operationId: runId }) + if (receipt.status === 'rejected') console.warn(receipt.reason) + } + + return ( + <> +

{messages.length} messages

+ + {isLoading ? : null} + + ) +} +``` + +- The cancel works on any server that reads the same log store. The input goes to the server that runs the turn. +- If the turn already ended, the receipt can be `rejected` with the reason `not_running`. +- The turn ends with the status `cancelled`. + ## Use a WebSocket -For one connection that carries events and inputs, authorize the upgrade, then hand the socket to `handleHarnessSocket`: +For one connection that carries events and inputs, hand the socket and its upgrade request to `handleHarnessSocket`: ```ts group=harness-connect -import { handleHarnessSocket } from '@tanstack/ai-harness' +import { handleHarnessSocket } from '@tanstack/ai-harness/server' import type { WebSocketLike } from '@tanstack/ai' -export function onUpgrade(socket: WebSocketLike) { - handleHarnessSocket({ - host, - harness: assistant, - socket, - principal: { id: 'user-1' }, - canAccess: (principal, threadId) => threadId.startsWith(`${principal.id}-`), - }) +export function onUpgrade(socket: WebSocketLike, request: Request) { + handleHarnessSocket({ host, harness: assistant, socket, request }) } ``` -The client sends `{ type: 'harness.subscribe', threadId }` first. After that, each `harness.input` frame gets a `harness.receipt`, and every event arrives as a `harness.event` frame with its cursor. +The client sends `{ type: 'harness.subscribe', threadId }` first, and the host answers with a `harness.hello` frame. After that, each `harness.input` frame gets a `harness.receipt`, and every event arrives as a `harness.event` frame with its cursor. + +- The host reads the user from the upgrade request with `getUser`, the same as for HTTP. Every input on the socket runs as that user. +- A user that `getUser` refuses gets an `unauthorized` error frame, and the socket closes. +- The socket handles the frames of one connection in the order they came. So a prompt with a large image reaches the session before the frame after it. +- A user who cannot read the thread, or who loses access later, gets a `forbidden` error frame, and the socket closes. +- A frame over `media.maxBodyBytes` (default 32 MiB) gets an error frame, and the socket closes with code `1009`. Your WebSocket server reads the whole frame before this check, so the check does not cap memory. Set the frame limit of your server too, for example `maxPayload` of `ws`. +- On a host with no `access`, pass `openAccess`, the same as for `createHarnessHandler`. +- Pass `socket: true` to `createHarnessHandler` too. Then `GET capabilities` says `transport.websocket: true`, so clients know that they can use the socket. The default is `false`. ## Use it as the model of another chat @@ -342,13 +493,18 @@ const stream = chat({ Each outer thread gets its own inner session, so the harness keeps its own history. The inner session gets every content part of the last user message, so images and files go through too. +- Who runs the inner turn: when the outer `chat()` runs inside a harness turn, the inner session opens as the user of that turn. Else it opens as `user`, for example `harnessText(assistant, { host, user: { id: 'ada' } })`. +- Access: the inner thread is `/inner/`. When `host` has `access` and also runs the outer thread, the inner session is checked through the outer thread, so every user who can read a shared outer thread can use its inner session. From app code, `host.open`, `host.submit`, `host.fork`, and `host.transfer` refuse any thread id with `/inner/`. +- Usage: the outer run gets the token usage of every inner model call, so the usage totals of the outer session count them. +- Cancel: an abort of the outer call cancels the inner turn. For a remote harness, `harnessText({ url, token })` also tells the remote host to cancel the turn. + If a plugin picks the model of the harness, the harness has no adapter to read its input kinds from. Pass them with `harnessText(assistant, { host, inputModalities: ['text', 'image'] })`. ## What you have now -- One handler that serves standard AG-UI and the session stream. +- One handler that serves standard AG-UI and the session stream, as the user of each request. - Clients that can change only what you expose. -- A `useChat` screen on the harness, with approvals, tools that run in the browser, and a reload that joins the running turn. +- A `useChat` screen on the harness, with approvals, tools that run in the browser, a reload that joins the running turn, and a stop button that cancels the turn. - A typed web client that reconnects and resumes from a cursor. - Messages from the browser to a running agent. - A WebSocket, and a harness you can call from `chat()`. diff --git a/docs/harness/custom-ui.md b/docs/harness/custom-ui.md index d262a8d1da..e1a2ed9d06 100644 --- a/docs/harness/custom-ui.md +++ b/docs/harness/custom-ui.md @@ -123,18 +123,20 @@ The conversation: - `status`: `idle`, `running`, or `requires_action`. - `connection`: `open`, `reconnecting`, or `closed`. -What waits for the user: +What waits for the user. Each item is a [wait](./waits) of the session: - `approvals`: tool calls that wait for a yes or a no. - `clientTools`: tool calls that run in your UI and wait for their output. See [Run a tool in the browser](#run-a-tool-in-the-browser). - `interrupts`: everything else a turn can wait for, for example a value that a tool asks for. See [Answer other interrupts](#answer-other-interrupts). -- `questions`: questions from a command, a plugin, or an MCP server. See [Show a question](#show-a-question). +- `questions`: questions of `step.ask` (in a command, a durable tool, or an agent run) and of an MCP server. See [Show a question](#show-a-question). - `signIns`: connectors that need a sign-in, with a `url` and a `userCode` when the connector gives them. +`waits` lists every open wait in one array, as `snapshot().pendingWaits` does, with the `kind` of each one. Use it for a badge or a count. Answer each wait through its item in the lists above. + The session: - `threadId`: the id of the conversation. -- `agents`: the background agents that run now. Each one has an `id`, a `name`, `send`, and `cancel`. +- `agents`: the background agents that run now. Each one has an `id`, a `name`, a `status` (`running`, or `waiting` for an answer), `send`, and `cancel`. - `queuedTurns` and `waitingInputs`: how many messages wait for their turn, and which ones, in the order they run. - `plugins`: the saved state of each plugin, by plugin name. @@ -145,6 +147,8 @@ Settings and usage, for a settings panel or a status bar: - `settings`: the stored settings of the thread, for example the model it uses now. - `usage`: the token usage of the thread. It updates after each model call. +In a harness with `routing`, an assistant message also has `agent`: the name of the root agent that wrote it, or `undefined` for the main model. After a [handoff](./handoffs), the next agent writes a new message in the same turn. + An assistant message has `parts`. Each part is one of these: - `text` or `reasoning`: the text, which grows while it streams. @@ -158,35 +162,34 @@ A notice has a `kind`: - `error`: a turn or an action failed. - `rejected`: the session did not accept a message. - `command`: the text result of a command. -- `ui`: a line that you added with `view.notice(text)`. +- `ui`: a line that you added with `view.notice({ text })`. ## 3. Act on the session Send work: -- `view.send(text)`: sends a prompt. While a turn runs, the text steers that turn. Text that starts with `/` runs a command, for example `/goal all tests pass`. +- `view.send(text)`: sends a prompt. While a turn runs, the text steers that turn. Text that starts with `/` runs a command, for example `/goal all tests pass`. With `{ as: 'operator' }`, it sends the [reply of an operator](./handoffs#reply-as-the-operator), and no model runs. - `view.followUp(text)`: the message waits, and runs as its own turn after the current work. -- `view.continue()`: starts a turn from the saved messages, with no new message. -- `view.command(name, input)`: runs a command, for example from a button. -- `view.upload(bytes, { name, mimeType })`: stores a file. Send the record you get back with `view.send(text, [record])`. See [Show media](#show-media). - +- `view.continueWithoutMessage()`: starts a turn from the saved messages, with no new message. +- `view.runCommand({ name, input })`: runs a command, for example from a button. +- `view.putMedia({ body, mimeType, name })`: stores a file. Send the record you get back with `view.send(text, [record])`. See [Show media](#show-media). Change the session: -- `view.setConfig({ key, value })`: changes a config value, for example `view.setConfig({ key: 'mode', value: 'plan' })`. -- `view.configure(settings)`: changes the stored settings of the thread, for example `view.configure({ model: 'strong' })`. -- `view.reset(note)`: the model starts fresh from the next turn, with your `note` first. See [Reset with a handoff note](./fork-and-reset#reset-with-a-handoff-note). -- `view.revert(messageId)` and `view.unrevert()`: go back to a message, and undo that. +- `view.setPluginSetting({ key, value })`: changes a config value, for example `view.setPluginSetting({ key: 'mode', value: 'plan' })`. +- `view.updateThreadSettings({ settings })`: changes the stored settings of the thread, for example `view.updateThreadSettings({ settings: { model: 'strong' } })`. +- `view.reset({ note })`: the model starts fresh from the next turn, with your `note` first. See [Reset with a handoff note](./fork-and-reset#reset-with-a-handoff-note). +- `view.revert({ messageId })` and `view.unrevert()`: go back to a message, and undo that. Stop work: -- `view.cancel()`: cancels the running turn. `view.cancel(operationId)` cancels one operation, for example a background agent. -- `view.cancelInput(inputId)`: drops a message from `waitingInputs` before it runs. `view.setDelivery({ inputId, delivery: 'steer' })` moves it into the running turn instead. +- `view.cancel()`: cancels the running turn. `view.cancel({ operationId })` cancels one operation, for example a background agent. +- `view.cancelInput({ inputId })`: drops a message from `waitingInputs` before it runs. `view.changeInputDelivery({ inputId, delivery: 'steer' })` moves it into the running turn instead. - `view.dispose()`: stops the view when your screen closes. After that, each action throws an error. On a view with a [`HarnessClient`](#a-ui-in-the-browser), two actions need `expose` in the harness: - `view.command`, and a `/command` in `view.send`, run only the commands in `expose.commands`. -- `view.setConfig` sets only the keys in `expose.config`. +- `view.setPluginSetting` sets only the keys in `expose.config`. For any other command or key, the view adds a `rejected` notice with the text `Not accepted: not_exposed`. It also calls your `'error'` handler with that text. On a client, `commands` and `config` in the state list only the exposed items. @@ -194,7 +197,7 @@ See [Choose what clients can change](./connect#choose-what-clients-can-change). Answer what waits: -- `approval.approve()` and `approval.reject()`: answer one approval. `view.approve(id)` and `view.reject(id)` do the same by id. +- `approval.approve()` and `approval.reject()`: answer one approval. `view.approve({ id })` and `view.reject({ id })` do the same by id. - `view.approveAll()` and `view.rejectAll()`: answer all open approvals. - `call.resolve(output)` and `call.fail(message)`: answer one client tool call. - `request.resolve(value)` and `request.cancel()`: answer one item of `interrupts`. @@ -207,10 +210,12 @@ const [waiting] = view.store.get().approvals waiting?.approve({ editedArgs: { path: 'notes/draft.md' } }) ``` -`reject(payload)` sends data with a no, when the tool's approval schema takes some. +`reject({ payload })` sends data with a no, when the tool's approval schema takes some. If a turn waits for more than one approval, client tool, or interrupt, the turn continues after you answer all of them. +A background agent run can wait too, for example for an approval of `permissions()`. Its waits are in the same `approvals`, `clientTools`, and `interrupts` lists. The view sends the answer to each of them at once, and the run continues. See [When a background agent waits](./subagents#when-a-background-agent-waits). + Message a background agent: - `agent.send(text, { mode })`: sends a message to one item of `state.agents`. `mode` is `'steer'` (the default) or `'followUp'`. See [Message a running agent](./subagents#message-a-running-agent). @@ -224,7 +229,19 @@ if (agent) { `agent.cancel()` stops the run, and the runs it started. -A session and a `HarnessClient` can do all of this. If you give `createSessionView` your own `SessionViewSource`, `send`, `resolve`, and `cancel` are required. `send` takes `mode` for a steer or a follow-up, and `to` for a message to an agent run. Without `reset` or another optional member, its action adds an `error` notice instead. +A session and a `HarnessClient` can do all of this. You can also give `createSessionView` your own `SessionViewSource`. Its methods take the same arguments as the session methods, for example `resolve({ resume })` and `cancel({ operationId })`. `send` takes `mode` for a steer or a follow-up, and `to` for a message to an agent run. + +A source must have these members: + +- Inputs: `send`, `resolve`, `cancel`, `answer`, `runCommand`, and `setPluginSetting`. +- Reads: `events`, `snapshot`, `transcript`, and `describe`. + +These members are optional: + +- `continueWithoutMessage`, `cancelInput`, `changeInputDelivery`, `updateThreadSettings`, `reset`, `revert`, and `unrevert`. Without one, its view action adds an `error` notice. +- `putMedia`. Without it, `view.putMedia` rejects. +- `mediaUrl` and `loadMedia`. Without `mediaUrl`, media parts have no `url`. Without `loadMedia`, `load()` rejects. +- `agentTranscript`. Without it, `view.loadAgent` does nothing, so a saved child shows only its card. ## 4. Listen for events @@ -233,7 +250,7 @@ Some items need attention when they arrive. `view.on` calls your handler for eac ```ts group=harness-custom-ui view.on('approval', (approval) => { process.stdout.write('\u0007') - view.notice(`${approval.tool} waits for your answer.`) + view.notice({ text: `${approval.tool} waits for your answer.` }) }) ``` @@ -254,9 +271,14 @@ Progress: The view calls your handler after the item is in the state, so the handler can read `view.store.get()` and find the item. `view.on` returns a function that removes the handler. +The view also keeps its state right on its own: + +- Each event from `client.events()` or `session.events()` has the `threadId` it belongs to. The `RUN_STARTED` event of a child run has the `parentRunId` of the run that started it. Use them to show the run tree in your own UI. +- When the server sends `harness.gap`, some events are gone. The view stops its read, loads the snapshot and the transcript again, then reads on from the new snapshot. So no turn shows twice after the gap. See [Write a client in any language](./protocol#when-events-are-missing). + ## Show a question -A command or a plugin can ask the user a question. Each item of `state.questions` has: +A command, a durable tool, or an agent run can ask the user with `step.ask`. An MCP server can ask during a tool call. Each item of `state.questions` has: - `id` and `message`. - `schema`: the JSON Schema of the answer, when the question has one. Without a schema, the answer is text. @@ -268,7 +290,7 @@ When a question has a `url`, show the link. The user opens it, then answers. The ```ts group=harness-custom-ui view.on('question', (question) => { - if (question.url) view.notice(`Open ${question.url}, then answer.`) + if (question.url) view.notice({ text: `Open ${question.url}, then answer.` }) }) ``` @@ -321,7 +343,7 @@ const extra = fieldsThatApply(question?.schema, { delivery: 'mail' }) ## Answer other interrupts -Some turns wait for something that is not a yes or a no. A tool can stop the turn to ask for a value, or your own code can stop it for a form. The view puts these in `state.interrupts`. Each item has: +Some turns wait for something that is not a yes or a no. A tool can stop the turn to ask for a value, or your own code can stop it for a form. The view puts these in `state.interrupts`: the waits of kind `question` (for example a call of the `question` tool) and `human`. Each item has: - `id`, `reason`, and `message`: what the turn waits for. `reason` is the name the interrupt gave itself, for example `pick_region`. - `payload`: the data the interrupt sent for your UI, for example the fields of a form. @@ -333,7 +355,7 @@ This handler shows each one, and answers the region picker right away: ```ts group=harness-custom-ui view.on('interrupt', (request) => { - view.notice(request.message ?? `The turn waits for ${request.reason}.`) + view.notice({ text: request.message ?? `The turn waits for ${request.reason}.` }) if (request.reason === 'pick_region') request.resolve({ region: 'eu-west-1' }) }) ``` @@ -395,14 +417,15 @@ To send files with a message, give their records to `view.send`. This code sends ```ts group=harness-custom-ui import { readFile } from 'node:fs/promises' -const screenshot = await session.putMedia(await readFile('./bug.png'), { +const screenshot = await session.putMedia({ + body: await readFile('./bug.png'), mimeType: 'image/png', name: 'bug.png', }) await view.send('What is wrong in this screenshot?', [screenshot]) ``` -In a browser, get the record from `client.upload(file, { name: file.name, mimeType: file.type })`. The user message then shows the file in `media`. +In a browser, get the record from `client.putMedia({ body: file, mimeType: file.type, name: file.name })`. The user message then shows the file in `media`. ## Show subagents @@ -411,13 +434,26 @@ A child agent is an `agent` part in the assistant message of the turn that start - `status`: `running`, `done`, or `failed`, and `error` when it failed. - `result`: the typed result of the agent, when it gives one. -After a reload, each child shows up again with what it wrote, not only with its final result. The view reads the saved messages of each child (see [Show what a child wrote after a reload](./subagents#show-what-a-child-wrote-after-a-reload)) and fills its part. A background child that still runs keeps getting its new text in the same part. +After a reload, each saved child shows only its card: `name`, `status`, `error`, and `result`, with no `parts`. To show what the child wrote, call `view.loadAgent({ id })`, for example when the user opens the child. The view reads the saved messages of the child (see [Show what a child wrote after a reload](./subagents#show-what-a-child-wrote-after-a-reload)) and fills its part: + +```ts group=harness-custom-ui +const last = view.store.get().messages.at(-1) +const child = + last?.role === 'assistant' + ? last.parts.find((part) => part.type === 'agent') + : undefined +if (child?.type === 'agent') await view.loadAgent({ id: child.id }) +``` + +A nested child in that part loads the same way, with its own `loadAgent` call. A background child that still runs keeps getting its new text in the same part. + +A terminal screen that prints the old turns only one time (for example Ink `Static`) must load the saved children before its first render. [`examples/harness-cli/src/tui.tsx`](https://github.com/TanStack/ai/blob/main/examples/harness-cli/src/tui.tsx) does this in `loadSavedAgents`. To show a line when a child ends, listen for `'agent'`: ```ts group=harness-custom-ui view.on('agent', (child) => { - if (child.status !== 'running') view.notice(`${child.name} is ${child.status}.`) + if (child.status !== 'running') view.notice({ text: `${child.name} is ${child.status}.` }) }) ``` @@ -432,7 +468,7 @@ import { createPluginEvent } from '@tanstack/ai-harness' const Deployed = createPluginEvent<{ env: string }>('acme/deploy:done') -view.on(Deployed, (done) => view.notice(`Deployed to ${done.env}`)) +view.on(Deployed, (done) => view.notice({ text: `Deployed to ${done.env}` })) ``` `state.plugins` is plugin data, so check its shape before you use it. @@ -446,7 +482,8 @@ Your harness runs on a server, and your UI is a web page. Give `createSessionVie Serve the session with `createHarnessHandler`, and mount the handler on `/api/harness`: ```ts group=harness-custom-ui-server -import { createHarnessHandler, createHarnessHost, defineHarness } from '@tanstack/ai-harness' +import { createHarnessHost, defineHarness, membersOnly } from '@tanstack/ai-harness' +import { createHarnessHandler } from '@tanstack/ai-harness/server' import { memoryPersistence } from '@tanstack/ai-persistence' import { openaiText } from '@tanstack/ai-openai' @@ -455,21 +492,26 @@ export const assistant = defineHarness({ adapter: openaiText('gpt-6.1-sol'), }) -const host = createHarnessHost({ persistence: memoryPersistence() }) - -export const handler = createHarnessHandler({ - host, - harness: assistant, - authorize: (request) => - request.headers.get('authorization') === `Bearer ${process.env.HARNESS_TOKEN}` - ? { id: 'user-1' } - : null, - canAccess: (principal, threadId) => threadId.startsWith(principal.id), +const host = createHarnessHost({ + persistence: memoryPersistence(), + access: { + getUser: ({ request }) => + request.headers.get('authorization') === `Bearer ${process.env.HARNESS_TOKEN}` + ? { id: 'user-1' } + : null, + rules: membersOnly(), + }, }) + +export const handler = createHarnessHandler({ host, harness: assistant }) ``` +`membersOnly()` lets each user use the threads that they opened, and the threads shared with them. See [Access and tenants](./access). + The handler also serves `GET transcript` and `GET describe`. The view reads them when it starts. For the other routes, see [Connect clients](./connect). +`GET transcript`, `GET snapshot`, `GET events`, and the other read paths read the thread without a session, so they start no plugins on the server. `GET describe` opens the session, because the plugins give the commands and the settings. To read a thread in a route of your own, use `host.observe`. See [Read a thread without a session](./lifecycle#read-a-thread-without-a-session). + ### In the browser Make the view one time, in code that runs in the browser. Then read it in your components: diff --git a/docs/harness/deploy.md b/docs/harness/deploy.md index 5ac168fe12..9eee49458c 100644 --- a/docs/harness/deploy.md +++ b/docs/harness/deploy.md @@ -2,7 +2,7 @@ title: Deploy a harness id: harness-deploy order: 6 -description: "Run a harness in your server, as a worker process, on another machine, or as a single executable." +description: "Run a harness in your server, on background workers, as a worker process, on another machine, or as a single executable." keywords: - tanstack ai - harness @@ -16,10 +16,21 @@ Your harness works on your laptop. Now it needs to run next to your API, in its | Place | How | |---|---| | Your server | `createHarnessHost` and `createHarnessHandler` in your routes. No build step. | +| Background workers | `createHarnessWorker` on a host that shares the log store. See [Run background work on workers](./workers). | | A worker process | `buildHarness`, then `artifactText(dir)` as the model of a `chat()` call. | | Another machine | `createHarnessHandler` there, then `harnessText({ url, token })` here. | | A single executable | `buildHarness({ compile })` with Bun. | +## Run more than one server + +Give every server the same log store. Then any server can take a request for any thread: + +- One server owns a thread at a time. An input that reaches another server waits in the log for the owner. See [One host drives a thread](./durable-sessions#one-host-drives-a-thread). +- A [worker](./workers) runs the threads that no server has open: submitted inputs, schedules, and the work that a deploy cut. +- Set the same `media: { secret }` on the handler of each server, so each server checks the media URLs of the others. See [Serve the session over HTTP](./connect#serve-the-session-over-http). +- Stop each server with `host.close({ recoverable: true })`. A worker takes its cut work at once. See [Stop a host for a deploy](./durable-sessions#stop-a-host-for-a-deploy). +- If your agents use the [coding tools](./coding-tools) on the disk of a server, set `ownership: { hostId }` on each host. Then a thread stays on the server that has its folder, while that server sends heartbeats. Use an id that stays the same after a restart, like the pod name of a StatefulSet, and keep a restart shorter than `ownership.expiresAfterMs`. See [Move a thread to another server](./coding-workspaces#5-move-a-thread-to-another-server). + ## Build a worker artifact `buildHarness` bundles the harness with Bun and writes a manifest next to it. Install [Bun](https://bun.sh) on the build machine. @@ -61,7 +72,7 @@ The worker keeps the conversation of its thread between calls. Workers get the e ## Use a harness on another machine -On the other machine, serve the harness with `createHarnessHandler`, and check a bearer token in `authorize`. See [Serve the session over HTTP](./connect#serve-the-session-over-http). +On the other machine, serve the harness with `createHarnessHandler`. Check the bearer token in `getUser` of the host `access`. See [Serve the session over HTTP](./connect#serve-the-session-over-http). Here, use it as a model: @@ -88,6 +99,82 @@ await buildHarness({ Bun can build for another platform with `target`, for example `bun-darwin-arm64` or `bun-windows-x64`. +## Trace runs and report operations + +A turn is slow or fails on one of your servers, and you cannot see which run or which server. Give the host a tracer and an `onOperation` listener. The host then writes a span for each run, and tells your listener when each run starts and ends. + +Pass an OpenTelemetry tracer as it is. Install `@opentelemetry/api` in your app first. It is an optional peer dependency of the harness. + +```ts +import { trace } from '@opentelemetry/api' +import { createHarnessHost } from '@tanstack/ai-harness' + +const host = createHarnessHost({ + observability: { + tracer: trace.getTracer('support-app'), + onOperation: ({ phase, operation }) => { + if (phase !== 'settled') return + console.log(operation.kind, operation.status, operation.durationMs) + console.log('tokens', operation.usage?.totalTokens) + }, + }, +}) +``` + +The host writes these spans: + +| Span | When | +|---|---| +| `harness.operation` | For each turn, agent run, and command that this host runs. A child agent run on the same host is a child span of the run that started it. | +| `harness.recovery` | This host takes the work of a host that stopped. | +| `harness.wait` | A wait for a person ends. The span starts at the time that the wait opened, also when another host opened it. | +| `harness.transfer` | `host.transfer` moves a conversation. | + +Each span has `harness.thread_id`, and `harness.tenant_id` for a tenant. A `harness.operation` span also has `harness.run_id`, `harness.parent_run_id`, `harness.operation.kind`, `harness.agent`, and `harness.operation.status`. + +`onOperation` gets one `started` report and one `settled` report for each operation: + +- Only the host that runs the operation calls it. A server that only passed the input on reports nothing. +- A `settled` report has `status`, `durationMs`, and `usage`. The usage counts the model calls of the run, of its child runs, and of plugins in the run. +- A failed operation has `error`, with its `message`, and its `code` when it has one. +- If your listener throws, the host sends a `harness.plugin.warning` event and goes on. + +Without `observability`, the host writes no spans and does no work for them. + +### Join a turn to the trace of your app + +Send the `traceparent` of your current span with the message: + +```ts +import { context, propagation } from '@opentelemetry/api' +import { createHarnessClient } from '@tanstack/ai-harness/client' + +const client = createHarnessClient({ url: '/api/harness', threadId: 'thread-1' }) + +const carrier: Record = {} +propagation.inject(context.active(), carrier) +const traceparent = carrier.traceparent +await client.send('Where is my order?', traceparent ? { traceparent } : {}) +``` + +The client sends it as the `traceparent` header. The `harness.operation` span of the turn gets a link to your span, on the server that runs the turn. To send it from another language, see [Traces](./protocol#traces). + +## Known limits + +Check these before you run many servers or long threads. + +Across servers and restarts: + +- A [`harnessAgent`](./subagents#call-a-harness-as-a-child) child waits only in memory. Its parent run waits on the same host. Another host, or a crash, does not resume it. After a crash, the next owner of the log ends the waits of the child. +- A [log snapshot](./durable-sessions#keep-a-long-log-fast-to-open) holds the full fold of the thread, so it grows with the thread. To keep the log small, drop the old records with [`keepLogRecords: 'since-snapshot'`](./lifecycle#drop-the-records-before-a-snapshot). +- `host.resumePending` opens the threads that it takes one at a time. A `GET run` by run id looks through the open sessions one at a time too. + +Users and plugins: + +- With an [OAuth MCP server](./mcp#3-sign-in-to-an-oauth-server), the MCP SDK sends its own refresh request for each session. Only the save of the new token is single-flight, so two sessions can use a rotating refresh token at the same time. +- Plugin tools do not reach background agents. The `question` tool works only in a turn of the lead agent. + ## What you have now -- The same harness in your server, in a worker process, on another machine, or as a binary. +- The same harness in your server, on background workers, in a worker process, on another machine, or as a binary. +- A span and a report for each run, linked to the trace of your app. diff --git a/docs/harness/durable-sessions.md b/docs/harness/durable-sessions.md index d18ea03936..f87e131090 100644 --- a/docs/harness/durable-sessions.md +++ b/docs/harness/durable-sessions.md @@ -2,38 +2,51 @@ title: Durable sessions id: harness-durable-sessions order: 4 -description: "Keep a harness turn alive through a crash or a deploy. The next host continues it from the session log, and a turn that keeps failing stops after a limit." +description: "Keep a harness turn alive through a crash or a deploy. The next host continues it from the session log, one host owns a thread at a time, and a turn that keeps failing stops after a limit." keywords: - tanstack ai - harness - durability - resume - session log + - ownership - retries --- -A long agent turn can outlive the process that started it: a deploy restarts the server, or the machine crashes in the middle of a tool call. Give the host a session log, and the next host that opens the thread continues the turn. A turn that crashes the host every time stops after a limit. +A long agent turn can outlive the process that started it: a deploy restarts the server, or the machine crashes in the middle of a tool call. Every host keeps each thread in a session log. Put that log in a store that survives a restart, and the next host continues the turn. A turn that crashes the host every time stops after a limit. ## Give the host a session log -A durable host takes two stores: - -- `log`: one append-only log per thread. It keeps the events, the transcript, the inputs, and the tool steps. -- `runs` or `leases`: a lease tells the next host that the last one stopped. `runs` keeps run records with leases. `leases` is a store that you wire to your own lease system. See [Use your own leases](#use-your-own-leases). +The host needs one store: `log`. It keeps the events, the transcript, the inputs, the runs, the interrupts, the tool steps, and the state of each thread. It also records which host owns each thread. ```ts group=harness-durable import { createHarnessHost } from '@tanstack/ai-harness' -import { memoryLogStore, memoryPersistence } from '@tanstack/ai-persistence' +import { memoryPersistence } from '@tanstack/ai-persistence' // Swap in your own stores. Memory stores do not survive a restart. -const { runs, metadata } = memoryPersistence().stores +const host = createHarnessHost({ persistence: memoryPersistence() }) +``` + +- `memoryPersistence()` has a memory `log`. A host with no `persistence` uses it too. +- To keep the log in your database, implement the [`LogStore` contract](../persistence/store-reference#logstore). A host whose log store survives a restart is a durable host. +- `metadata` is optional. It keeps data wider than one thread, for example the saved rules of `permissions()`. + +### Keep a long log fast to open + +A thread that runs for weeks has a long log, and a new host reads all of it to open the thread. The owner of the log writes a snapshot of the folded log to the log store. A new host opens from the snapshot and reads only the records after it. Set when the owner writes one: -const host = createHarnessHost({ - persistence: { stores: { log: memoryLogStore(), runs, metadata } }, +```ts group=harness-durable +const snapshotHost = createHarnessHost({ + persistence: memoryPersistence(), + log: { saveSnapshotAfter: { bytes: 500_000, ms: 60_000 } }, }) ``` -`metadata` is optional. With it, the host keeps a fold checkpoint of each log, so a long thread opens faster. It also keeps the interrupts that the last turn waits on, so `session.resolve` continues that turn on the next host. To store the log in your database, implement the [`LogStore` contract](../persistence/store-reference#logstore). +- `bytes`: after this many bytes of new records, as the JSON length of the records. The default is 1 000 000. +- `ms`: this long after the first record that no snapshot has. The default is 300 000 (5 minutes). +- The first limit that the log reaches wins. Only the owner writes snapshots. The type is `SnapshotTiming`. + +To also drop the old records, set `lifecycle.retention.keepLogRecords` to `'since-snapshot'`. See [Drop the records before a snapshot](./lifecycle#drop-the-records-before-a-snapshot). ## Limit attempts and time @@ -57,7 +70,7 @@ A turn that passes its time limit is stopped. It ends with the error code `timeo ## What happens after a crash -When a host opens the thread and finds a turn whose lease expired, it checks these, in order: +A host that stops also stops renewing its ownership of the thread. When the ownership expires, the next host that takes the thread finds the turn that did not end. It checks these, in order: 1. The log already has the final answer: the input ends `completed`. This wins over the checks below, so finished work is never lost. 2. An abort was asked for the turn: the input ends `aborted`. The model is not called. @@ -65,19 +78,19 @@ When a host opens the thread and finds a turn whose lease expired, it checks the 4. The time limit passed: the input ends `failed`, with the error code `timeout`. 5. Else the turn runs again as the next attempt, and clients get a `harness.operation.resumed` event. -An input that was stored but never ran, runs now. `session.settled(inputId)` resolves to how an input ended. See [Send inputs safely](./inputs). +An input that was stored but never ran, runs now. `session.settled({ inputId })` resolves to how an input ended. See [Send inputs safely](./inputs). -A background agent that you started with `resume: true` runs again on the next host. See [Keep a background agent going after a crash](./subagents#keep-a-background-agent-going-after-a-crash). For any other background agent, when its lease expired, the next host: +A background agent that you started with `runAgainAfterCrash: true` runs again on the next host. A background child that the model started with `background: true` runs again too, unless you set `durability: { resumeBackgroundAgents: false }`. See [Keep a background agent going after a crash](./subagents#keep-a-background-agent-going-after-a-crash). The children and groups that it started before the stop do not start again: they give their stored results. See [Children of a run that runs again](./subagents#children-of-a-run-that-runs-again). For any other background agent whose host stopped, the next host: 1. Ends the run `failed`, with the message `The host stopped during this agent run.` 2. Adds a note about the failure to the transcript. -3. Starts a new turn, if you started the agent with `{ wake: true }`. +3. Starts a new turn, if you started the agent with `{ startTurnWhenDone: true }`. `durability.recover` does not run for background agents. ## Stop a host for a deploy -`host.close()` ends each running turn `aborted`, so no host runs it again. When the server stops only for a deploy, close it with `recoverable: true`. The running turns stop, but they stay open in the log, and the next host continues them at once: +`host.close()` closes every session of the host and refuses later opens. It ends each running turn `aborted`, so no host runs it again. When the server stops only for a deploy, close it with `recoverable: true`. The running turns stop, but they stay open in the log, and another host continues them at once: ```ts group=harness-durable process.on('SIGTERM', async () => { @@ -86,92 +99,42 @@ process.on('SIGTERM', async () => { }) ``` -- Running turns and background agents stop. The host writes no end for them and gives their leases back, so the next host does not wait for the lease to expire. +- Running turns and background agents stop. The host writes no end for them. +- The host gives each busy thread back with a wake. A [worker](./workers) takes it at once, so it does not wait for the ownership to expire. - A tool that the stop cut gets the same result as after a crash. See [Mark tools that are safe to run again](#mark-tools-that-are-safe-to-run-again). - A turn that a user cancelled still ends `aborted`. -- `session.close({ recoverable: true })` does the same for one session. +- `session.close()` releases only your handle. It does not stop the work of the session. See [Lifecycle and retention](./lifecycle#open-and-release-a-session). During a rolling deploy, old and new hosts run at the same time. A host that reads a log record type from a newer version does not write to that log. Its session stops with `LogVersionError`, and the new hosts continue the thread. ## Resume pending work after a deploy -A host recovers a thread when something opens it. After a deploy, nobody opens a thread whose user closed the tab, so its queued turn or background agent waits. Give the host `stores.workClaims`, then call `host.resumePending` to find that work and run it. - -Add `workClaims` to the stores of every replica: - -```ts group=harness-durable -const claimStores = memoryPersistence().stores - -const sweeper = createHarnessHost({ - persistence: { - stores: { - log: memoryLogStore(), - runs: claimStores.runs, - metadata: claimStores.metadata, - workClaims: claimStores.workClaims, - }, - }, -}) -``` - -Call `resumePending` once when the server starts: - -```ts group=harness-durable -const opened = await sweeper.resumePending({ harnesses: [assistant] }) -console.log(`Resumed ${opened.length} threads`, opened) -``` - -Call it again on a timer, from a cron route or a Durable Object alarm. Then a replica that stops while the others keep running does not leave work behind: +A host recovers a thread when something opens it. After a deploy, nobody opens a thread whose user closed the tab, so its queued turn or background agent waits. Call `host.resumePending` to find that work and run it: ```ts group=harness-durable -export async function GET() { - const resumed = await sweeper.resumePending({ - harnesses: [assistant], - limit: 50, - }) - return Response.json({ resumed }) -} +const resumed = await host.resumePending({ harnesses: [assistant] }) +console.log(`Resumed ${resumed.length} threads`, resumed) ``` -`memoryPersistence()` includes a memory `workClaims` store. It does not survive a restart. To keep claims in your database, implement the [`WorkClaimStore` contract](../persistence/store-reference#workclaimstore). This is server-side only. Clients do nothing for it. - -### When a thread holds a claim - -The session claims its thread for its host when the thread gets work: - -- An input that leaves work: `prompt`, `steer`, `followUp`, `resolve`, `reset`, or an `agent` input. -- A queued turn that the session recovers. -- A background agent run. - -The session renews the claim on the lease timer: every `lease.renewMs`, for `lease.ttlMs`. See [One host drives a thread](#one-host-drives-a-thread) to set them. - -The session gives the claim back when the thread is idle: no queued or running turn, no waiting steer, and no running agent. A thread that waits for a human (an approval or a sign-in) is idle, so a sweep never takes it. - -A question from `ctx.session.ask` is different. The question waits in the memory of its host, so the session keeps the claim until the answer comes. Then an answer that reaches another host goes to this host. See [One host drives a thread](#one-host-drives-a-thread). - -The other host cannot tell if the question is a secret, so it stores the answer in `stores.inbox` as the user typed it. The host that asked removes that entry after it applies the answer. The log keeps a secret answer as `[secret]`. Keep the inbox in a store you trust as much as the log. - -### What a sweep does - `host.resumePending({ harnesses, close, limit })`: -1. Reads the expired claims, oldest first. It reads at most `limit` claims. The default is 100. -2. Skips a thread whose harness is not in `harnesses`. That claim stays listed and counts against `limit`, so pass every harness that the host serves. -3. Claims each thread with compare-and-set. If two hosts sweep at the same time, each thread goes to one of them. -4. Opens the thread. The recovery steps in [What happens after a crash](#what-happens-after-a-crash) run its pending work. +1. Lists the logs that have work and no live owner: an owner that stopped, inputs that wait, or a schedule that is due. It lists at most `limit` logs. The default is 100. +2. Lists only the logs of the harnesses in `harnesses`. Pass every harness that the host serves. +3. Takes each log. If two hosts sweep at the same time, each log goes to one of them. +4. Opens the thread. The recovery steps in [What happens after a crash](#what-happens-after-a-crash) run its pending work, then the inputs that wait run. + +It resolves with the threads that it took, as `{ threadId, harness, tenantId }` entries. Each thread opens in its own tenant, with no user. -It resolves with the threads that it opened, as `{ threadId, harness }` entries. Without `stores.workClaims`, it throws. +If a thread that the sweep took does not open, the host gives it back with a wake in 30 seconds. The next sweep or worker tries it again. The `close` option sets what happens to a session that the sweep opened: - `'whenIdle'` (default): the session closes when its work ends. If the app opened the same thread too, the session stays open. -- `'never'`: the session stays open, like any session that you open. +- `'never'`: the session closes like any session that no handle holds, after `lifecycle.closeIdleSessionsAfterMs`. See [When a session closes](./lifecycle#when-a-session-closes). -### Watch for claim warnings +Call it once at boot, or from a cron route or a Durable Object alarm. To run this work all the time, on as many machines as you want, use a worker. See [Run background work on workers](./workers). -A failed claim or release write does not fail the work. The session publishes a `harness.plugin.warning` custom event with `plugin: 'harness:work-claims'` and the error message. Without that claim, a sweep cannot find the work after a crash. Watch for this event to find these threads. - -If this host already has the thread open, the sweep claims it, but the session does not recover again. Call `host.recover(threadId)` for that thread. See [Check a skipped turn again](#check-a-skipped-turn-again). +If this host already has the thread open, the sweep does not recover it again. Call `host.recover(threadId)` for that thread. See [Check a skipped turn again](#check-a-skipped-turn-again). ## Retry model errors @@ -202,7 +165,7 @@ The steps above are the default. To change them for one input, add `durability.r Return `undefined` to keep the decision. Or return one of these: -- `{ action: 'run', overrides? }`: run the input now. `overrides` are the [turn overrides](./turn-control#give-one-prompt-its-own-settings) of this attempt, for example the adapter that the first attempt used. The log does not keep overrides, so without them a recovered attempt uses the defaults. +- `{ action: 'run', overrides? }`: run the input now. `overrides` are the [turn overrides](./turn-control#give-one-prompt-its-own-settings) of this attempt. The log keeps the JSON overrides of the input (`model`, `modelOptions`, `reasoning`, and `promptCache`). Give `adapter` and `tools` here again, because the log cannot keep objects. - `{ action: 'settle', outcome, error? }`: end the input as `'completed'`, `'failed'`, or `'aborted'`. Without a model call. This example fails an input with your own error code after 2 runs: @@ -311,104 +274,114 @@ const twoNotes = defineHarness({ ## One host drives a thread -Only one host writes to the log of a thread at a time. On a serverless platform, two requests for one thread often reach two instances. Give every instance `workClaims` and `inbox`. Then an input that reaches the second instance goes to the first: +On a serverless platform, two requests for one thread often reach two instances. Only one host writes to the log of a thread at a time. Give every instance the same log store, and the log decides who owns the thread: + +- The first write of a host takes the log. From then on, this host owns the thread. +- One timer renews every thread that the host owns. The host gives a thread back when it has no work. +- An input that reaches another instance waits in the log for the owner. The receipt is `queued`, and the owner runs it within a second. If the owner refuses it, `session.settled({ inputId })` on that instance rejects with `InputRejectedError`. See [Send inputs safely](./inputs). +- Every instance reads the thread, also while the owner runs a background job. A client gets the events from the log, on any instance. A `POST run` on the second instance streams the turn that the owner runs. +- The thread settings, the config values, and the state of an interrupted turn follow the changes that other hosts write. + +Set the timing with the `ownership` option of the host: ```ts group=harness-durable -const replicaStores = memoryPersistence().stores - -const replica = createHarnessHost({ - persistence: { - stores: { - log: memoryLogStore(), - runs: replicaStores.runs, - workClaims: replicaStores.workClaims, - inbox: replicaStores.inbox, - }, - }, +const ownedHost = createHarnessHost({ + persistence: memoryPersistence(), + ownership: { expiresAfterMs: 60_000, renewEveryMs: 20_000 }, }) ``` -The host that runs the thread holds its work claim. An input that reaches another host: +- `expiresAfterMs`: how long the ownership lasts after each renewal. The default is 30 000. After it, another host can take the thread. +- `renewEveryMs`: how often the timer renews. The default is 10 000. When you set only `expiresAfterMs`, the default is a third of `expiresAfterMs`. It must be less than `expiresAfterMs`, else `createHarnessHost` throws a `RangeError`. +- `hostId`: a stable id for this host, for example the machine or pod name. With it, a thread whose coding tools use a folder on this host stays on this host. Other hosts refuse its tool calls. The host sends a heartbeat to the log store each `renewEveryMs`. When no heartbeat came for `expiresAfterMs`, other hosts can take its threads. See [A folder on one server](./coding-workspaces#a-folder-on-one-server). -- With `inbox`: waits in the inbox, and the owner applies it within a second. The receipt is `queued`. -- Without `inbox`: is refused. The receipt is `rejected`, with the reason `owned_elsewhere`. +An input that waits for the owner keeps its JSON overrides (`model`, `modelOptions`, `reasoning`, and `promptCache`). The `adapter` and `tools` overrides are objects, so the owner runs the input without them. -In both cases, the running turn continues. A client reads the events of the thread from the log, on any instance. A `POST run` on the second instance streams the turn that the owner runs. +A [wait](./waits) for a person does not always keep a host busy: -No store keeps the `overrides` and the `ephemeral` messages of a prompt. So the owner runs a forwarded prompt without them. They apply only to a prompt that reaches the owner. +- The approvals and the questions that a turn stopped for, and the waits of background agent runs, are only in the log. The session goes idle, and an answer on any host continues the run. +- A `step.ask` in code that runs, an MCP server that asks during a tool call, and an OAuth sign-in wait in the memory of the host that runs them. That host keeps the thread until the answer comes. -When the owner stops, its claim expires. Then the next host that opens the thread, or a `resumePending` sweep, runs the inputs that wait. +An answer that reaches another host goes to the owner through the log. Until then, a plain answer waits in the log store as the user typed it. The log keeps a secret answer as `[secret]`. -Without `workClaims`, a host cannot see the owner. When a second host writes to the thread, the first host's session stops, and its running turn fails with a clear error. +### Secret answers on another host -The lease decides when a stopped host counts as gone. Change its timing on the host: +A question with `secret: true` asks for a key or a password. The answer must not wait in the log store as the user typed it. So when the answer reaches a host that does not own the thread, it goes through the credential store: -```ts group=harness-durable -const leasedHost = createHarnessHost({ - persistence: { stores: { log: memoryLogStore(), runs } }, - lease: { ttlMs: 60_000, renewMs: 20_000 }, -}) -``` +1. The host saves the answer in the `credentials` store of `createHarnessHost`, as a one-time reference for the user who answered. +2. Only the reference goes into the log store. +3. The owner reads the answer, deletes it from the store, and answers the question. -## Check a skipped turn again +If the host does not find the question in the log, it treats the answer as a secret too. -A host checks the log of a thread when it opens the thread. A turn that another host still runs at that time is skipped. If that other host stops later, the turn waits until something checks again. Call `host.recover()` on a timer: +Pass the same `credentials` store to every host. A secret answer can be rejected: -```ts group=harness-durable -setInterval(() => { - void leasedHost.recover() -}, 15_000) -``` +- `secret_answer_needs_credentials`: the host has no `credentials` store, so it cannot send the answer. +- `unknown_secret_answer`: the reference was used before, it expired, or it belongs to another user. -- `host.recover()` checks every session that this host has open. `host.recover(threadId)` checks one thread. -- `session.recover()` does the same for one session. -- Each call first reads the log records that other hosts wrote. Then it runs the turns whose lease expired. -- The turns that this session already runs or queues stay as they are. Two calls at the same time run a turn once. +The store keeps each reference as a credential of the type `secret_answer`: -## Use your own leases +- Its id starts with `harness:answer:`, and its `expiresAt` is 24 hours later. After that time, the harness does not read it. +- The owner deletes it when it reads it. The references that nobody read go when the question ends. +- `run.credentials.list()` does not show it. Leave the `secret_answer` type out of your own `store.list` results too, for example on a page of connected accounts. -Your job queue or submission table can already have leases. Give the host a `LeaseStore` as `stores.leases`, and it uses your system to find a crashed host. Each call gets the `threadId`, `inputId`, `operationId`, and `attempt` of one turn attempt: +### When a host loses a thread -- `acquire`: the host takes the lease when the attempt starts. It also gets `ownerId` and `expiresAt`. -- `renew`: the host extends the lease while the attempt runs. It also gets `ownerId` and `expiresAt`. -- `release`: the host gives the lease back when the attempt ends. It also gets `ownerId` and `expiresAt`. -- `isAlive`: after a crash, the host asks if another host still runs the attempt. +A host can stop renewing for a while: a long garbage collection pause, or a lost network. Then another host takes the thread. When the first host sees this, it stops its work on the thread: -A host with `leases` does not need `runs`. If you give both, `leases` decides. The `lease` option sets the timing, as in the section above. +1. Its queued turns end `cancelled`, and its running work stops. The new owner runs that work again from the log. +2. Its session stays open and still reads the thread. +3. Its next write takes the thread back, when no other host owns it. -```ts group=harness-durable -import type { LeaseStore, TurnLease, TurnLeaseKey } from '@tanstack/ai-persistence' +The store refuses each late write of the first host, so the log never gets the records of two owners. -const held = new Map() -const keyOf = (k: TurnLeaseKey) => - `${k.threadId}:${k.inputId}:${k.operationId}:${k.attempt}` +## Fence your own side effects -// Swap the Map for your queue or table. -const leases: LeaseStore = { - acquire: async (lease) => { - held.set(keyOf(lease), lease) - }, - renew: async (lease) => { - held.set(keyOf(lease), lease) - }, - release: async (lease) => { - held.delete(keyOf(lease)) - }, - isAlive: async (key) => (held.get(keyOf(key))?.expiresAt ?? 0) > Date.now(), +Your payment system or job queue can refuse a call from a host that lost the thread, or a second call for the same side effect. Each run gives you two values for that in `ctx.context.run`: + +- `epoch`: the ownership epoch of the log when the run started. A later owner has a larger one. Give it to a system that refuses an old epoch. +- `idempotencyKey(step, scope?)`: a key for one side effect, `${logId}:${scope}:${step}`. `scope` defaults to the run id. A run again after a host stop, on any host, gets the same key. + +```ts group=harness-durable +import type { HarnessRunContext } from '@tanstack/ai-harness' + +declare const payments: { + charge: (charge: { amount: number; idempotencyKey: string }) => Promise<{ id: string }> } -const leasedByYou = createHarnessHost({ - persistence: { stores: { log: memoryLogStore(), leases } }, +const charge = toolDefinition({ + name: 'charge', + description: 'Charge the card', +}).server(async (_args, ctx) => { + const key = ctx.context.run.idempotencyKey('charge', ctx.toolCallId) + return payments.charge({ amount: 100, idempotencyKey: key }) }) ``` -For the full contract, see [`LeaseStore`](../persistence/store-reference#leasestore). +Pass the tool call id as `scope` in a tool, so two calls of one tool get two keys. A step of a [durable tool](./durable-tools) gets its own key as `idempotencyKey`. + +## Check a skipped turn again + +A host checks the log of a thread when it opens the thread. A turn that another host still runs at that time is skipped. If that other host stops later, the turn waits until something checks again. Call `host.recover()` on a timer: + +```ts group=harness-durable +setInterval(() => { + void ownedHost.recover() +}, 15_000) +``` + +- `host.recover()` checks every session that this host has open. `host.recover(threadId)` checks one thread. +- `session.recoverPendingWork()` does the same for one session. +- Each call first reads the log records that other hosts wrote. Then it runs the turns whose owner stopped. +- The turns that this session already runs or queues stay as they are. Two calls at the same time run a turn once. + +A [worker](./workers) does this for the threads that no host has open. ## What you have now - Turns that continue after a crash or a deploy, also on threads that nobody opens again. - Inputs that reach any instance and run on the instance that owns the thread. +- A host that stops its work when it loses a thread, and keys that let your own systems refuse a second side effect. - A turn that stops after `maxAttempts` or `timeoutMs`, with a clear error code. - Tools that run again safely or tell the model to check first, in your own words. - Retries of short model errors, with a count that survives a crash. -- Your own recovery rule, your own leases, and a timer that checks skipped turns again. diff --git a/docs/harness/durable-tools.md b/docs/harness/durable-tools.md index eb402b8917..56bf6b1092 100644 --- a/docs/harness/durable-tools.md +++ b/docs/harness/durable-tools.md @@ -46,14 +46,19 @@ export const checkout = durableTool( ) ``` -Add it to `tools` in `defineHarness`, as any other tool. On a [durable host](./durable-sessions), each finished step is in the session log before `step.do` resolves. +Add it to `tools` in `defineHarness`, as any other tool. Each finished step is in the [session log](./durable-sessions) before `step.do` resolves. + +A step value must not hold a secret of the session, for example a key that a tool read. The log must not keep it, so `step.do` throws: on the first run, and on a run again after a crash. Read the secret outside `step.do`, and return an id, not a token. See [Keep tokens out of the log and the model](./auth#keep-tokens-out-of-the-log-and-the-model). ## Rules for steps - Give each step a name that is the same on every run, for example `charge:${orderId}`. - The same name twice in one tool call throws. -- A step value must be JSON. Keep it small: store big data somewhere else and return its id. -- A step is stored once, but it can run more than once: a crash after `fn` ends and before the value is stored runs `fn` again. Make that case safe, for example with an idempotency key. +- A step value must be JSON. A `Date`, a `Map`, or a class instance is a type error, so return `date.toISOString()`, not `date`. `step.do` returns the JSON copy, the same value that a run again gets. Keep it small: store big data somewhere else and return its id. +- A step is stored once, but it can run more than once: a crash after `fn` ends and before the value is stored runs `fn` again. `fn` gets `{ idempotencyKey }`. Pass it to the system that does the side effect, so that system refuses the second call. +- The key is `${logId}:${toolCallId}:${name}`. It has no attempt part, so each run of the step gets the same key. +- To wait inside a tool, for minutes or for days, use `step.sleep`. See [Wait inside a tool](./workers#4-wait-inside-a-tool). +- To ask the user inside a tool, use `step.ask({ name, message, schema })`. The log keeps the question, so a user on any server can answer it. After a crash, `step.ask` returns the logged answer. See [Wait for a person](./waits). `durableTool` sets `replay: 'safe'`, so the harness runs the tool again after a crash. @@ -114,13 +119,17 @@ After a crash, the harness looks at each call of the batch: - A call that finished keeps its result. It does not run again. - A `durableTool` call, or another tool with `replay: 'safe'`, runs again. - Any other call gets a tool error: nobody knows if the tool ran, so the model checks before it tries again. +- A call of an agent tool gets a tool error too. An agent tool that waits for a person keeps its place in the log, and it goes on after the answer. See [When an agent tool waits](./subagents#when-an-agent-tool-waits). + +The kept result is the one that the model saw before the crash. The log keeps it after every `onAfterToolCall` hook, without the secrets of the session. If a hook replaced the result, the replacement wins, also on an error. ## Outside a durable session -The same tool works in plain `chat()` and on a host without a log. There, `step.do` runs `fn` each time, and `append` throws, because there is no log to hold the records. +The same tool works in plain `chat()`. There, `step.do` runs `fn` each time, and `step.sleep` waits its full time. `step.ask` and `append` throw, because there is no log to hold the question or the records. ## What you have now -- Side effects that do not repeat after a crash. +- Steps that keep their result after a crash. A step that a crash cuts can run again, so its idempotency key lets your system refuse the second side effect. +- Questions to the user that any server can answer. - Your own records that land together with the tool batch, or not at all. - One tool definition for durable and plain runs. diff --git a/docs/harness/fork-and-reset.md b/docs/harness/fork-and-reset.md index f17ae9dcc6..a2f257c5be 100644 --- a/docs/harness/fork-and-reset.md +++ b/docs/harness/fork-and-reset.md @@ -48,11 +48,15 @@ export async function branch(threadId: string, messageId: string) { The fork copies: - The transcript, up to and including the message `at`. -- The [stored settings](./thread-settings) of the thread, and the values that `setConfig` stored for its plugins. +- The [stored settings](./thread-settings) of the thread, and the values that `setPluginSetting` stored for its plugins. - The media that the copied messages use. The copies get new ids. The fork does not copy plugin state, pending interrupts, queued inputs, running work, or [usage totals](./usage). If `newThreadId` already has messages, `host.fork` throws. It also throws for an `at` that the thread does not have. +A fork stays in its harness. If the index entry of the thread names another harness, `host.fork` throws. To move a conversation to another harness, use [`host.transfer`](./handoffs#5-move-a-conversation-to-another-harness). + +On a host with [`access`](./access), pass `user` too. The user needs the `fork` action on the old thread, and owns the new thread. + If a thread waits for an approval, fork it at a message before the tool call. A fork of the whole transcript copies the tool call, but the fork has no interrupt to answer it. ## Fork before or through a message @@ -74,19 +78,21 @@ export async function editAndResend(threadId: string, messageId: string, text: s - `{ before: id }` copies the messages before `id`. Before the first message, the fork has no messages, but it keeps the settings. - `{ through: id }` copies the messages up to and including `id`, as `at` does. -- The host picks the new thread id. The new index entry has the title of the old one plus ` (fork)`, and the same owner. +- The host picks the new thread id. The new index entry has the title of the old one plus ` (fork)`. +- `members` picks who can use the fork. `'self'`: only the user who forks, as the owner. `'all'`: the user who forks is the owner, and the other members keep their roles. See [Fork a shared thread](./shared-threads#6-fork-a-shared-thread). +- Without host `access` and without `members`, the fork keeps the owner and the members of the old thread. - `host.sessions.fork` calls `host.fork`, so it copies the same things as `host.fork`. ## Reset with a handoff note -`session.reset(note)` starts a fresh model context. From the next turn, the model sees only the note and what comes after the reset: +`session.reset({ note })` starts a fresh model context. From the next turn, the model sees only the note and what comes after the reset: ```ts group=harness-fork-and-reset export async function startFresh(threadId: string) { const session = await host.open(assistant, { threadId }) - return session.reset( - 'Summary: the user moves the billing service to the new region. The DNS change is done. The database move is next.', - ) + return session.reset({ + note: 'Summary: the user moves the billing service to the new region. The DNS change is done. The database move is next.', + }) } ``` diff --git a/docs/harness/goal.md b/docs/harness/goal.md index 045d37c0c6..2f85d346c1 100644 --- a/docs/harness/goal.md +++ b/docs/harness/goal.md @@ -38,7 +38,7 @@ const session = await host.open(coder, { threadId: 'fix-tests' }) ## 2. Give it a goal ```ts group=harness-goal -await session.command('goal', 'all tests pass') +await session.runCommand({ name: 'goal', input: 'all tests pass' }) ``` The first turn starts. Each new turn tells the model the reason of the last check: @@ -47,7 +47,9 @@ The first turn starts. Each new turn tells the model the reason of the last chec Keep working on the goal: all tests pass. Last check: The date parser test still fails. ``` -Run `session.command('goal')` to see the result: +Each new turn runs as the user who set the goal. In a [shared thread](./shared-threads), the goal uses that user's credentials and keys, not the ones of the last person who wrote. + +Run `session.runCommand({ name: 'goal' })` to see the result: ```text Goal: all tests pass diff --git a/docs/harness/handoffs.md b/docs/harness/handoffs.md new file mode 100644 index 0000000000..0a4bd2b7d0 --- /dev/null +++ b/docs/harness/handoffs.md @@ -0,0 +1,330 @@ +--- +title: Hand off a conversation +id: harness-handoffs +order: 14 +description: "Let one agent hand the conversation to another agent, to a person, or to another harness. Only the data that you name moves with it." +keywords: + - tanstack ai + - harness + - handoff + - transfer + - operator + - human in the loop + - activeAgent +--- + +A user asks your support agent for a refund. The support agent cannot do refunds, but your billing agent can. Then the user wants a person, and a person from your team must take over. Later, the whole case must move to the harness of another team. + +A handoff moves the conversation to another agent or to a person. A transfer moves it to another thread, also of another harness. In both cases, control moves on purpose, and only the data that you name moves with it. + +## 1. Let agents hand off + +Give the harness its root agents and `routing`. A router that returns `undefined` lets the agent that answers now keep the turn: + +```ts group=harness-handoffs +import { defineAgent } from '@tanstack/ai' +import { createHarnessHost, defineHarness } from '@tanstack/ai-harness' +import { memoryPersistence } from '@tanstack/ai-persistence' +import { openaiText } from '@tanstack/ai-openai' + +const billing = defineAgent({ + name: 'billing', + description: 'Handles refunds, invoices, and payments', + run: (ctx) => + ctx.chat({ + adapter: openaiText('gpt-6.1-sol'), + systemPrompts: ['You handle refunds, invoices, and payments.'], + }), +}) + +export const support = defineHarness({ + name: 'acme/support', + adapter: openaiText('gpt-6.1-sol'), + systemPrompts: ['You are the support desk. Hand billing questions to billing.'], + agents: [billing], + routing: { router: () => undefined }, +}) + +export const host = createHarnessHost({ persistence: memoryPersistence() }) + +export async function ask(threadId: string, text: string) { + const session = await host.open(support, { threadId }) + return session.send(text) +} +``` + +Send `I want a refund for order 42.` The main model calls its `handoff` tool with `{ to: 'billing', note }`. Then billing answers in the same turn. The turn text is `billing:\n...`. + +What the handoff does: + +- The log keeps the active agent. Later turns go to billing until it hands off again, or until the router picks another agent. +- The main model and each root agent get a `handoff` tool. Its `to` lists the root agents. A root agent can also hand back to `'main'`, the main model. +- One turn can hand off 10 times at most. +- An agent that is only in `subagents` cannot be a target. A child that an agent starts cannot hand off either. +- If the app has its own tool named `handoff`, the turn fails with a duplicate tool error. Set `routing: { router, giveModelsHandoffTool: false }` to give no model the `handoff` tool. The router and your code can still hand off. + +## 2. Hand off from code + +An agent can decide in code. In a root agent that answers a turn, call `ctx.harness.handoff`: + +```ts group=harness-handoffs +export const triage = defineAgent({ + name: 'triage', + description: 'Sorts new requests', + run: async (ctx) => { + if (!ctx.harness) throw new Error('This agent must run in a harness.') + await ctx.harness.handoff({ to: 'billing', note: 'Refund for order 42' }) + return 'I pass you to billing.' + }, +}) +``` + +A tool of the main model uses `ctx.context.run.handoff`. Type the tool with `HarnessRunContext`: + +```ts group=harness-handoffs +import { toolDefinition } from '@tanstack/ai' +import { z } from 'zod' +import type { HarnessRunContext } from '@tanstack/ai-harness' + +export const escalate = toolDefinition({ + name: 'escalate', + description: 'Send the user to billing', + inputSchema: z.object({ orderId: z.string() }), +}).server(async ({ orderId }, ctx) => { + await ctx.context.run.handoff({ to: 'billing', note: `Refund for order ${orderId}` }) + return 'Billing answers next.' +}) +``` + +- The part that handed off ends. Then the target answers, in the same turn. +- Outside a turn of a harness with `routing`, both calls reject. +- A target that is not a root agent rejects, and the agent that called it keeps the turn. + +The router can hand off too. Return `{ handoff: 'billing', note }` from `routing.router`. See [Route a turn to an agent](./subagents#route-a-turn-to-an-agent). + +## What the target gets + +- Its own system prompts and tools. The tools of the agent that handed off do not go with it. +- The saved transcript of the thread, the same messages that the members see. +- The note, in the transcript, in the log, and in the `harness.handoff` event. +- The provider keys of the user who sent the turn. + +The `handoff` tool call of a model is in the transcript already. A handoff from the router or from code adds the same two messages before the target answers: + +- An assistant message with a `handoff` tool call. Its arguments are `{ to, note }`. +- The tool result, for example `Handed off to billing. billing answers next.` + +Both messages have `metadata.harness.handoff.by` (`'router'` or `'code'`). When an agent handed off, they also have its name in `metadata.harness.agent`. The messages go after the messages that the earlier model call got, so the prompt cache of the provider still holds. + +With `routing.giveModelsHandoffTool: false`, the models have no `handoff` tool. Then the harness adds one assistant message with the same text and the note. + +The `agents()` plugin has its own primary agent, its `agent` config and `/agent`. That one changes the prompt and the tools of the main model. A handoff does not change it, and it does not change the active agent. See [Switch between agent profiles](./agents). + +## 3. Show who answers + +Each message that an agent writes has its name in `metadata.harness.agent`. In the [session view](./custom-ui), an assistant message has `agent`. It is `undefined` for the main model: + +```tsx ignore +import { useSelector } from '@tanstack/react-store' +import { createHarnessClient } from '@tanstack/ai-harness/client' +import { createSessionView } from '@tanstack/ai-harness/view' + +const view = createSessionView( + createHarnessClient({ url: '/api/harness', threadId: 'user-1-thread' }), +) + +export function Messages() { + const messages = useSelector(view.store, (state) => state.messages) + return messages.map((message) => + message.role === 'assistant' ? ( +

+ {message.agent ?? 'Support'}: + {message.parts.map((part) => (part.type === 'text' ? part.text : '')).join('')} +

+ ) : null, + ) +} +``` + +Each handoff also streams a `CUSTOM` event named `harness.handoff` (`HARNESS_EVENTS.handoff`). Its value has these fields: + +- `from` and `to`: an agent name, `null` for the main model, or `{ human: true }` for a person. +- `note`: the note, when there is one. +- `by`: `'tool'`, `'code'`, `'router'`, or `'operator'`. +- `runId`: the turn. A handoff to a person also has `waitId`. + +## 4. Hand off to a person + +Some users need a person. Set `routing.allowHandoffToHuman: true`. Then the `handoff` tool also takes `{ human: true }`: + +```ts group=harness-handoffs +import { membersOnly } from '@tanstack/ai-harness' +import { createHarnessHandler } from '@tanstack/ai-harness/server' + +export const desk = defineHarness({ + name: 'acme/desk', + adapter: openaiText('gpt-6.1-sol'), + systemPrompts: ['If the user asks for a person, hand off to { human: true }.'], + agents: [billing], + routing: { router: () => undefined, allowHandoffToHuman: true }, +}) + +// Your own login check: a session token to a user. +const users = new Map([ + ['Bearer token-ada', { id: 'ada', tenantId: 'acme' }], + ['Bearer token-olga', { id: 'olga', tenantId: 'acme' }], +]) + +export const deskHost = createHarnessHost({ + persistence: memoryPersistence(), + access: { + getUser: ({ request }) => users.get(request.headers.get('authorization') ?? '') ?? null, + rules: membersOnly(), + }, +}) + +export const handler = createHarnessHandler({ host: deskHost, harness: desk }) +``` + +When an agent hands off to a person: + +- The handoff opens a `human` wait. The note is its `message`. The wait does not expire. +- The turn ends `interrupted`, and the session pauses. +- A user message during the pause goes into the transcript. No model answers it. + +`ctx.harness.handoff({ to: { human: true } })` and `run.handoff` work the same way. Without `routing.allowHandoffToHuman`, a handoff to a person rejects. + +### Reply as the operator + +A person from your team replies with `as: 'operator'`: + +1. Share the thread with the operator, with the role `operator`. Ada owns the thread here. See [Share a thread between people](./shared-threads#2-share-the-thread). +2. Send the reply as the operator. + +```ts group=harness-handoffs +const ada = { id: 'ada', tenantId: 'acme' } +const olga = { id: 'olga', tenantId: 'acme' } + +export async function addOperator(threadId: string) { + await deskHost.sessions.share({ threadId, userId: 'olga', role: 'operator', user: ada }) +} + +export async function reply(threadId: string, text: string) { + const session = await deskHost.open(desk, { threadId, user: olga }) + await session.send(text, { as: 'operator' }) +} +``` + +- The reply goes into the transcript as an assistant message, with `metadata.harness.operator` set to the id of the operator. No model runs. +- The sender needs the `operator` access action. With `membersOnly()`, only the role `operator` has it. The owner and members do not, so the user who waits cannot act as the person. To change this, pass `membersOnly({ operator: { roles: ['owner', 'operator'] } })`. +- Files in the reply go into a user message after it, because a model takes files only from a user. `metadata.harness.operatorAttachment` marks it, and a view shows the files with the reply. +- Without an open `human` wait, the receipt is `rejected` with `no_human_wait`. + +In the browser, the operator uses the same option: `client.send(text, { as: 'operator' })`, or `view.send(text, { as: 'operator' })` in a [session view](./custom-ui). The reply shows in each open view at once. + +### End the pause + +The operator ends the pause with an answer to the `human` wait: + +```ts group=harness-handoffs-client +import { createHarnessClient } from '@tanstack/ai-harness/client' +import type { desk } from './harness' + +const client = createHarnessClient({ + url: '/api/desk', + threadId: 'case-42', + headers: { authorization: 'Bearer token-olga' }, +}) + +const snapshot = await client.snapshot() +const wait = snapshot.pendingWaits.find((open) => open.kind === 'human') +if (wait) { + await client.answer({ questionId: wait.waitId, value: { handBack: 'billing' } }) +} +``` + +- `{ resume: true }` gives the thread back to the agent that answered before the handoff. +- `{ handBack: 'billing' }` gives it to that agent. `{ handBack: 'main' }` gives it to the main model. +- Any other value is `rejected` with `invalid_answer`. An unknown agent is `rejected` with `unknown_agent`. +- The wait can also end with `session.resolve` or a cancel. Then the agent that answered before the handoff gets the thread back. Each of these needs the `operator` action. +- The answer starts no turn. The held user messages wait for the next message, or for `session.continueWithoutMessage()`. + +## 5. Move a conversation to another harness + +The case must leave the support harness. The billing team runs its own harness, with its own tools and plugins. `host.transfer` makes a new thread under the target harness and copies only the envelope: + +```ts group=harness-handoffs +export const billingDesk = defineHarness({ + name: 'acme/billing', + adapter: openaiText('gpt-6.1-sol'), +}) + +export const billingHandler = createHarnessHandler({ host: deskHost, harness: billingDesk }) + +// Mount on POST /api/move-to-billing?threadId=... +export async function moveToBilling(request: Request) { + const user = await deskHost.getUserOfRequest({ request }) + if (!user) return new Response('Unauthorized', { status: 401 }) + const threadId = new URL(request.url).searchParams.get('threadId') + if (!threadId) return new Response('Missing threadId', { status: 400 }) + const result = await deskHost.transfer({ + from: { harness: desk, threadId }, + to: { harness: billingDesk }, + envelope: { note: 'Refund for order 42. The user waits.', messages: 'summary' }, + user, + transferId: `refund-${threadId}`, + }) + return new Response(result.threadId) +} +``` + +`envelope.note` is the first message of the target. `envelope.messages` picks what comes after it: + +- `'none'` (default): no message. +- `'summary'`: one summary message. The model of the source harness writes it, with the keys of the user. The summary call stops after 2 minutes, and then the transfer fails. Its usage counts in the source thread under the source `transfer`. See [Usage of the router and transfers](./usage#usage-of-the-router-and-transfers). +- `'all'`: the settled transcript, with new message ids and a copy of each media file. + +Only settled messages move. A tool call with no result yet stays behind. Plugin state, shared state, credentials, waits, settings, and plugin config never move. The source thread keeps running. + +The browser then opens the target thread on the handler of the target harness: + +```ts group=harness-handoffs-client +import type { billingDesk } from './harness' + +const response = await fetch('/api/move-to-billing?threadId=case-42', { + method: 'POST', + headers: { authorization: 'Bearer token-ada' }, +}) +const threadId = await response.text() + +const billingClient = createHarnessClient({ + url: '/api/billing', + threadId, + headers: { authorization: 'Bearer token-ada' }, +}) +await billingClient.send('Hi, I am still here.') +``` + +### Rules of a transfer + +- Both threads are in the tenant of `user`. A transfer to another tenant cannot be named. +- With access, the user needs `transfer` on the source. With `membersOnly()`, only the owner has it. A target that exists needs `send` too. +- A new target has the user as its only member, the owner. A thread that exists must belong to `to.harness`. +- A retry with the same `transferId` makes no second thread and copies nothing again. +- If another host owns the target log, the transfer throws `LogOwnedElsewhereError`. + +How the two threads stay linked: + +- Both logs get a `harness.transferred` record with the same `transferId`. See [Read the log](./session-log#read-the-log). +- The index entry of the target has `metadata['harness:transferredFrom']`. The source has `metadata['harness:transferredTo']`. The host writes these keys itself, and your `metadataSchema` does not check them. +- `host.events()` yields a `transferred` event. Copy your own access rows for the target there. + +A [fork](./fork-and-reset) stays in its harness. `host.fork` with a thread of another harness throws, so use `host.transfer` for that move. + +## What you have now + +- Agents that hand the conversation to each other, from a tool, from code, or from the router. +- A log that keeps who answers, also after a restart. +- A UI that shows the name of the agent on each message. +- A pause for a person, operator replies, and a way to give the thread back. +- A conversation that moves to another harness with only the envelope that you name. diff --git a/docs/harness/host-options.md b/docs/harness/host-options.md new file mode 100644 index 0000000000..1810b60403 --- /dev/null +++ b/docs/harness/host-options.md @@ -0,0 +1,94 @@ +--- +title: Host options +id: harness-host-options +order: 1 +description: "Every option of createHarnessHost in one place: what it does, its default, and when you need it. A local app needs none of them." +keywords: + - tanstack ai + - harness + - createHarnessHost + - host options + - configuration +--- + +`createHarnessHost` has many options, and their names do not always tell you if you need them. Most apps need only one or two. A local app, like a coding agent on your own computer, needs none. + +The host is the part of your server that runs the sessions. A session is one conversation with your agent. The host keeps the state of each session, runs its turns, and gives clients its events. + +## Start with no options + +On your own computer, with one user, create the host with no options: + +```ts group=harness-host-options-local +import { createHarnessHost, defineHarness } from '@tanstack/ai-harness' +import { openaiText } from '@tanstack/ai-openai' + +const assistant = defineHarness({ + name: 'acme/assistant', + adapter: openaiText('gpt-6.1-sol'), +}) + +const host = createHarnessHost() + +await using session = await host.open(assistant, { threadId: 'my-first-chat' }) +const turn = await session.send('Hello!') +console.log(turn.text) +``` + +This host keeps everything in memory. When the process stops, the sessions are gone, and the host prints one warning about this at the start. There is no login check, so every call acts for the one local user. + +## Find the options that you need + +Find your situation below. Add only the options of that situation. + +- **The sessions must stay after a restart.** Add `persistence`. See [Durable sessions](./durable-sessions). +- **More than one person uses the app.** Add `access`. Each call then needs the `user` that it acts for. See [Access and tenants](./access). +- **You run more than one server.** Give every server the same `persistence`. If your agents use files on the disk of a server, add `ownership`. See [Deploy a harness](./deploy#run-more-than-one-server). +- **Users bring their own API keys.** Add `credentials`, so the keys stay after a restart. See [Where the keys are stored](./provider-keys#where-the-keys-are-stored). +- **You want traces and metrics.** Add `observability`. See [Trace runs and report operations](./deploy#trace-runs-and-report-operations). + +A server for many users usually starts like this: + +```ts group=harness-host-options-server +import { createHarnessHost, membersOnly } from '@tanstack/ai-harness' +import { memoryPersistence } from '@tanstack/ai-persistence' + +const host = createHarnessHost({ + // Use a store on your own database in production. + persistence: memoryPersistence(), + access: { + // Your own login check. Return null when nobody is logged in. + getUser: async ({ request }) => { + const id = request.headers.get('x-user-id') + return id ? { id } : null + }, + rules: membersOnly(), + }, +}) +``` + +## All options + +Each option is optional. Leave out an option to get its default. + +| Option | What it does | Default | Read more | +| --- | --- | --- | --- | +| `persistence` | Where the host saves the state of each session. | In memory. Lost when the process stops. | [Durable sessions](./durable-sessions) | +| `access` | Who can use which session. `getUser` finds the user of a request. `rules` decide what that user can do. | No checks. One user. | [Access and tenants](./access) | +| `credentials` | Where the API keys and sign-in tokens of your users are saved. | In memory. | [Connect model providers](./provider-keys#where-the-keys-are-stored) | +| `log` | How often the host writes streamed text to the log, and when it saves a snapshot. | Text every 100 ms. A snapshot after 1 MB or 5 minutes. | [Session log](./session-log#tune-how-text-streams-into-the-log) | +| `ownership` | How servers share the sessions. One server runs a session at a time. | Ownership lasts 30 s and renews every 10 s. Sessions are not pinned to a server. | [Durable sessions](./durable-sessions#one-host-drives-a-thread) | +| `lifecycle` | When the host closes idle sessions, how many stay open, and how much each keeps in memory. | Close after 60 s idle. At most 500 open. | [Lifecycle and retention](./lifecycle) | +| `observability` | Sends traces to OpenTelemetry. Also calls you when each turn, agent run, and command starts and ends. | No traces. | [Deploy a harness](./deploy#trace-runs-and-report-operations) | +| `presence` | Calls you when a client connects to a session, and when it disconnects. | No calls. | [Share a thread](./shared-threads#8-show-who-is-here) | +| `harnesses` | The harnesses whose sessions this host can delete without opening them first. | Only the harnesses that this host opened. | [Delete a thread for good](./lifecycle#delete-a-thread-for-good) | + +Some words in this table: + +- **Log:** the record of everything that happened in a session. The host reads it again after a restart. +- **Snapshot:** a saved copy of the state of a session, so the host does not read the full log to open it. +- **Tenant:** an organization in your app. With `access`, the sessions of one tenant stay apart from the sessions of other tenants. + +## What you have now + +You know which options your app needs. A local app runs with `createHarnessHost()`. A server adds `persistence` and `access` first, and more options only when it needs them. diff --git a/docs/harness/inputs.md b/docs/harness/inputs.md index 386a240f66..fe8d78fa50 100644 --- a/docs/harness/inputs.md +++ b/docs/harness/inputs.md @@ -11,8 +11,8 @@ keywords: - settled - steer - cancelInput - - setDelivery - - background + - changeInputDelivery + - moveToolCallToBackground --- A network retry can send the same prompt twice, and a client that reconnects does not know how its last prompt ended. Give each input your own id. A retry with the same id runs nothing again, and the session tells you how the input ended. @@ -23,18 +23,14 @@ On the server, pass `inputId` to `send`. `await turn.receipt` resolves when the ```ts group=harness-inputs import { createHarnessHost, defineHarness } from '@tanstack/ai-harness' -import { memoryLogStore, memoryPersistence } from '@tanstack/ai-persistence' +import { memoryPersistence } from '@tanstack/ai-persistence' import { openaiText } from '@tanstack/ai-openai' const assistant = defineHarness({ name: 'acme/assistant', adapter: openaiText('gpt-6.1-sol'), }) -const host = createHarnessHost({ - persistence: { - stores: { log: memoryLogStore(), runs: memoryPersistence().stores.runs }, - }, -}) +const host = createHarnessHost({ persistence: memoryPersistence() }) const session = await host.open(assistant, { threadId: 'user-1-thread' }) const turn = session.send('Summarize the report.', { inputId: 'req-42' }) @@ -44,6 +40,7 @@ console.log(receipt.status) // 'accepted' or 'queued' - The same id and the same message: you get the first input's operation, and the model is not called again. - The same id and another message: the receipt has `status: 'rejected'` and `reason: 'conflict'`, and `await turn` rejects with `InputRejectedError`. +- An input to a thread that was deleted: the receipt has `status: 'rejected'` and `reason: 'deleted'`, and `await turn` rejects with `InputRejectedError`. Nothing runs. - `send` with `mode: 'steer'` or `mode: 'followUp'`, and `resolve`, take `{ inputId }` too. - [`configure`](./thread-settings) and [`reset`](./fork-and-reset#reset-with-a-handoff-note) are inputs too. They take `{ inputId }` and run once per id. @@ -67,21 +64,30 @@ console.log(receipt.operationId) Make the id once per user action (a button press), and keep it for every retry of that action. +This works for every input over HTTP and the socket. A `command` or an `agent` input that the client sends again with the same `inputId` also runs once. The `inputId` of its receipt is the id that the client sent. + ## Wait for the outcome -On the server, `session.settled(inputId)` waits until the input ends. On a durable host it reads the log, so it also works after a restart and on another host that opens the thread. +On the server, `session.settled({ inputId })` waits until the input ends. It reads the log, so on a [durable host](./durable-sessions) it also works after a restart and on another host that opens the thread. ```ts group=harness-inputs -const settlement = await session.settled('req-42') +const settlement = await session.settled({ + inputId: 'req-42', + signal: AbortSignal.timeout(60_000), +}) if (settlement.outcome === 'failed') { console.log(settlement.error?.code) // 'attempts_exhausted', 'timeout', or undefined } ``` +`signal` is optional. When it aborts first, `settled` rejects with its reason. The input keeps running. + +On a host that does not own the thread, the input waits for the owner. If the owner refuses it, `settled` rejects with `InputRejectedError`. The owner refuses an input when the thread is busy and the input has `busy: 'reject'`, or when the log has its id with another message. The log then has a `harness.offer.rejected` record. + - `completed`: the turn has its answer. - `failed`: the turn failed. `error.code` is set for a limit. - `aborted`: someone cancelled the turn. -- `interrupted`: the turn waits for human input. The `resolve` that answers it is a new input. +- `interrupted`: the turn waits for a person. The `resolve` that answers it is a new input. See [Wait for a person](./waits). If the turn fails after an approved tool ran (for example, the next model call fails), the interrupt is used up, and the tool does not run again. Its result stays in the transcript. To go on, send a new message. If a resume stopped before its tool ran, the interrupt stays open, and you can resolve it again. @@ -98,30 +104,30 @@ for await (const entry of client.events()) { ## Continue without a new message -Sometimes the transcript already holds what the model must answer, for example a tool result that your own system stored. Start a turn with `continue`, and the model answers the stored transcript. No new user message goes in: +Sometimes the transcript already holds what the model must answer, for example a tool result that your own system stored. Start a turn with `continueWithoutMessage`, and the model answers the stored transcript. No new user message goes in: ```ts group=harness-inputs -const continued = session.continue({ inputId: 'req-43' }) +const continued = session.continueWithoutMessage({ inputId: 'req-43' }) await continued ``` From the client: ```ts group=harness-inputs-client -const continuedReceipt = await client.continue({ inputId: 'req-43' }) +const continuedReceipt = await client.continueWithoutMessage({ inputId: 'req-43' }) ``` - `inputId` works as for `send`: a retry with the same id runs once. -- `continue` takes `overrides`, `principal`, and `context` too, as `send` does. +- `continueWithoutMessage` takes `overrides`, `user`, and `context` too, as `send` does. On a host with [`access`](./access), a session acts only as the user that opened it, so leave out `user`. - When the turn starts, the last message of the transcript must be a user message or a tool result. Else the input is rejected with the reason `nothing_to_continue`, and `await continued` rejects with `InputRejectedError`. -- A `continue` that a host left after a crash runs on the next host, as a prompt does. +- A `continueWithoutMessage` that a host left after a crash runs on the next host, as a prompt does. ## Give the model a note for one turn -A reminder such as "call `finish` or `give_up`" helps the model in one turn, but it does not belong in the transcript. Pass it as `ephemeral` to `send` or `continue`: +A reminder such as "call `finish` or `give_up`" helps the model in one turn, but it does not belong in the transcript. Pass it as `ephemeral` to `send` or `continueWithoutMessage`: ```ts group=harness-inputs -const reminded = session.continue({ +const reminded = session.continueWithoutMessage({ inputId: 'req-44', ephemeral: [{ role: 'user', content: 'Call finish or give_up now.' }], }) @@ -137,7 +143,7 @@ const remindedReceipt = await client.send('Summarize the report.', { }) ``` -- Each model call of the turn gets the messages right after the turn's own message (or the last stored message, for `continue`), before the answers and tool results of the turn. +- Each model call of the turn gets the messages right after the turn's own message (or the last stored message, for `continueWithoutMessage`), before the answers and tool results of the turn. - The transcript, the session log, and the message store never keep them. - A retry with the same `inputId` gets the first input, also when its `ephemeral` messages are different. - A turn that recovery runs again after a crash has no `ephemeral` messages, because the log does not keep them. @@ -177,7 +183,7 @@ const joined = session.send('Keep it under five steps.', { }) // One turn answers both, so both get its result. await Promise.all([running, joined]) -console.log((await session.settled('req-44')).operationId === running.id) // true +console.log((await session.settled({ inputId: 'req-44' })).operationId === running.id) // true ``` A message that arrives while the model writes its final answer still gets an answer in the same turn. @@ -185,7 +191,7 @@ A message that arrives while the model writes its final answer still gets an ans Waiting `steer` inputs join before each model call, in the order they arrived. They join as a prefix: the join stops at the first input that has an abort request, or that `turn.canJoin` refuses. The inputs after it wait. - The `operationId` in the receipt of a `busy: 'steer'` or `mode: 'steer'` message is the running turn. A cancel with that id cancels the turn. -- A `busy: 'steer'` message also has its own operation, `joined`. To stop it before it joins, call `joined.cancel()` or `session.cancel(joined.id)`. It does not join, and it settles `aborted` when the running turn ends. +- A `busy: 'steer'` message also has its own operation, `joined`. To stop it before it joins, call `joined.cancel()` or `session.cancel({ operationId: joined.id })`. It does not join, and it settles `aborted` when the running turn ends. - The session rejects a cancel that comes after the join started, with `reason: 'not_running'`. - A refused or late input runs as its own turn after the running one. @@ -193,7 +199,7 @@ To choose which inputs join, or to add records when they join, see [Choose which ## Cancel or move a waiting message -A user sends two more messages while a turn runs. Before they run, the user wants to cancel one, and wants the other to reach the running turn now. `session.inputs()` lists the inputs that wait. `cancelInput` cancels one, and `setDelivery` moves one: +A user sends two more messages while a turn runs. Before they run, the user wants to cancel one, and wants the other to reach the running turn now. `session.waitingInputs()` lists the inputs that wait. `cancelInput` cancels one, and `changeInputDelivery` moves one: ```ts group=harness-inputs const migration = session.send('Plan the migration.', { inputId: 'req-45' }) @@ -202,20 +208,20 @@ const tests = session.send('Then write the tests.', { inputId: 'req-46', mode: ' const style = session.send('Use tabs, not spaces.', { inputId: 'req-47', mode: 'followUp' }) await Promise.all([tests.receipt, style.receipt]) -for (const input of session.inputs()) { +for (const input of session.waitingInputs()) { console.log(input.inputId, input.delivery, input.message) } -await session.cancelInput('req-46') -await session.setDelivery({ inputId: 'req-47', delivery: 'steer' }) +await session.cancelInput({ inputId: 'req-46' }) +await session.changeInputDelivery({ inputId: 'req-47', delivery: 'steer' }) await migration ``` -- `inputs()` lists the inputs in the order they run: first the steers for the running turn, then the queued turns. +- `waitingInputs()` lists the inputs in the order they run: first the steers for the running turn, then the queued turns. - `delivery: 'steer'`: the input joins the running turn at the next model call. `'queue'`: it runs as its own turn, after the turns before it. - A cancelled input never runs, also after a restart. It settles `aborted`. -- On a durable host, the log keeps a new delivery, and a restart obeys it. Without a log, the delivery stays in memory. -- If the input started, or joins the running turn now, the receipt has `status: 'rejected'` and `reason: 'not_waiting'`. To stop it, cancel its operation with `session.cancel(operationId)`. +- The log keeps a new delivery, and a restart obeys it. +- If the input started, or joins the running turn now, the receipt has `status: 'rejected'` and `reason: 'not_waiting'`. To stop it, cancel its operation with `session.cancel({ operationId })`. A client reads the waiting inputs from the snapshot, and sends the same two inputs: @@ -225,31 +231,32 @@ for (const input of waitingInputs) { console.log(input.inputId, input.delivery) } -await client.cancelInput('req-46') -await client.setDelivery({ inputId: 'req-47', delivery: 'steer' }) +await client.cancelInput({ inputId: 'req-46' }) +await client.changeInputDelivery({ inputId: 'req-47', delivery: 'steer' }) ``` Over `POST control`, they are `{ op: 'cancelInput', inputId }` and `{ op: 'setDelivery', inputId, delivery }`. In these two inputs, `inputId` is the id of the input that waits. A move sends a `harness.input.delivery` event with `inputId` and `delivery`, so every client can update its list. ## Move a running tool call to the background -A long command or a subagent blocks the turn, and the user wants to go on. `session.background()` moves the running tool call to the background: +A long command or a subagent blocks the turn, and the user wants to go on. `session.moveToolCallToBackground()` moves the running tool call to the background: ```ts group=harness-inputs const build = session.send('Run the full test suite.', { inputId: 'req-48' }) // Later, while the `bash` call of that turn runs: -const moved = await session.background() +const moved = await session.moveToolCallToBackground() console.log(moved.status) // 'accepted', or 'rejected' with 'not_running' await build ``` - The tool call returns at once. The model gets a short note with the job id, and the turn goes on. - The job keeps running. When it ends, the model gets its result as a note, and an idle session starts a new turn. -- Without an argument, it moves every running call that supports it. Pass a `toolCallId` to move one call. +- Without an argument, it moves every running call that supports it. Pass `{ toolCallId }` to move one call. - `bash` and the single `subagent` tool support it. Other tools keep running as before. - With no running call that supports it, the receipt has `status: 'rejected'` and `reason: 'not_running'`. -- On a durable host (one with `stores.log`), the log keeps each job until it ends. When a crash stops a job, the next host adds the note "Background job call-1 stopped when the host restarted." -- Without `stores.log`, the job lives in memory only. A crash stops it, and no note comes. +- The log keeps each job until it ends. When a crash stops a job, the next host adds the note "Background job call-1 stopped when the host restarted." + +The session stays open while a job of this host runs, also when no handle holds it. A sweep with `close: 'whenIdle'` waits for these jobs too. The wake turn of a `bash` job runs as the user whose run started the job. A client sends `{ op: 'background' }` over `POST control`. It runs no new code, so it needs no `expose` entry: @@ -336,8 +343,9 @@ On `POST run`, the AG-UI `forwardedProps` of the request are the `context`, for - The `context` is part of the input for the `inputId` check. An input with the same id and another `context` gets `'conflict'`. - When `HarnessConfig.context` and the input `context` are both plain objects, tools get them merged, and the harness value wins for a key in both. In all other cases, tools get the harness value. If the harness has no value, tools get the input value. +- The harness adds the run of the turn to a plain-object context as `run`, and it replaces a `run` key that a client sent. A context that is not a plain object gets no `run`, so tools that read `ctx.context.run` fail. See [Act as the user of a run](./plugins#act-as-the-user-of-a-run). - Plugins and the router get it as `turn.context`. See [Pick the model of a turn](./plugins#pick-the-model-of-a-turn). -- The `context` is client data, so do not trust it. Who sent the input is the principal from `authorize`. See [Share a thread](./shared-threads). +- The `context` is client data, so do not trust it. Who sent the input comes from the server: the user of the request, or of the session handle. See [Access and tenants](./access). ## What you have now diff --git a/docs/harness/lifecycle.md b/docs/harness/lifecycle.md new file mode 100644 index 0000000000..fe9c3a3b7d --- /dev/null +++ b/docs/harness/lifecycle.md @@ -0,0 +1,217 @@ +--- +title: Lifecycle and retention +id: harness-lifecycle +order: 4 +description: "Release session handles with await using, let idle sessions close, cap the open sessions, read a thread without a session, bound what each session keeps in memory, and delete a thread for good." +keywords: + - tanstack ai + - harness + - lifecycle + - retention + - await using + - idle close + - delete +--- + +Your server opens a session for each request. Each open session holds its plugins, its MCP processes, and its recent events in memory. A server that never lets them go grows until it runs out of memory. The host closes the sessions that nobody uses, caps how many stay open, and bounds what each session keeps. + +## Open and release a session + +Each `host.open` gives you a handle: your own reference to the live session of the thread. When you are done with the handle, release it. With `await using`, the release runs at the end of the block: + +```ts group=harness-lifecycle +import { createHarnessHost, defineHarness } from '@tanstack/ai-harness' +import { memoryPersistence } from '@tanstack/ai-persistence' +import { openaiText } from '@tanstack/ai-openai' + +export const assistant = defineHarness({ + name: 'acme/assistant', + adapter: openaiText('gpt-6.1-sol'), +}) + +const host = createHarnessHost({ persistence: memoryPersistence() }) + +export async function reply(threadId: string, message: string) { + await using session = await host.open(assistant, { threadId }) + const turn = await session.send(message) + return turn.text +} +``` + +- The handles of one thread share one live session. A release lets go of this handle only. +- `await session.close()` also releases the handle. A second call does nothing. +- After the release, each call on the handle throws `This handle is closed.` Call `host.open` again to get a new handle. +- A release does not stop work. A turn that runs goes on until it ends. +- If you drop a handle with no release, the garbage collector releases it, and the host logs one warning. +- A handle that you did not release holds its session. The session can still close under it, for example when its log fails or a [delete](#delete-a-thread-for-good) runs. Then the next call on the handle opens the session again. On a host with [`access`](./access), `isAllowed` checks that call again. + +## When a session closes + +The host closes a session when no handle holds it and it was idle for `lifecycle.closeIdleSessionsAfterMs`. Set the limits on the host: + +```ts group=harness-lifecycle +const busyHost = createHarnessHost({ + persistence: memoryPersistence(), + lifecycle: { closeIdleSessionsAfterMs: 30_000, maxOpenSessions: 200 }, +}) +``` + +- `closeIdleSessionsAfterMs`: how long an idle session with no handle stays open, in milliseconds. The default is 60 000. `0` closes it at once. A session is not idle while a background job of this host runs, for example a `bash` job. +- `maxOpenSessions`: how many sessions the host keeps open. The default is 500. An open past the limit first closes the least recently used sessions that are idle and that no handle holds. +- If every open session is busy or held, the open goes on, and the host logs one warning. +- A close cleans up the plugins of the session. The transcript and the other data stay in the stores, so the next open continues the thread. A workspace that `bind` gave the thread closes with `'moved'`, so an owned sandbox stays. +- The idle timer does not keep the Node process alive. + +A plugin with `lifetime: 'host'` stays set up while another session uses it. See [Share one setup between sessions](./plugins#share-one-setup-between-sessions). + +## Stop the host + +`host.close()` closes every session of the host, for every handle. Then `host.open` throws: + +```ts group=harness-lifecycle +await busyHost.close() +``` + +- Without options, the running work is cancelled. +- With `recoverable: true`, the running work stops, and another host runs it again. Use it when a host stops for a deploy. See [Stop a host for a deploy](./durable-sessions#stop-a-host-for-a-deploy). +- After the close, a call on a handle throws. `snapshot()` still gives the last state. +- The close also ends the event reads of each [view](#read-a-thread-without-a-session), and `host.observe` throws from then on. + +## Read a thread without a session + +A preview route or a status page only reads a thread. An open sets up every plugin, and can start MCP processes. `host.observe` reads the thread and opens no session. + +On the server, read the transcript in your route: + +```ts group=harness-lifecycle-observe +import { createHarnessHost, defineHarness } from '@tanstack/ai-harness' +import { memoryPersistence } from '@tanstack/ai-persistence' +import { openaiText } from '@tanstack/ai-openai' + +const assistant = defineHarness({ + name: 'acme/assistant', + adapter: openaiText('gpt-6.1-sol'), +}) +const host = createHarnessHost({ persistence: memoryPersistence() }) + +export async function GET(request: Request) { + const threadId = new URL(request.url).searchParams.get('threadId') + if (!threadId) return new Response('Missing threadId', { status: 400 }) + const view = await host.observe(assistant, { threadId }) + return Response.json(await view.transcript()) +} +``` + +In the browser, fetch the route: + +```ts group=harness-lifecycle-observe-client +const response = await fetch('/api/preview?threadId=chat-1') +console.log(await response.json()) +``` + +On a host with `access`, pass the `user` of the request. The thread needs the `read` action, else `observe` throws `AccessDeniedError`. A thread with no index entry gets the same error, so a read creates no thread. + +What the view reads for the chat: + +- `transcript()`: the saved messages. +- `snapshot()`: the status and the waiting approvals. When the session is open on this host, it also has the running operations. +- `events({ from, signal })`: the events of the thread after a cursor. +- `agentTranscript({ subagentRunId })`: the messages of one child agent run. + +What the view reads for the rest: + +- `getMedia({ id })` and `loadMedia({ id, range })`: a media file and its bytes. +- `usage()`: the token totals of the thread. +- `indexEntry()`: the index entry of the thread. +- `logState()`: the `durability.logState` fold of the log. See [One log for a session tree](./shared-logs#keep-state-for-the-whole-tree). +- `jobs()`: the background jobs that did not end. See [Watch background jobs](./coding-workspaces#4-watch-background-jobs). + +The view sets up no plugins, writes nothing, and never takes the log, so a read never fails because another host owns the thread. When the session is open on this host, the view reads through it, but it never holds the session open. The read paths of [`createHarnessHandler`](./connect) use the view, so a client that only reads starts no plugins. `GET describe` opens the session, because the plugins give the commands and the settings. + +## Bound what a session keeps in memory + +A thread can run for weeks. Each turn and each agent run that settles leaves its receipt, its result, and its events in memory. `lifecycle.retention` keeps a window of each and drops the oldest: + +```ts group=harness-lifecycle +const leanHost = createHarnessHost({ + persistence: memoryPersistence(), + lifecycle: { + retention: { keepSettledOperations: 200, keepSettledAgentRuns: 100, maxEventBytesInMemory: 2_000_000 }, + }, +}) +``` + +| Limit | Default | Past the limit | +| --- | --- | --- | +| `keepSettledOperations` | 500 | `session.operation(id)` is `undefined` for the oldest settled turns, commands, and resets. | +| `keepSettledAgentRuns` | 500 | `session.agentRun(id)` and `session.agentRuns()` leave out the oldest settled agent runs. | +| `maxEventBytesInMemory` | 8 000 000 | `session.events()` keeps the newest events, by the JSON length of each event. A reader with an older cursor reads the older events from the log. | + +The log keeps what memory drops: + +- `session.settled({ inputId })` of an old input reads the log. +- `agentRun(id)` and `agentRuns()` also find the runs in the log. +- `events({ from })` reads the older events from the log. +- `maxEventBytesInMemory` also bounds the log tail that the host keeps in memory. + +### Drop the records before a snapshot + +The log store keeps every record by default. To keep it small, let the owner drop the records that a [snapshot](./durable-sessions#keep-a-long-log-fast-to-open) already holds: + +```ts group=harness-lifecycle +const trimmedHost = createHarnessHost({ + persistence: memoryPersistence(), + lifecycle: { retention: { keepLogRecords: 'since-snapshot' } }, +}) +``` + +- `'all'` (default): the store keeps every record. +- `'since-snapshot'`: after each snapshot, the owner truncates the records before it. The type is `LogRecordRetention`. +- A reader whose cursor is in the dropped part gets one `harness.gap` event, then the stream goes on. See [Cursors](./connect#cursors). +- `session.revert()` still works. The snapshot keeps the last 500 file steps of [`snapshots()`](./snapshots#go-back-to-an-earlier-message). + +CAUTION: With `'since-snapshot'`, do not change `durability.logState` or `durability.addRecordsToModelContext` often. When the host cannot use the snapshot (its `logState` version or its `project` changed, or the snapshot has a newer version), a thread with dropped records cannot open. The open rejects with `LogTruncatedError`. Keep `'all'` if you change `reduce` often. + +## Delete a thread for good + +A user deletes a chat, and its data must go with it: + +```ts group=harness-lifecycle +await leanHost.sessions.delete({ threadId: 'chat-1' }) +``` + +First, `delete` closes the sessions of the thread and of its child threads, for every handle. The host finds the children through the `parentThreadId` of their index entries. Then it deletes this data: + +- The log, with everything in it: the transcripts, the settings, the plugin config and state, the runs, the interrupts, the usage, the snapshots, and the inputs that wait for an owner. +- The index entry of each thread. +- The uploaded media and the generated artifacts. + +The first-party plugins also end the thread: + +- [`workspaceTools()`](./coding-workspaces) reattaches an owned sandbox of the thread and closes it with `'closed'`, so it is destroyed. +- [`snapshots()`](./snapshots#remove-the-snapshots-of-a-thread) removes the snapshot repository of the thread. +- This needs the index entry of the thread, and the harness of the thread on this host. + +A host knows a harness when it opened a session of it. After a deploy or a restart, a new host has opened nothing. Give it the harnesses that it can delete threads of: + +```ts group=harness-lifecycle +const deleteHost = createHarnessHost({ + persistence: memoryPersistence(), + harnesses: [assistant], +}) +``` + +A host that does not know the harness of a thread still deletes the data. It warns, and the plugins do not end the thread: an owned sandbox stays. + +- On a host with `access`, `delete` needs the `delete` action. A child thread is checked through its root thread. A thread with no index entry throws `AccessDeniedError`. +- While the delete runs, an open of the thread throws. After the delete, a handle that you did not release opens a new, empty thread on its next call. +- A host without `stores.sessions` still closes the open sessions of the thread and deletes its data. It has no index entries, so it finds no child threads. +- `host.events()` yields a `session-deleted` event. See [Show a live status](./sessions#show-a-live-status). + +## What you have now + +- Handles that release at the end of a block, so idle sessions close by themselves. +- A cap on the open sessions of a host. +- Read routes that start no plugins and never take a thread. +- Sessions that keep a bounded window in memory, also after weeks of turns. +- A delete that removes the data of a thread and its children. diff --git a/docs/harness/mcp-server.md b/docs/harness/mcp-server.md index 97053a0c3b..da1d879e08 100644 --- a/docs/harness/mcp-server.md +++ b/docs/harness/mcp-server.md @@ -50,11 +50,15 @@ const host = createHarnessHost({ persistence: memoryPersistence() }) export const server = await createHarnessMcpServer({ host, harness: assistant, + // One user on this machine, so no login. + openAccess: true, }) serveMCPStdio(server) ``` +The host has no `access`, so the server needs `openAccess`. `true` opens each session with no user. `{ user }` opens each session as that one user. + For Claude Code, run this command one time: ```bash @@ -89,13 +93,13 @@ To talk to the harness: - `chat`: send a message and wait for the answer. While a turn runs, the message waits in the queue. To send files too, see [Send files and get media](#send-files-and-get-media). - `steer`: add a message to the running turn. - `cancel`: cancel the running turn. -- `status`: show the status, the interrupts and questions that wait, the background agents, and the queued turns. +- `status`: show the status, every wait, the background agents, and the queued turns. To answer the harness: -- `approve` and `reject`: answer every approval that waits. -- `resolve`: answer every interrupt that waits, in one call. -- `answer`: answer a question from a command or a plugin. +- `approve` and `reject`: answer every approval of the stopped turn. +- `resolve`: answer every interrupt of the stopped turn in one call, or the waits of a background agent run. +- `answer`: answer a question from a command, a durable tool, or an MCP server. To run what the harness has: @@ -119,17 +123,25 @@ Because of that, the input of an agent, a command, or a tool cannot have its own "status": "completed", "text": "You have three open tickets.", "interrupts": [], - "questions": [] + "questions": [], + "waits": [] } ``` -## Interrupts and questions +## Waits -A turn can stop for an interrupt. The turn then waits for your answer. Each interrupt has a `kind`: +The work of a tool call can stop for a [wait](./waits). The result lists the open waits of that work in `waits`. Each wait has a `kind`: - `approval`: a tool call that needs your yes or no. -- `client-tool`: a tool that the client runs. The turn waits for the tool output. -- `generic`: a question from a middleware. The answer must match its `responseSchema`. +- `question`: an answer that fits the `schema` of the wait, for example from the `question` tool or a `step.ask`. +- `sign-in`: a connector that needs you to sign in. +- `client-tool`: a tool that the client runs. The work waits for the tool output. +- `human`: a person who takes over, from an interrupt of `defineInterrupt`. + +The result also splits the waits by how you answer them: + +- `interrupts`: the waits of a stopped turn. Each one has `id`, `kind`, `message`, and `responseSchema`. An approval and a client tool also have `tool` and `args`, the tool call that waits. +- `questions`: the questions that a command, a durable tool, or an MCP server asks while it runs. For an approval, the client asks you in its own window (MCP elicitation). Answer `yes` to run the tool. Any other answer rejects it. @@ -137,7 +149,7 @@ In these cases, `chat` returns the interrupts with `status: "interrupted"`: - The client cannot ask you. - You close the question without an answer. -- An interrupt of another kind also waits. +- A wait of another kind also waits. ```json { @@ -146,7 +158,7 @@ In these cases, `chat` returns the interrupts with `status: "interrupted"`: "interrupts": [ { "id": "interrupt-1", - "kind": "generic", + "kind": "human", "message": "Review the plan", "responseSchema": { "type": "object", @@ -155,15 +167,26 @@ In these cases, `chat` returns the interrupts with `status: "interrupted"`: } } ], - "questions": [] + "questions": [], + "waits": [ + { + "waitId": "interrupt-1", + "kind": "human", + "runId": "run-1", + "message": "Review the plan", + "schema": { + "type": "object", + "properties": { "note": { "type": "string" } }, + "required": ["note"] + } + } + ] } ``` -An approval and a client tool also have `tool` and `args`, the tool call that waits. - -Then the client calls `approve`, `reject`, or `resolve`. That call waits for the turn to continue and returns the same JSON as `chat`. `approve` and `reject` answer approvals only. When an interrupt of another kind also waits, they refuse, and the client uses `resolve`. +Then the client calls `approve`, `reject`, or `resolve`. That call waits for the turn to continue and returns the same JSON as `chat`. `approve` and `reject` answer approvals only. When a wait of another kind also waits, they refuse, and the client uses `resolve`. -`resolve` takes one decision for each interrupt that waits. This example answers a turn with one approval and one generic interrupt: +`resolve` takes one decision for each interrupt that waits. This example answers a turn with one approval and one `human` wait: ```json { @@ -176,13 +199,20 @@ Then the client calls `approve`, `reject`, or `resolve`. That call waits for the - For an approval, set `approved`. `true` runs the tool, and `false` rejects it. - For a client tool, set `payload` to the tool output. -- For a generic interrupt, set `payload` to a value that matches its `responseSchema`. +- For any other kind, set `payload` to a value that matches its schema. `resolve` refuses a list that does not answer every interrupt. It also refuses a decision without the field that its kind needs. -To approve every tool call without a question, set `approvals: 'auto'`. Use it only with tools that you trust. It answers approvals only, so other interrupts still come back in the result. +To approve every tool call without a question, set `approvals: 'auto'`. Use it only with tools that you trust. It answers approvals only, so other waits still come back in the result. -A command or a plugin can also ask you a question. The result then has `status: "waiting"` and the question in `questions`. The client calls `answer` with the question `id` as `questionId` and your `value`. `answer` waits for the work that asked, then returns its result. +### Questions and background agents + +Some work keeps running while it waits. The result then has `status: "waiting"`: + +- A question in `questions`: the client calls `answer` with the question `id` as `questionId` and your `value`. +- A wait of a background agent run (an approval of `permissions()`, for example): the client calls `resolve` with one decision for each wait of that run. The `interruptId` is the `waitId`. + +`answer` and `resolve` wait for the work, then return its result. The `status` tool lists every wait, and each background agent with `status: "running"` or `"waiting"`. ## Send files and get media @@ -211,19 +241,41 @@ When the work makes media, the results of `chat`, `approve`, `reject`, `resolve` ## Serve it over HTTP -The server also has a `fetch` handler. Mount it on a route of your app, for example `/mcp`, and check a token first: +Several people use the server over HTTP, so each request needs its user. Give the host [`access`](./access), and leave out `openAccess`. The server then reads the user of each HTTP request with `getUser`: -```ts group=harness-mcp-server -export async function handleMcp(request: Request) { - const token = request.headers.get('authorization') - if (token !== `Bearer ${process.env.MCP_TOKEN}`) { - return new Response('Unauthorized', { status: 401 }) - } - return server.fetch(request) -} +```ts group=harness-mcp-server-http +import { createHarnessHost, defineHarness, membersOnly } from '@tanstack/ai-harness' +import { createHarnessMcpServer } from '@tanstack/ai-mcp/harness' +import { memoryPersistence } from '@tanstack/ai-persistence' +import { openaiText } from '@tanstack/ai-openai' + +const assistant = defineHarness({ + name: 'acme/assistant', + adapter: openaiText('gpt-6.1-sol'), +}) + +// Your own token check: a bearer token to a user. +const users = new Map([['Bearer token-ada', { id: 'ada', tenantId: 'acme' }]]) + +const host = createHarnessHost({ + persistence: memoryPersistence(), + access: { + getUser: ({ request }) => users.get(request.headers.get('authorization') ?? '') ?? null, + rules: membersOnly(), + }, +}) + +const server = await createHarnessMcpServer({ host, harness: assistant }) + +// Mount it on a route of your app, for example `/mcp`. +export const handleMcp = (request: Request) => server.fetch(request) ``` -The server opens sessions with no principal. Every caller that passes your check can use every thread and its media, so give each person their own server and token. +Each request runs as its user, and the host checks each tool call. A user reaches only the threads and the media that `isAllowed` allows. + +- A request that `getUser` refuses gets `401`. A refused action comes back as a tool error. +- A server on a host with `access` works over HTTP only. Over stdio, it throws. +- It opens no session at the start, so it lists only the agents of the harness definition. The commands and the agents that plugins add get no tool on this server. Then add it to Claude Code: @@ -236,13 +288,14 @@ claude mcp add --transport http assistant https://your-app.example.com/mcp --hea - `threadId`: the conversation of a tool call without a `threadId`. Default `main`. - `approvals`: `'ask'` (default) asks in the client, or returns the approvals when the client cannot ask. `'auto'` approves every tool call. Both answer approvals only. - `filePaths`: the folders that a `path` attachment can read from. The real path of the file, after every symlink, must be inside one of them. Default: none, so the server refuses every `path` attachment. +- `openAccess`: for a host with no `access` only. `true` opens each session with no user, and `{ user }` opens each session as that user. - `name` and `version`: the MCP server name and version. Default: the harness name and `1.0.0`. ## What you have now - Your harness in Claude Code, Claude Desktop, Cursor, or any other MCP client. - Approvals that the client asks you about, or answers with `approve` and `reject`. -- Every kind of interrupt, answered with one `resolve` call. +- Every kind of wait in one list, answered with `resolve` or `answer`. - The agents, commands, and tools of your harness as MCP tools. - Records that the client adds to a thread with `append`. - Files that the client sends with a message, and the media of each turn in the result. diff --git a/docs/harness/mcp.md b/docs/harness/mcp.md index 3430546b44..0bf3122af1 100644 --- a/docs/harness/mcp.md +++ b/docs/harness/mcp.md @@ -38,34 +38,35 @@ import { defineHarness } from '@tanstack/ai-harness' import { mcp } from '@tanstack/ai-mcp/harness' import { openaiText } from '@tanstack/ai-openai' +// One time, outside `plugins`, so the sessions share the stdio server. +const tools = mcp({ + servers: { + files: { + type: 'stdio', + command: 'npx', + args: ['-y', '@modelcontextprotocol/server-filesystem', '.'], + }, + docs: { + type: 'http', + url: 'https://mcp.example.com/mcp', + headers: { authorization: `Bearer ${process.env.DOCS_TOKEN}` }, + }, + linear: { type: 'http', url: 'https://mcp.linear.app/mcp', oauth: true }, + }, +}) + export const assistant = defineHarness({ name: 'acme/assistant', adapter: openaiText('gpt-6.1-sol'), - plugins: () => [ - mcp({ - servers: { - files: { - type: 'stdio', - command: 'npx', - args: ['-y', '@modelcontextprotocol/server-filesystem', '.'], - }, - docs: { - type: 'http', - url: 'https://mcp.example.com/mcp', - headers: { authorization: `Bearer ${process.env.DOCS_TOKEN}` }, - }, - linear: { type: 'http', url: 'https://mcp.linear.app/mcp', oauth: true }, - }, - }), - ], + plugins: () => [tools], }) ``` -The name of a server is the prefix of its tools. A `search` tool of the `docs` server is `docs_search` for the model. +The name of a server is the prefix of its tools. A `search` tool of the `docs` server is `docs_search` for the model. To choose other names, see [Rename the tools of a server](#rename-the-tools-of-a-server). -- `stdio`: the server runs as a child process on this machine (Node only). It stops when the session closes. -- `http`: a server that speaks Streamable HTTP. Send an API key in `headers`. -- `oauth: true`: the user signs in with `/connect linear`. See [Sign in to an OAuth server](#3-sign-in-to-an-oauth-server). +- `stdio`: the server runs as a child process on this machine (Node only). The host starts it one time for each harness, and every session that uses the same `mcp()` object shares it. It stops when the last session that uses it closes, or at `host.close()`. A `plugins` function that calls `mcp()` for each session starts a process for each session. +- `http`: a server that speaks Streamable HTTP. Send an API key in `headers`. Each session connects on its own. +- `oauth: true`: the user signs in with `/connect linear`. Each user connects with their own sign-in. See [Sign in to an OAuth server](#3-sign-in-to-an-oauth-server). ## 2. Check the servers @@ -82,9 +83,23 @@ linear: failed: Not signed in. Run /connect linear. In your own UI, read the same list from `session.snapshot().plugins['tanstack/mcp'].servers`. Each entry is an `McpServerStatus`: - `name`: the name of the server. -- `status`: `connecting`, `connected`, or `failed`. A `failed` entry has an `error`. +- `status`: `connecting`, `connected`, `reconnecting`, or `failed`. A `failed` entry has an `error`. - `toolCount`: how many tools the server gives the model. +### When a server stops + +A `stdio` server can crash, and a connection can close. Then `/mcp` shows `files: reconnecting`, and the turns run without the tools of that server. `mcp()` connects a server without `oauth: true` again: + +- It waits 0.5 s, then 1 s, then 2 s, and so on, up to 30 s between tries. A random part makes each wait shorter, so many sessions do not connect at the same moment. +- The waits start again at 0.5 s only after a connection stays open for 30 s. A server that crashes right after it connects waits longer each time. +- It tries until the server connects, or until the session closes. For a `stdio` server, it tries until the host stops the server. +- The next turn after the new connection has the tools again. All sessions that share a `stdio` server use the new process. +- A server that fails its first connect shows `failed`, and tries again with the same waits. A server that is not up yet, like a slow `npx`, gets its tools at a later turn. + +An OAuth server, also one from `mcpConnector`, connects again at the next turn after its connection closes. After `tools/list_changed`, the next turn lists its tools again. If that list fails, the old list stays. + +Some servers change their tools while they run. When a server sends `tools/list_changed`, the next turn lists the tools of that server again, and `/mcp` shows the new count. + ## 3. Sign in to an OAuth server 1. Run `/connect linear` in the CLI. @@ -102,7 +117,13 @@ The sign-in follows the MCP authorization spec: - A new `/connect linear` replaces them. Nothing from the old sign-in stays. - `/disconnect linear` deletes them. -Sometimes the server no longer accepts the sign-in, for example after the user removes the app in Linear. Then the turn runs without the Linear tools, and clients get a `harness.auth_required` event. The CLI prints `Sign in to linear. Run /connect linear.`, and the model tells the user to run `/connect linear`. If the server refuses the saved refresh token, the connector deletes the saved sign-in. +Sometimes the server no longer accepts the sign-in, for example after the user removes the app in Linear. Then the turn runs without the Linear tools, and clients get a `harness.auth_required` event. The CLI prints `Sign in to linear. Run /connect linear.`, and the model tells the user to run `/connect linear`. If the server refuses the saved refresh token, the connector deletes the saved sign-in. It keeps a newer token that another session saved after this session read its own. + +When a token expires, the MCP SDK of the session refreshes it. The connector saves the new token with [`credentials.refresh`](./auth#refresh-a-token): + +- The token goes back to the scope where the sign-in was, so a tenant sign-in stays the tenant's. +- If the store has `getVersioned` and `setIf`, the save is a compare-and-set. When another session saved first, its token stays, and the next connect uses it. +- Each session still sends its own refresh request. So two sessions can refresh the same token at the same time. The model never sees the token. To keep sign-ins after a restart, give the host a credential store. [Auth and connectors](./auth) shows one. With `oauth: true`, `headers` and `authProvider` do not apply. @@ -111,6 +132,52 @@ The model never sees the token. To keep sign-ins after a restart, give the host - A tool of an `oauth: true` server asks for approval before it runs, unless the server marks it read-only. - The tools of the other servers run without a question. +Set `needsApproval` on a server to change this. `true` asks before every tool of the server. A function gets the MCP tool and returns `true` to ask: + +```ts group=harness-mcp +export const careful = mcp({ + servers: { + files: { + type: 'stdio', + command: 'npx', + args: ['-y', '@modelcontextprotocol/server-filesystem', '.'], + // Ask before each tool that the server does not mark read-only. + needsApproval: (tool) => tool.annotations?.readOnlyHint !== true, + }, + docs: { type: 'http', url: 'https://mcp.example.com/mcp', needsApproval: true }, + }, +}) +``` + +## Rename the tools of a server + +Two servers can give a tool with the same name. Set `toolName` on a server to choose the names that the model sees. It gets the MCP tool and returns the name: + +```ts group=harness-mcp +export const renamed = mcp({ + servers: { + docs: { + type: 'http', + url: 'https://mcp.example.com/mcp', + toolName: (tool) => `docs_v2_${tool.name}`, + }, + }, +}) +``` + +When two servers still give the same tool name, the model gets the tool of the server that comes first in `servers`. The other tool is dropped, and the console shows one warning with the names of both servers. + +To treat a clash as a config error, set `onDuplicateTool: 'error'`. Then the later server shows `failed` with an error that names both servers and the tool, and the model gets none of its tools. The other servers keep working: + +```ts group=harness-mcp +export const strict = mcp({ + onDuplicateTool: 'error', + servers: { + docs: { type: 'http', url: 'https://mcp.example.com/mcp' }, + }, +}) +``` + ## Answer a question from a server An MCP server can ask the user for input in the middle of a tool call. MCP calls this elicitation. The session asks the user, and the tool call waits for the answer. @@ -118,7 +185,9 @@ An MCP server can ask the user for input in the middle of a tool call. MCP calls - A form request becomes a question with the `message` and the requested `schema`. - A URL request becomes a question with a `url`. The user opens the page, then answers. The session does not open the page. - `mcp()` and `mcpConnector` both ask this way, for servers on MCP spec 2025 and on spec 2026-07-28. +- A spec 2025 request does not name the tool call that asks. So the session asks the user only when all the running calls of that server belong to one run. Else the server gets `decline`, so a question never goes to another user. - One tool call can ask more than once. On spec 2026-07-28, one call can ask up to 5 times. One more request fails the tool call. +- The question is a live [wait](./waits) of the run that made the tool call. The host that runs the call keeps the thread until the answer comes. When that run stops, the wait ends `cancelled`. Answer with `session.answer({ questionId, value })`, or from a [`HarnessClient`](./connect). The value tells the server what the user did: @@ -145,7 +214,7 @@ export function answerWithCity(session: HarnessSession) { } ``` -Open questions also stay in `session.snapshot().pendingQuestions`, so a client that connects later still sees them. +Open questions also stay in `session.snapshot().pendingWaits`, so a client that connects later still sees them. ## Set a time limit @@ -211,13 +280,13 @@ export const lateTools = definePlugin({ }) ``` -- A tool whose name is already taken is skipped. The first tool keeps the name. +- A tool whose name is already taken is skipped. The first tool keeps the name. When two owners give the same name, clients get one `harness.plugin.warning` event that names both. - If `discoverTools` throws, the turn runs without that plugin's tools, and clients get a `harness.plugin.warning` event. ## What you have now - Tools from local, remote, and OAuth MCP servers, with the server name as a prefix. -- A status for each server, in `/mcp` and in your UI. +- Servers that connect again after they stop, with a status in `/mcp` and in your UI. - One browser sign-in per service, kept in your credential store. - A question before the tools that change data. - Server requests for input, asked as session questions. diff --git a/docs/harness/media.md b/docs/harness/media.md index d12e77e3f9..b924ffdfc8 100644 --- a/docs/harness/media.md +++ b/docs/harness/media.md @@ -21,7 +21,8 @@ A session saves a file with `putMedia`. `mediaPart(record)` makes the content pa ```ts group=harness-media-code import { readFile } from 'node:fs/promises' -import { createHarnessHost, defineHarness, mediaPart } from '@tanstack/ai-harness' +import { createHarnessHost, defineHarness } from '@tanstack/ai-harness' +import { mediaPart } from '@tanstack/ai-harness/media' import { memoryPersistence } from '@tanstack/ai-persistence' import { openaiText } from '@tanstack/ai-openai' @@ -33,7 +34,8 @@ const assistant = defineHarness({ const host = createHarnessHost({ persistence: memoryPersistence() }) const session = await host.open(assistant, { threadId: 'thread-1' }) -const scan = await session.putMedia(await readFile('./receipt.png'), { +const scan = await session.putMedia({ + body: await readFile('./receipt.png'), mimeType: 'image/png', name: 'receipt.png', }) @@ -45,13 +47,16 @@ console.log(turn.text) ``` - The transcript keeps a `harness-media:` URL for the file. Only the model call gets the bytes. +- The host keeps up to 32 MB of base64 text in memory, for all its sessions together, so the next turn does not encode a file again. Above that, a turn encodes the oldest files again. - If the harness refuses the file, `putMedia` throws a `MediaError`. Its `status` is `413` for a file over the size limit, and `415` for a kind that the harness does not take. The session also has: -- `getMedia(id)`: the record, or `null` when the thread has no file with this id. -- `loadMedia(id, range?)`: the bytes, or one range of them. -- `mediaUrl(id)`: `{ url }`, with a data URL for a file up to 1 MB. A bigger file gets `{}`. Use `loadMedia` for it, and serve the bytes from your own route. +- `getMedia({ id })`: the record, or `null` when the thread has no file with this id. +- `loadMedia({ id, range })`: the bytes, or one range of them. `range` is optional. +- `mediaUrl({ id })`: `{ url }`, with a data URL for a file up to 1 MB. A bigger file gets `{}`. Use `loadMedia` for it, and serve the bytes from your own route. + +Over HTTP, `createHarnessClient` gets a signed URL from the handler instead. The request for that URL still needs the login of the user, for example the cookies that an `` on the same site sends. A request with no user gets `401`, unless `access.getGuestUser` gives a guest that `isAllowed` lets read. See [Serve media to a request with no login](./access#serve-media-to-a-request-with-no-login). ## Media that agents make @@ -128,7 +133,7 @@ A media record has: - `source`: `'user'` for a file that a user sent, and `'generated'` for a file that an agent made. - `runId` and `subagentRunId`: the turn and the child agent run that made the file. -At the end of a turn, the harness adds its records to the last assistant message, in `metadata.harness.media`. So the media comes back after a restart. Read the records with `mediaOfMessage(message)`. A failed or cancelled turn keeps its media in the store only. +At the end of a turn, the harness adds its records to the last assistant message, in `metadata.harness.media`. So the media comes back after a restart. Read the records with `mediaOfMessage(message)` from `@tanstack/ai-harness/media`. A failed or cancelled turn keeps its media in the store only. - If the harness cannot save a file, the agent still gets its result. The turn publishes a `harness.plugin.warning` event. - An agent that you run from code (`session.agents`, or `ctx.agents` in a plugin) also saves its media and publishes the event. This media is not added to the transcript. @@ -167,6 +172,81 @@ gpt-6.1-sol cannot read audio files. Add media.transcribe, or send a text summar In a child agent, `ctx.chat` gets the bytes of the files too. There, only `accepts` limits the kinds, because the harness does not know the adapter of the child. +## Files inside a client message + +A browser can put a file in the message itself, as base64 bytes or as a URL. Base64 bytes make the transcript, the log, and each model call big. A URL can point to a server that the model provider then fetches. So the harness checks each part of an input from a client before the turn starts. + +The browser sends the bytes in the message: + +```ts group=harness-media-client +import { createHarnessClient } from '@tanstack/ai-harness/client' +import type { assistant } from './harness' + +const client = createHarnessClient({ + url: '/api/harness', + threadId: 'user-1-thread', +}) + +async function toBase64(file: File) { + const bytes = new Uint8Array(await file.arrayBuffer()) + return btoa(Array.from(bytes, (byte) => String.fromCharCode(byte)).join('')) +} + +export async function askAbout(file: File) { + const receipt = await client.send([ + { type: 'text', content: 'What is on this receipt?' }, + { + type: 'image', + source: { type: 'data', value: await toBase64(file), mimeType: file.type }, + }, + ]) + if (receipt.status === 'rejected') console.log(receipt.reason) +} +``` + +On the server, the harness does this for each part: + +- Base64 bytes (a `data` source, or a base64 `data:` URL) go to the media store. The message keeps a `harness-media:` reference, the same as a file from `putMedia`. `maxBytes`, `kinds`, and the model check apply. +- An `http:` or `https:` URL is refused. Set `externalUrls` to allow it. +- A provider file handle (`source: { type: 'file' }`) is refused. The adapter reads the file with your provider key, so a client that knows the file id of another user could make the model read it. Set `allowProviderFiles: true` to allow it. +- A URL with another scheme, for example `file:`, is always refused. + +`POST run` (the route of `useChat`) checks the parts of the last user message the same way. + +Three options control this: + +```ts group=harness-media-inline +import { defineHarness } from '@tanstack/ai-harness' +import { openaiText } from '@tanstack/ai-openai' + +export const assistant = defineHarness({ + name: 'acme/assistant', + adapter: openaiText('gpt-6.1-sol'), + media: { + externalUrls: (url) => url.hostname === 'cdn.acme.com', + allowProviderFiles: false, + maxBodyBytes: 8 * 1024 * 1024, + }, +}) +``` + +- `externalUrls`: `true` allows every `http:` and `https:` URL. A function decides for each URL. Default `false`. +- `allowProviderFiles`: `true` allows provider file handles. Default `false`. +- `maxBodyBytes`: the largest request body or socket frame, in bytes. Default 32 MiB. A bigger body answers `413`. A bigger frame closes the socket. The socket checks a frame only after your WebSocket server read it, so also set the frame limit of your server, for example `maxPayload` of `ws`. + +If the harness refuses a part, no turn starts. The receipt is `rejected`, and its `reason` tells you why: + +| `reason` | Why | +| ----------------------- | -------------------------------------------------------------------------------------------------------------- | +| `media_too_large` | The bytes are bigger than `maxBytes`. | +| `media_not_allowed` | The kind is not in `kinds`, or the type has no kind. | +| `media_invalid` | The bytes are not base64. | +| `media_url_not_allowed` | `externalUrls` refuses the URL, the scheme is bad, or it is a file handle and `allowProviderFiles` is not set. | + +`POST run` has no receipt. It answers with the `reason` as `error`, and with `413` for `media_too_large`, `415` for `media_not_allowed`, or `400` for the other two. + +Server code that calls `session.send` directly is not checked, so it can send file handles. The checks are for inputs from a client. + ## Keep media across restarts The host keeps media in three stores of its persistence: diff --git a/docs/harness/overview.md b/docs/harness/overview.md index 42ab38fabe..d5b3cbac0d 100644 --- a/docs/harness/overview.md +++ b/docs/harness/overview.md @@ -31,6 +31,36 @@ octane: @tanstack/ai @tanstack/ai-harness @tanstack/ai-persistence @tanstack/ai- +## Which import to use + +Import each name from the entry that holds it. Only four entries load Node modules: `plugins/node`, `plugins/coding`, `build`, and `worker`. All other entries also load on an edge runtime. One exception at run time: the loopback helpers of `oauth` (`startLoopbackReceiver` and `loopbackLogin`) start a local Node server, so call them only on Node. + +For your app and its routes: + +| Import | Holds | +| --- | --- | +| `@tanstack/ai-harness` | `defineHarness`, `createHarnessHost`, `definePlugin`, `durableTool`, `harnessText`, `oauthConnector`, and the session and agent types. | +| `@tanstack/ai-harness/server` | `createHarnessHandler` and `handleHarnessSocket`. For your own transport: `applyInput`, `parseHarnessInput`, `parseControlFrame`, `capabilitiesOf`, and `HARNESS_PROTOCOL_VERSION`. | +| `@tanstack/ai-harness/client` | `createHarnessClient`, a typed client for a browser or another process. | +| `@tanstack/ai-harness/view` | `createSessionView`, the live state of a session for a UI. | + +For plugins: + +| Import | Holds | +| --- | --- | +| `@tanstack/ai-harness/plugins` | The first-party plugins that load no Node module: `permissions`, `modelPicker`, `todos`, `goal`, `providerKeys`, and more. | +| `@tanstack/ai-harness/plugins/node` | The first-party plugins that read files. Node only: `agents`, `builtInAgents`, `fileCommands`, `projectInstructions`, and `boundToolOutput`. | +| `@tanstack/ai-harness/plugins/coding` | The coding tools. Node only: `workspaceTools`, `snapshots`, and `formatter`. | + +For helpers and builds: + +| Import | Holds | +| --- | --- | +| `@tanstack/ai-harness/oauth` | The OAuth helpers: `loopbackLogin`, `deviceLogin`, `exchangeCode`, `refreshCredential`, and more. | +| `@tanstack/ai-harness/media` | The media helpers: `mediaPart`, `mediaOfMessage`, `MediaError`, and more. | +| `@tanstack/ai-harness/build` | `buildHarness` and `artifactText`, to ship a harness as a worker. Node only. | +| `@tanstack/ai-harness/worker` | `runHarnessWorker`, which runs a harness on stdin and stdout. Node only. | + ## 1. Define the harness `defineHarness` takes the `chat()` options you know (`adapter`, `systemPrompts`, `tools`, `middleware`, `subagents`), plus a `name`. Importing it starts nothing. @@ -72,7 +102,11 @@ const host = createHarnessHost({ persistence: memoryPersistence() }) const session = await host.open(assistant, { threadId: 'thread-1' }) ``` -Opening the same `threadId` again returns the live session. +Opening the same `threadId` again gives another handle to the same live session. Release a handle with `await session.close()`, or open it with `await using`, so an idle session can close. See [Lifecycle and retention](./lifecycle). + +Does the host serve many users? Give it `access`, so each user reaches only their own sessions. See [Access and tenants](./access). + +To see every option of `createHarnessHost`, and which ones your app needs, see [Host options](./host-options). ## 3. Send a message @@ -128,7 +162,7 @@ console.log(price.cents) The session adds a short note about the result, or the error, to the transcript. The model sees it on the next turn. To run the agent in the background and start a new turn when it finishes or fails, use `start`: ```ts group=harness-first -session.agents.pricer.start({ vendor: 'globex' }, { wake: true }) +session.agents.pricer.start({ vendor: 'globex' }, { startTurnWhenDone: true }) ``` ## 6. Add a plugin @@ -159,23 +193,21 @@ export const withClock = defineHarness({ }) ``` -`plugins` is a function, so every session gets its own plugin instances. Set `lifetime: 'turn'` on a plugin to set it up again for each turn. +`plugins` is a function, so every session gets its own plugin instances. Set `lifetime: 'turn'` on a plugin to set it up again for each turn, or `lifetime: 'host'` to share one setup between all the sessions of the host. If two plugins add a tool with the same name, `host.open` fails and names both plugins. To write your own, see [Write a harness plugin](./plugins). ## Keep sessions across restarts -The host takes the same stores as `withPersistence`. Add an `inbox` store and the session also keeps the messages it accepted but did not start yet. When the session opens again, it runs them. +Each host keeps every thread in a session log: the transcript, the inputs, the runs, the approvals that wait, the settings, and the plugin state. The host needs one store for it, `log`: -- `messages`: the transcript. Required. -- `runs`: the record of every turn and agent run. When the host stops during a background agent run, the next session ends that run `failed` and notes it in the transcript. -- `interrupts`: approvals that wait for a user. -- `inbox`: accepted messages that did not run yet. -- `metadata`: the session config, plugin state, and the interrupts that the last turn waits on. With it, `session.resolve` continues that turn after a restart, also a turn that a router sent to agents. +- `log`: the session log. Required. +- `sessions`: one index entry per thread, for a list of sessions. See [List, rename, and fork sessions](./sessions). +- `metadata`: data wider than one thread, for example the saved rules of `permissions()`. -`memoryPersistence()` has every store, but it keeps them in memory only. Write your own stores to keep data in your database. See [Build your own adapter](../persistence/build-your-own-adapter). +`memoryPersistence()` has every store, but it keeps them in memory only. To keep the log in your database, implement the [`LogStore` contract](../persistence/store-reference#logstore). Then a turn continues after a crash or a deploy. See [Durable sessions](./durable-sessions). -To continue a turn after a crash or a deploy, give the host a session log. See [Durable sessions](./durable-sessions). +Before you run many servers or long threads, read the [known limits](./deploy#known-limits). ## Prompt caching @@ -198,7 +230,7 @@ const support = await host.open(assistant, { }) ``` -If the `threadId` already has a live session, `host.open` returns that session with its first `promptCache` value. The agents that the model calls as tools use the retention of the turn. `session.usage()` shows the cache reads and writes, see [Track usage and cost](./usage). For the options and the cost, see [Prompt Caching](../advanced/prompt-caching). +If the `threadId` already has a live session, `host.open` gives a handle to that session, which keeps its first `promptCache` value. The agents that the model calls as tools use the retention of the turn. `session.usage()` shows the cache reads and writes, see [Track usage and cost](./usage). For the options and the cost, see [Prompt Caching](../advanced/prompt-caching). To change it for one prompt, pass `overrides.promptCache`. See [Give one prompt its own settings](./turn-control#give-one-prompt-its-own-settings). diff --git a/docs/harness/permissions.md b/docs/harness/permissions.md index d2a59313c4..accc245887 100644 --- a/docs/harness/permissions.md +++ b/docs/harness/permissions.md @@ -55,21 +55,27 @@ const session = await host.open(notes, { threadId: 'notes-1' }) The rule makes each `save_note` call wait for the user. -## 2. Answer the first question +## 2. Answer the first approval -`session.onQuestion` calls you once for each question. Answer it with `session.answer`: +The plugin decides each call before it runs. A call that asks opens one approval [wait](./waits), and the turn stops with status `requires_action`. Answer every approval of the turn in one `resolve`: ```ts group=harness-permissions -const stopAnswering = session.onQuestion(({ question }) => - // question.message is, for example: Allow save_note {"path":"meeting.md","text":"..."}? - session.answer({ questionId: question.questionId, value: 'once' }), -) - await session.send('Save a note with the time of the meeting.') -stopAnswering() + +// wait.message is, for example: Allow save_note {"path":"meeting.md","text":"..."}? +const waits = session.snapshot().pendingWaits +await session.resolve({ + inputId: 'approve-1', + resume: waits.map((wait) => ({ + interruptId: wait.waitId, + status: 'resolved', + payload: 'once', + })), +}) +await session.settled({ inputId: 'approve-1' }) ``` -The listener says `once` to each question. The agent writes the file and continues. +The answer `once` lets the call run. The agent writes the file and continues. No process waits for the user while the turn is stopped, so the answer can come from any server, also after a restart. ## Pick a mode @@ -85,12 +91,14 @@ The `mode` setting decides how much runs without a question: Switch the mode from server code: ```ts group=harness-permissions -await session.setConfig({ key: 'mode', value: 'plan' }) +await session.setPluginSetting({ key: 'mode', value: 'plan' }) // Or run the command. It returns 'Mode: plan.' -await session.command('mode', 'plan') +await session.runCommand({ name: 'mode', input: 'plan' }) ``` -`session.command('mode')` with no input shows the current mode. In `acceptEdits` mode, an edit outside `root` or of a `.env` file still asks. +`session.runCommand({ name: 'mode' })` with no input shows the current mode. In `acceptEdits` mode, an edit outside `root` or of a `.env` file still asks. + +A mode change applies from the next turn. A turn that runs keeps the mode that it started with, so a switch to `bypass` does not change its later tool calls. ### Keep the mode on the server @@ -155,6 +163,8 @@ Two more rules apply to each tool: - A tool with no rule runs without a question. To ask for each tool that has no rule, set `default: 'ask'`. - When the last rule of a tool is a `deny` with no `resource`, the model does not get the tool. +The plugin decides each call in the guard hook, [`onGuardToolCall`](../advanced/middleware#onguardtoolcall). It runs after the input check and after every middleware that changes the arguments. So the rules see the arguments that the tool gets. + ### Paths - Inside `root`, a rule matches the path from `root`, for example `src/app.ts`. Outside `root`, it matches the full path, for example `/etc/hosts`. @@ -198,24 +208,28 @@ export const exampleEnv = permissions({ - `websearch` has no rule, so it runs. - The path rules above also apply. For example, `read_file` asks for a `.env` file. - The coding tools check the real path of a link too, and the stricter decision wins. So a link `notes` to `.env` asks, like `.env`. +- The rules decide a path outside `root` and the real path of a link before the call runs. A coding tool never asks while it runs. - `grep` skips each file that `read_file` asks about or denies, also after an `always` answer. A note gives the number of skipped files. - Your `rules` come after these rules, so they win. `{ tool: 'webfetch', decision: 'allow' }` lets `webfetch` run without a question. ### Formatters -A formatter runs code from the project, for example `prettier.config.js` or a package in `node_modules`. The model can edit these files. So [`formatter()`](./coding-tools-backends#format-each-file-after-a-write) asks before each run, like `bash`. +A formatter runs code from the project, for example `prettier.config.js` or a package in `node_modules`. The model can edit these files. So the rules decide each run of [`formatter()`](./coding-tools-backends#format-each-file-after-a-write), like `bash`. It adds one rule: `{ tool: 'formatter:*', decision: 'ask', kind: 'execute' }`. Each run is named `formatter:` and the formatter name, for example `formatter:prettier`. -| Mode | What the formatter does | -| --- | --- | -| `default` | It asks before it runs. | -| `acceptEdits` | It asks before it runs. The edit itself runs without a question. | -| `bypass` | It runs without a question. | -| `plan` | It does not run. No file changes in this mode. | +The rules decide the formatter as a part of the call that writes, before that call runs. The call gets one approval for the write and its formatter. The formatter runs after the write, so no question comes in the middle of a call. -- An `always` answer saves `formatter:prettier` for the project, so the user answers once. -- A `reject` skips the formatter for that write. The file stays as the tool wrote it, and the tool result says that the formatter did not run. +| Mode | What happens | +| --- | --- | +| `default` | The write asks, and the approval also lets the formatter run. | +| `acceptEdits` | A write that a formatter formats asks, because the formatter asks. | +| `bypass` | The write and the formatter run without a question. | +| `plan` | Neither runs. No file changes in this mode. | + +- A `reject` stops the call. Nothing is written, and the formatter does not run. +- An `always` answer also saves the formatter for the project, tied to a hash of its config files (`formatter:prettier@`). The user answers once, until those files change. +- A rule that denies the formatter skips it. The write still runs, and the tool result says that the formatter did not run. - Without `permissions()`, the formatter runs without a question. This rule lets prettier run without a question: @@ -227,19 +241,20 @@ export const trustedFormatter = permissions({ }) ``` -## Answer a question +## Answer an approval -A call that asks waits for one of three answers: +The plugin decides every call of a model message before any of them runs. Each call that asks opens one approval wait of kind `approval`, and the turn stops. Send the answer as the `payload` of that wait in `session.resolve`. The turn continues when every wait of the turn has an answer. | Answer | What happens | | --- | --- | -| `once` | This call runs. | +| `once` or `true` | This call runs. | | `always` | This call runs, and the plugin saves an `allow` rule for it. | -| `reject` | The call does not run. The model gets a tool error. | +| `reject` or `false` | The call does not run. The model gets a tool error. | -- Send the answer as a string, `'once'`, or as an object, `{ answer: 'once' }`. +- Send the answer as a string, `'once'`, as an object, `{ answer: 'once' }`, or as `true` or `false`. - A reject can carry a `message` for the model: `{ answer: 'reject', message: 'Use pnpm, not npm.' }`. The model gets your message as the tool error. -- Any other answer rejects the call. This includes `y` and `yes`. +- Any other answer rejects the call. This includes `'y'` and `'yes'`. +- The `schema` of the wait describes these answers, so a client can show a form for it. ### What `always` saves @@ -251,7 +266,7 @@ A call that asks waits for one of three answers: ### See and forget saved answers -1. Run `session.command('permissions')`. The command lists the saved rules of the project: +1. Run `session.runCommand({ name: 'permissions' })`. The command lists the saved rules of the project: ```text Saved rules: @@ -259,7 +274,7 @@ Saved rules: 2. read_file .env ``` -2. Run `session.command('permissions', 'forget 2')` to delete rule 2. The open session asks again at the next `read_file` call for `.env`. +2. Run `session.runCommand({ name: 'permissions', input: 'forget 2' })` to delete rule 2. The open session asks again at the next `read_file` call for `.env`. A client can run the command only when `expose.commands` has `permissions`. @@ -285,9 +300,10 @@ if (first) await deleteSavedPermission(persistence.stores, root, first) The same check runs in every agent run of the session: - Each call in a child run gets the same rules and the same mode. This includes the agents that the model calls, the agents that plugins start, and their children. -- A question from a child waits in the same session. Answer it the same way. +- A background agent run that needs an approval stops with status `waiting`. Its approval is in `pendingWaits` of the same session. Answer it with `resolve`, and the run continues. See [When a background agent waits](./subagents#when-a-background-agent-waits). +- An agent that the model calls as a tool inside a turn stops the turn for its approval. The wait has the `agent` that asks. After the `resolve`, that agent goes on. See [When an agent tool waits](./subagents#when-an-agent-tool-waits). - In `plan` mode, a child cannot change a file. Its call gets the tool error `This tool is not allowed in plan mode.` -- The `permissions` of an [agent profile](./agents) count only while that profile is the primary agent. They do not apply when the profile runs as a subagent. +- The `permissions` of an [agent profile](./agents) count while that profile is the primary agent, in its turns and in each child run. A child run does not use the rules of its own profile. - [Coding agents](./coding-agents#plan-mode) such as Claude Code start read-only in `plan` mode. ## Rules for your own tool @@ -372,12 +388,12 @@ export const planTools = definePlugin({ ## Known limits - `plan` denies only edits, commands, and calls that ask. A tool with no rule, or with an `allow` rule and no `kind`, still runs in `plan` mode. Give each tool that changes something a rule with a `kind`. -- A client that can send inputs to the thread can answer the questions, also with `always`. Control who can open a thread. +- A client that can answer the waits of the thread can answer the approvals, also with `always`. Control who can open a thread. - `permissions()` does not follow links. Your own tool can check the real path of a link with `PermissionDecisionCapability`. ## What you have now -- A question before each risky tool call. +- An approval before each risky tool call, that any server can answer. - Modes to plan, to edit fast, or to run everything. - A mode that a client cannot change, unless you expose it. - Rules for folders, files, and commands, and saved answers for each project that you can list and forget. diff --git a/docs/harness/plugins.md b/docs/harness/plugins.md index e0b9907427..e86601f49b 100644 --- a/docs/harness/plugins.md +++ b/docs/harness/plugins.md @@ -24,14 +24,14 @@ export const today = definePlugin({ }) ``` -Add it with `plugins: () => [today]` in `defineHarness`. `setup` runs once per session. +Add it with `plugins: () => [today]` in `defineHarness`. `setup` runs once per session. To share one setup between all the sessions of a host, see [Share one setup between sessions](#share-one-setup-between-sessions). ## What `setup` can return For the model: - `tools`: tools for the model, made with `toolDefinition`. -- `prompts`: text for the system prompt. A function runs for each turn, so it can show current state. +- `prompts`: text for the system prompt. A function runs for each turn, so it can show current state. For text that changes often, see [Put changing text last](#put-changing-text-last). - `middleware`: chat middleware for the lead turn, the same type as `chat({ middleware })`. `agentMiddleware` is for agent runs, see [Middleware in every agent](#middleware-in-every-agent). - `generationMiddleware`: middleware for the activities agents call. - `adapter`: a picker for the model of each turn, see [Pick the model of a turn](#pick-the-model-of-a-turn). @@ -48,6 +48,59 @@ If two plugins add the same tool, command, setting, or prompt id, `host.open` fa Return these from `setup`, not from the top level of `definePlugin`. A key like `adapter` or `tools` at the top level is a type error, and `definePlugin` throws for it. Other keys at the top level stay on the plugin object, so a plugin can give your code its own functions. +### Put changing text last + +Set `placement: 'context'` on a prompt whose text changes often: + +```ts group=harness-plugins +let openTickets = 0 + +export const tickets = definePlugin({ + name: 'acme/tickets', + setup: () => ({ + prompts: [ + { + id: 'acme/tickets:count', + text: () => `Open tickets: ${openTickets}.`, + placement: 'context', + }, + ], + }), +}) +``` + +A provider caches the start of each request: the system prompts, then the history. A change of a system prompt makes the cache miss from that prompt on. A `'context'` prompt is the last system prompt, so the system prompts before it stay in the cache. + +- The model gets all `'context'` text as one system prompt, after the other system prompts. The harness makes it again at each model call. +- The text has the authority of a system prompt, not of a user message. +- The default is `'system'`: the text goes in the system prompts, in plugin order. +- The log does not keep `'context'` text. +- The first-party todos, goal, and plan mode prompts use `'context'`. + +### Refresh state at the start of a turn + +A prompt function must return at once, so it cannot read a file or call a server. Do that work in `onTurnStart`. The harness calls it once at the start of each turn, before it reads the prompts: + +```ts group=harness-plugins +import { readFile } from 'node:fs/promises' + +let rules = '' + +export const projectRules = definePlugin({ + name: 'acme/project-rules', + setup: () => ({ + onTurnStart: async () => { + rules = await readFile('RULES.md', 'utf8') + }, + prompts: [() => rules], + }), +}) +``` + +- `onTurnStart` gets the turn. `turn.run` has the user of the turn and the credentials of that user. +- It runs one time for each turn, also when the turn makes many model calls. +- If it throws, the turn continues, and clients get a `harness.plugin.warning` event. + ## Add commands and settings Commands are actions for the user, not the model: slash commands and buttons in your app. Settings are typed, saved per session, and apply at the next turn. @@ -67,7 +120,8 @@ export const greeter = definePlugin({ description: 'Greet someone', input: z.object({ name: z.string() }), run: async ({ name }, command) => { - const sure = await command.session.ask({ + const sure = await command.step.ask({ + name: 'confirm', message: `Greet ${name}?`, schema: z.boolean(), }) @@ -80,9 +134,11 @@ export const greeter = definePlugin({ }) ``` -- `session.command('greet', { name: 'Ada' })` runs the command. A Standard Schema input (zod here) is checked, and the command gets the parsed value. A JSON Schema input is not checked, the same as a JSON Schema tool input. -- `session.setConfig({ key: 'tone', value: 'warm' })` changes the setting. A value the option does not accept is rejected. -- `ask` waits for `session.answer({ questionId, value })`. The question goes to `session.onQuestion` listeners, and it stays in `session.snapshot().pendingQuestions` until someone answers. +- `session.runCommand({ name: 'greet', input: { name: 'Ada' } })` runs the command. A Standard Schema input (zod here) is checked, and the command gets the parsed value. A JSON Schema input is not checked, the same as a JSON Schema tool input. +- `session.setPluginSetting({ key: 'tone', value: 'warm' })` changes the setting. A value the option does not accept is rejected. +- `step.ask` waits for `session.answer({ questionId, value })`. The log keeps the question as a [wait](./waits), so a user on any server can answer it. The question goes to `session.onQuestion` listeners, and it stays in `session.snapshot().pendingWaits` until someone answers. +- When the host stops while the command waits, the next host runs the command again from the top. Put each side effect before the ask in `command.step.do`, so it does not run twice. +- `ctx.session.ask` is only for an MCP server that asks during a tool call. Anywhere else it throws. Use `step.ask`, the `question` tool, or a [permission rule](./permissions#rules-for-your-own-tool). ## Change commands while the session runs @@ -166,9 +222,11 @@ export const helpers = definePlugin({ - `ctx.agents.delete(name)` removes an agent of this plugin, and its subagent tool. Another owner's name does nothing. - `session.agent(name)`, `ctx.agents.list()`, and `routing` see the change at once. +`ctx.agents` also runs agents: `start`, `run`, and `group`, with `g.start` for each child of a group. It has the same type, `AgentActions`, as `ctx.harness.agents` in agent code. See [Run agents from a plugin](./subagents#run-agents-from-a-plugin). + ## Keep state -`ctx.state(initial)` gives a plugin its own state, saved in the metadata store. Without a metadata store, the state stays in memory, and a restart loses it. Clients see each change as an AG-UI `STATE_SNAPSHOT` event. +`ctx.state(initial)` gives a plugin its own state for the thread. The session log keeps each change as a record, so any host that opens the thread reads the state back, also after a restart. Clients see each change as an AG-UI `STATE_SNAPSHOT` event. ```ts group=harness-plugins export const counter = definePlugin({ @@ -187,7 +245,12 @@ export const counter = definePlugin({ }) ``` -When two writers race, `update` runs your function again with fresh state. +- The state is JSON. +- A key or a token of the session in the state goes into the log as `[redacted]`. `update` resolves with that same value. See [Keep tokens out of the log and the model](./auth#keep-tokens-out-of-the-log-and-the-model). +- The updates of one plugin run one at a time. Each `update` function gets the value that the update before it wrote. +- [`host.sessions.delete`](./lifecycle#delete-a-thread-for-good) deletes the state with the log. A fork does not copy it. +- For a long list of records, such as one record for each step, append records to the log and set `keepRecords`. Read them with `ctx.session.keptRecords(type)`. See [Keep records after a truncate](./session-log#keep-records-after-a-truncate). +- Agents get a state with the same `get` and `update` from `ctx.harness.state({ key, initial })`. See [Share state between agents](./subagents#share-state-between-agents). ## Middleware in every agent @@ -224,6 +287,98 @@ export const usageByAgent = definePlugin({ - `ctx.subagentName` is the name of the agent. In the lead turn, it is `undefined`. - `ctx.subagentRunId` is different for each run of an agent. For a nested child, `ctx.parentSubagentRunId` is the id of the child that started it. - Each model call goes to `onUsage` one time, in the run that made the call. The lead turn does not count the usage of a child again. +- A background run that a turn starts keeps the plugins of that turn. So the `agentMiddleware` of a plugin with `lifetime: 'turn'` runs in it too. The turn plugins are cleaned up when the turn and its runs have settled. +- A tool result that your `onAfterToolCall` replaces is the result that the session log keeps. After a crash, the model gets that result again. See [The other tools of a cut batch](./durable-tools#the-other-tools-of-a-cut-batch). + +## Act as the user of a run + +Several people can write in one thread, and a background agent can run after its turn ended. A tool must use the GitHub token of the user who started its own run, not of the user who sent the latest message. Each run carries its user, and that user's credentials and keys, as the run context: + +```ts group=harness-plugins +import { toolDefinition } from '@tanstack/ai' +import type { HarnessRunContext } from '@tanstack/ai-harness' + +export const github = definePlugin({ + name: 'acme/github', + setup: () => ({ + tools: [ + toolDefinition({ name: 'my_repos', description: 'List my GitHub repos' }).server( + async (_args, ctx) => { + const credential = await ctx.context.run.credentials.require('github') + const token = credential.type === 'oauth' ? credential.accessToken : credential.value + const response = await fetch('https://api.github.com/user/repos', { + headers: { authorization: `Bearer ${token}` }, + }) + return response.json() + }, + ), + ], + }), +}) +``` + +Where each part of a plugin reads the run: + +- A tool or a middleware hook: `ctx.context.run`. Type it with `HarnessRunContext`, for example `.server(...)` or `ChatMiddleware`. +- A command: `run` of the command context. Its `credentials` is the same as `run.credentials`. +- A plugin with `lifetime: 'turn'`: `ctx.turn.run`. Its `ctx.credentials` and `ctx.keys` belong to the same user. +- A session plugin (the default lifetime): `ctx.credentials` and `ctx.keys` have no user, so each call throws. `ctx.session.user` is the owner of the session. +- A plugin with `lifetime: 'host'`: its setup has no user and no session. Its tools read the user of each call from `ctx.context.run`. + +`prepareTools` gets `run` too: the run of the turn, with its id and its user. `onTurnStart` gets the turn, with its run in `turn.run`. + +A plugin that starts a turn picks its user. `ctx.session.send(text, { user })` and `ctx.session.note(text, { wake: true, user })` run that turn as `user`. Without it, the turn runs as `ctx.session.user`. To continue the work of a run, pass the `user` of that run. + +The run has these fields: + +| Field | What it is | +| --- | --- | +| `user` | The user who started the run: `{ id, tenantId }`. `undefined` in a session with no user. | +| `tenantId` | The tenant of the session. `undefined` in the default tenant. | +| `threadId`, `runId`, `parentRunId` | The thread, this run, and the run that started this one. | +| `logId` | The id of the log of the session tree in the store, with its tenant. | +| `epoch` | The ownership epoch of the log when the run started. See [Fence your own side effects](./durable-sessions#fence-your-own-side-effects). | +| `idempotencyKey(step, scope?)` | A key for one side effect of the run. A run again on any host gets the same key. | +| `signal` | Aborts when the run stops. | +| `credentials`, `keys` | The credentials and the model provider keys of the user of the run. The session replaces their values with `[redacted]` in what it keeps and sends. To refresh an OAuth token, use `credentials.refresh`. See [Auth and connectors](./auth#refresh-a-token). | + +## Call a model from a plugin + +Your plugin needs a model call of its own: a judge, a title, or a summary. A plain `chat()` call does not count in `session.usage()`, and it keeps running after the user cancels the turn. Call the model with `ctx.model.chat` instead: + +```ts group=harness-plugins +export const recap = definePlugin({ + name: 'acme/recap', + setup: (ctx) => ({ + commands: { + recap: defineCommand({ + description: 'Sum up the conversation in three lines', + run: async (_input, { run }) => { + const transcript = await ctx.session.transcript() + const { text } = await ctx.model.chat({ + run, + label: 'recap', + options: { + messages: [...transcript, { role: 'user', content: 'Sum up this conversation in three lines.' }], + }, + }) + return text + }, + }), + }, + }), +}) +``` + +The options: + +- `run`: the run that the call is for. The call uses the keys of its user, and the budget of the run applies. A plugin with `lifetime: 'turn'` can leave it out. Then the call uses the run of its turn. +- `adapter`: the model. Without it, the call uses the model of the session: the thread setting `model`, else the harness `adapter`. A `keyedAdapter(...)` gets the key of the user of the run. +- `options`: `messages`, `systemPrompts`, and `outputSchema`. With `outputSchema`, the call resolves to the checked value. Without it, the call resolves to `{ text, chunks }`. +- `label`: a name for the call in its usage record. +- `timeoutMs`: the time limit of the call. The default is 2 minutes. + +The call stops when its run stops, for example on a cancel, and when the time limit ends. Then it rejects. Its usage counts in the session under `plugin:`, here `plugin:acme/recap`. See [Usage of plugins](./usage#usage-of-plugins). The `goal()`, `title()`, and `compact()` plugins call their models this way. ## Pick the model of a turn @@ -253,8 +408,9 @@ The `turn` has these fields: | `inputId` | The id of the input, when the turn has one. | | `message` | The user message of the input. `undefined` for a resolve and a continue. | | `context` | The [context of the input](./inputs#send-context-with-a-message). It is client data, so do not trust it. | -| `principal` | Who sent the input. Default: the principal that opened the session. See [Share a thread](./shared-threads). | +| `user` | Who sent the input. Default: the user that opened the session. See [Share a thread](./shared-threads). | | `overrides` | The [overrides of the prompt](./turn-control#give-one-prompt-its-own-settings), when it has some. | +| `run` | The run of the turn: its user, and that user's credentials and keys. See [Act as the user of a run](#act-as-the-user-of-a-run). | The session uses the picks like this: @@ -280,7 +436,7 @@ export const modelOfTheOrg = definePlugin({ name: 'acme/model-of-the-org', lifetime: 'turn', setup: async (ctx) => { - const plan = await planOf(ctx.turn?.principal?.tenantId) + const plan = await planOf(ctx.turn?.user?.tenantId) const adapter = plan === 'strong' ? anthropicText('claude-sonnet-5-5') @@ -355,13 +511,76 @@ Read `ctx.collect` at run time (in a command, a tool, or a middleware hook). Dur ## Clean up resources -Open resources with `ctx.resources.acquire(open, close)`. They close when the session closes, newest first. If a later plugin fails in `setup`, every plugin set up so far is cleaned up, and no model request is sent. +Open resources with `ctx.resources.acquire(open, close)`. They close when the plugin is cleaned up, newest first: when the session closes, or for `lifetime: 'turn'`, after the turn and the background runs that it started. If a later plugin fails in `setup`, every plugin set up so far is cleaned up, and no model request is sent. Set `lifetime: 'turn'` to set a plugin up again for each turn. +## Share one setup between sessions + +A plugin that opens a database pool, or starts a process, does that work again for each session. A host with 300 open sessions then holds 300 pools. Set `lifetime: 'host'`, and the host sets the plugin up one time and shares it with every session: + +```ts group=harness-plugins-host +import { toolDefinition } from '@tanstack/ai' +import { definePlugin } from '@tanstack/ai-harness' + +// Your shared client, for example a database pool. +async function connect() { + return { countOrders: async () => 42, end: async () => {} } +} + +export const orders = definePlugin({ + name: 'acme/orders', + lifetime: 'host', + setup: async (ctx) => { + const db = await ctx.resources.acquire(connect, (pool) => pool.end()) + return { + tools: [ + toolDefinition({ name: 'count_orders', description: 'Count the orders' }).server( + async () => db.countOrders(), + ), + ], + } + }, +}) +``` + +- The host sets the plugin up one time for each harness and plugin object. Every session of that harness that uses the same object shares this setup. +- Create the plugin one time, outside the `plugins` function, as `orders` is above. Then use it with `plugins: () => [orders]`. A `plugins` function that calls `definePlugin` or `mcp()` for each session gives each session its own setup. +- The host cleans it up when the last session that uses it closes, or at `host.close()`. +- Its setup context has no session, no credentials, no keys, and no commands. It has `harness`, the name of the harness, and `resources`. +- A tool of the plugin reads the user of its call from `ctx.context.run`. See [Act as the user of a run](#act-as-the-user-of-a-run). + +Some parts need the session: a command, a setting, plugin state, or a question to the user. Return them from `session`. It runs for each session, with the same context as a session plugin: + +```ts group=harness-plugins-host +import { defineCommand } from '@tanstack/ai-harness' + +export const pool = definePlugin({ + name: 'acme/pool', + lifetime: 'host', + setup: async (ctx) => { + const db = await ctx.resources.acquire(connect, (shared) => shared.end()) + return { + session: (session) => ({ + commands: { + orders: defineCommand({ + description: 'Count the orders', + run: async () => `${await db.countOrders()} orders in ${session.session.threadId}`, + }), + }, + }), + } + }, +}) +``` + +The session gets the shared contributions and the `session` contributions together. The resources of `session` close with the session. + +The [MCP plugin](./mcp) works this way for `stdio` servers: one process serves every session of the host. + ## Reload plugins -You changed a plugin, an agent file, or the list that `plugins()` returns, and an open session still runs the old setup. `session.reload()` sets the plugins up again: +You changed a plugin, an agent file, or the list that `plugins()` returns, and an open session still runs the old setup. `session.reloadPlugins()` sets the plugins up again: ```ts group=harness-plugins import { watch } from 'node:fs' @@ -371,7 +590,7 @@ const host = createHarnessHost() const session = await host.open(assistant, { threadId: 'thread-1' }) // One session. -await session.reload() +await session.reloadPlugins() // Every open session of the harness, each time a file in the folder changes. watch('./plugins', () => void host.reload(assistant)) @@ -384,11 +603,11 @@ A reload works like this: 3. It calls `plugins()` again and runs `setup` of each session plugin. 4. Clients get a `harness.reloaded` event. Read the commands, the settings, and the agents again. -The transcript, the log, the thread settings, and the inbox stay. The next turn gets the new tools, prompts, and middleware. The `ctx.on` listeners of the old plugins stop. +The transcript, the log, the thread settings, and the inputs that wait stay. The next turn gets the new tools, prompts, and middleware. The `ctx.on` listeners of the old plugins stop. The harness does not watch files. Your host decides when to reload, as the `watch` call above does. -If a `setup` throws, the session keeps working with no plugins. The promise rejects with the error, and the `harness.reloaded` event has its `error` message. Fix the plugin and call `reload()` again. +If a `setup` throws, the session keeps working with no plugins. The promise rejects with the error, and the `harness.reloaded` event has its `error` message. Fix the plugin and call `reloadPlugins()` again. ## See the plan @@ -398,8 +617,10 @@ If a `setup` throws, the session keeps working with no plugins. The promise reje - A plugin that adds tools, prompts, commands, settings, and state to any harness. - Middleware that sees every model call, in the lead turn and in every agent. +- Tools and commands that act as the user of their own run. +- Model calls of your plugin that count in the session usage and stop with their run. - A picker that chooses the model of each turn from its message, context, and sender. - Plugins that share services, lists, and events without knowing each other. - Commands, agents, and whole plugins that change while the session runs. -Next: add the first-party plugins from `@tanstack/ai-harness/plugins`. [Ask before risky tool calls](./permissions) and [Switch between agent profiles](./agents) are two of them. +Next: add the first-party plugins from `@tanstack/ai-harness/plugins` and `@tanstack/ai-harness/plugins/node`. [Ask before risky tool calls](./permissions) and [Switch between agent profiles](./agents) are two of them. diff --git a/docs/harness/protocol.md b/docs/harness/protocol.md new file mode 100644 index 0000000000..9e3a1efad9 --- /dev/null +++ b/docs/harness/protocol.md @@ -0,0 +1,480 @@ +--- +title: The harness wire protocol +id: harness-protocol +order: 3 +description: "Every route, frame, and harness.* event of the harness session stream. Use it to write your own client or transport in any language." +keywords: + - tanstack ai + - harness + - protocol + - AG-UI + - websocket + - server-sent events +--- + +You want a client for a harness that is not `useChat` or `createHarnessClient`: a Python service, a mobile app, or a proxy of your own. This page lists every route, frame, and `harness.*` event, so you can write that client in any language. + +Send a prompt to a thread with `curl`: + +```bash +curl -X POST https://example.com/api/harness/control \ + -H 'authorization: Bearer my-token' \ + -H 'content-type: application/json' \ + -d '{"threadId":"thread-1","input":{"op":"prompt","message":"Hello"}}' +``` + +The command prints the receipt, for example `{"inputId":"in-1","status":"accepted","operationId":"op-1"}`. The first prompt also makes the thread. Then open the session stream of the thread: + +```bash +curl -N -H 'authorization: Bearer my-token' \ + 'https://example.com/api/harness/events?threadId=thread-1&v=2' +``` + +With no cursor, the stream starts at the first event of the thread. It shows the hello frame, then one SSE event per session event, and a ping comment when the stream is idle: + +```text +data: {"type":"harness.hello","v":2,"threadId":"thread-1"} + +id: 4 +data: {"type":"harness.event","cursor":"4","threadId":"thread-1","operationId":"op-1","event":{"type":"TEXT_MESSAGE_CONTENT","messageId":"m-1","delta":"Hi"}} + +: ping +``` + +The rest of this page is the reference for each part. + +## Transports + +The server is the handler of [Connect clients to a harness](./connect). One handler and one socket serve every transport on this page: + +```ts group=harness-protocol-server +import { createHarnessHost, defineHarness, membersOnly } from '@tanstack/ai-harness' +import { createHarnessHandler, handleHarnessSocket } from '@tanstack/ai-harness/server' +import { memoryPersistence } from '@tanstack/ai-persistence' +import { openaiText } from '@tanstack/ai-openai' +import type { WebSocketLike } from '@tanstack/ai' + +export const assistant = defineHarness({ + name: 'acme/assistant', + adapter: openaiText('gpt-6.1-sol'), +}) + +const host = createHarnessHost({ + persistence: memoryPersistence(), + access: { + getUser: ({ request }) => + request.headers.get('authorization') === `Bearer ${process.env.HARNESS_TOKEN}` + ? { id: 'user-1' } + : null, + rules: membersOnly(), + }, +}) + +// Mount it on `/api/harness/*`. +export const handler = createHarnessHandler({ host, harness: assistant }) + +// Call it from the upgrade handler of your server. +export function onUpgrade(socket: WebSocketLike, request: Request) { + handleHarnessSocket({ host, harness: assistant, socket, request }) +} +``` + +Pick the transport that your client can use: + +| Transport | Events | Inputs | Use it when | +| --- | --- | --- | --- | +| HTTP | `GET events` (SSE) | `POST control` (JSON) | Your client can read SSE. This works through most proxies. | +| WebSocket | `harness.event` frames | `harness.input` frames | You want one connection for events and inputs. | +| Stdio worker | NDJSON on stdout | NDJSON on stdin | You start the harness as a child process. | + +The socket and the stdio worker use the same frames. The HTTP routes send the same `harness.*` frames as SSE `data`. + +### HTTP routes + +These routes are enough for a full client. [Connect clients to a harness](./connect#serve-the-session-over-http) lists the others (media, sessions, and more). + +- `GET events?threadId=&v=&from=`: the session stream as SSE. Each SSE `data` is one frame. On a host with `access`, a thread that has no session yet answers `404`, so send the first input before you open it. +- `POST control`: the body is `{ threadId, input }`. The answer is the receipt as JSON. +- `GET snapshot?threadId=` and `GET transcript?threadId=`: the state and the saved messages of the thread. When the client starts, load both. +- `GET capabilities`: the AG-UI capabilities. `custom.tanstack.protocol` is the protocol version of the server. +- `GET host-events?v=`: the status changes and session list changes that the user may read, as SSE. Its `harness.hello` frame has no `threadId`. +- `POST run` and `GET run`: standard AG-UI for `useChat`. These routes send plain AG-UI events, with no `harness.*` frames around them. + +`POST control` answers with these statuses: + +- `200`: the receipt, `{ inputId, status, operationId?, reason? }`. +- `400`: `{ error }`. The input has a bad shape, or `threadId` is missing. +- `401`: `getUser` returned `null`. +- `403`: `{ error: 'forbidden' }`. The host refused the action. + +A `rejected` receipt is still `200`. Its `reason` says why, for example `not_exposed`. + +### The WebSocket + +The first frame from the client must be `harness.subscribe`. The host reads the user from the upgrade request, with `getUser`. A browser cannot set headers on a WebSocket, so `getUser` must read a cookie or a token in the URL. + +```ts group=harness-protocol-socket-client +import type { ControlFrame, HostFrame } from '@tanstack/ai-harness/server' + +function isFrame(value: unknown): value is HostFrame { + return ( + typeof value === 'object' && + value !== null && + 'type' in value && + typeof value.type === 'string' + ) +} + +const socket = new WebSocket('wss://example.com/api/harness/socket') +const send = (frame: ControlFrame) => socket.send(JSON.stringify(frame)) + +socket.addEventListener('open', () => { + send({ type: 'harness.subscribe', threadId: 'thread-1', v: 2 }) +}) + +socket.addEventListener('message', (message) => { + const frame: unknown = JSON.parse(String(message.data)) + if (!isFrame(frame)) return + if (frame.type === 'harness.hello') { + send({ + type: 'harness.input', + requestId: 'req-1', + input: { op: 'prompt', message: 'Hello' }, + }) + } + if (frame.type === 'harness.receipt') console.log(frame.requestId, frame.status) + if (frame.type === 'harness.event') console.log(frame.cursor, frame.event.type) + if (frame.type === 'harness.error') console.warn(frame.code, frame.message) +}) +``` + +The socket closes with these codes: + +| Code | Why | What the client does | +| --- | --- | --- | +| `4401` | `getUser` refused the user. | Sign in again. | +| `4403` | The user cannot read the thread, or lost access. | Stop. | +| `4400` | The server does not speak the `v` of the subscribe. | Stop. Use a client of the version of the server. | +| `4408` | The client read too slowly. See [Stream values](#stream-values). | Subscribe again from the last cursor. | +| `1011` | The event feed of the thread ended. | Subscribe again from the last cursor. | + +### The stdio worker + +`runHarnessWorker` reads frames from stdin and writes frames to stdout, one JSON object per line. Put it in its own script: + +```ts group=harness-protocol-worker +import { defineHarness } from '@tanstack/ai-harness' +import { runHarnessWorker } from '@tanstack/ai-harness/worker' +import { openaiText } from '@tanstack/ai-openai' + +const assistant = defineHarness({ + name: 'acme/assistant', + adapter: openaiText('gpt-6.1-sol'), +}) + +await runHarnessWorker(assistant) +``` + +Start the script as a child process, then write frames to it: + +```ts group=harness-protocol-worker-client +import { spawn } from 'node:child_process' +import { createInterface } from 'node:readline' +import type { ControlFrame } from '@tanstack/ai-harness/server' + +const worker = spawn('node', ['worker.js'], { stdio: ['pipe', 'pipe', 'inherit'] }) +const send = (frame: ControlFrame) => worker.stdin.write(`${JSON.stringify(frame)}\n`) + +send({ type: 'harness.subscribe', threadId: 'thread-1', v: 2 }) +send({ type: 'harness.input', requestId: 'req-1', input: { op: 'prompt', message: 'Hello' } }) + +for await (const line of createInterface({ input: worker.stdout })) { + console.log(line) +} +``` + +- The worker runs the frames in order, so an input right after the subscribe is safe. +- The worker has no login. Every input runs with no user. +- The worker sends no heartbeat. +- When stdin closes, the worker exits. It also exits after an `unsupported_version` error. + +To ship a harness as a worker with a checked bundle, see [Deploy a harness](./deploy). + +## Frames + +Each frame is one JSON object with a `type`. Ignore a frame with a `type` that you do not know. A later version of the server can add frames. + +From the client to the host (socket and stdio worker): + +| `type` | Fields | What it does | +| --- | --- | --- | +| `harness.subscribe` | `threadId`, `from?`, `v?` | The first frame. Starts the event stream after the cursor `from`. | +| `harness.input` | `requestId`, `input`, `traceparent?` | Sends one input. `input` is the same object as the `input` of `POST control`. | +| `harness.snapshot` | none | Asks for the snapshot of the session. | + +`input` is one of the ops of [Send inputs to a session](./inputs), for example `{ op: 'prompt', message }` or `{ op: 'cancel', operationId }`. Pick a new `requestId` for each input. The receipt and any error frame repeat it. + +From the host to the client: + +| `type` | Fields | When | +| --- | --- | --- | +| `harness.hello` | `v`, `threadId` | First, after a subscribe that the host accepts. On SSE, the first event of `GET events`. | +| `harness.receipt` | `requestId`, `status`, `inputId?`, `operationId?`, `reason?` | The answer to one `harness.input`. | +| `harness.event` | `cursor`, `threadId`, `operationId`, `event` | One event of the session. | +| `harness.snapshot` | `snapshot` | The answer to a `harness.snapshot` frame. | +| `harness.error` | `message`, `code`, `requestId` | Something failed. See [Errors](#errors). | +| `harness.ping` | none | A heartbeat on the socket. Ignore it. | + +### The event frame + +- `event` is an AG-UI event in its wire form. It has only the keys of the AG-UI spec. +- `operationId` is the run that made the event. It is also the AG-UI `runId`. +- `threadId` is the thread that keeps the event. On the stream of a thread, it is that thread. +- The AG-UI `RUN_STARTED` event of a child run has `parentRunId`: the run that started it. A child run is an agent run that agent code started, or that a tool of a turn started. A top-level run has no `parentRunId`. +- An event of a subagent also has `subagentRunId` on the `event` itself. + +The stream of a thread has the events of every run of its tree, in one order. To rebuild the tree, keep the `parentRunId` of the `RUN_STARTED` of each `operationId`. For example, a lead agent that runs two helpers in a group gives these frames (the `event` is cut): + +```text +{"type":"harness.event","cursor":"9","threadId":"thread-1","operationId":"op-lead","event":{"type":"RUN_STARTED","runId":"op-lead",...}} +{"type":"harness.event","cursor":"12","threadId":"thread-1","operationId":"op-a","event":{"type":"RUN_STARTED","runId":"op-a","parentRunId":"op-lead",...}} +{"type":"harness.event","cursor":"13","threadId":"thread-1","operationId":"op-b","event":{"type":"RUN_STARTED","runId":"op-b","parentRunId":"op-lead",...}} +``` + +`op-a` and `op-b` are children of `op-lead`. + +The status of a receipt is `accepted`, `queued`, or `rejected`. A `rejected` receipt has a `reason`, for example `not_exposed`, `not_running`, or `unknown_agent`. + +Give each input an `inputId`, and send the same id again after a lost answer. A `prompt`, `command`, `agent`, or `append` input with the same `inputId` runs once. The `inputId` of its receipt is the id that you sent. + +The socket handles the frames of one connection in the order they came. + +## Cursors and reconnects + +Each `harness.event` has a `cursor`. On SSE, the `id` of each event is the same cursor. A cursor is an opaque string. Store the last one, and do not parse it. + +To continue after a disconnect, connect again with the last cursor: + +- On SSE, send it as the `Last-Event-ID` header, or as `?from=`. When you send both, the header wins. +- On the socket and the stdio worker, send it as `from` in `harness.subscribe`. + +```bash +curl -N -H 'authorization: Bearer my-token' -H 'Last-Event-ID: 4' \ + 'https://example.com/api/harness/events?threadId=thread-1&v=2' +``` + +The stream starts with the first event after that cursor. A disconnect does not stop a running turn. To stop a turn, send a `cancel` input. + +A cursor works on every server and after a restart only on a durable host (`stores.log`). See [Cursors](./connect#cursors). + +### When events are missing + +Sometimes the server no longer has the events after your cursor. A truncate of the log removes old records, for example. Then the stream sends one `CUSTOM` event with the name `harness.gap`, and continues from the first event that it has: + +```json +{ + "type": "harness.event", + "cursor": "120", + "threadId": "thread-1", + "operationId": "session", + "event": { + "type": "CUSTOM", + "name": "harness.gap", + "value": { "from": "12", "resumeAt": "120", "reason": "truncated" } + } +} +``` + +- `from` is the cursor that you sent. +- `resumeAt` is the cursor of this event. The stream continues after it. +- `reason` is `truncated` (a truncate of the log) or `evicted` (the server memory dropped old events). + +When you get `harness.gap`, do these steps: + +1. Delete the state that you built from events. +2. Load `GET snapshot` and `GET transcript` again. On the socket, send `harness.snapshot`. +3. Store `resumeAt` as your cursor, and continue to read the stream. + +A [session view](./custom-ui) does these steps for you. + +## Protocol version + +The protocol version is `2`. + +- On SSE, send `?v=2` on `GET events`. +- On the socket and the stdio worker, send `v: 2` in `harness.subscribe`. +- With no `v`, the server uses its current version. + +The `harness.hello` frame gives the version that the server uses. If the server does not speak your `v`, it sends one error frame and closes the stream: + +```json +{ + "type": "harness.error", + "message": "Protocol version 3 is not supported. This host speaks version 2.", + "code": "unsupported_version", + "requestId": "req-1" +} +``` + +The server speaks one version. Use a client of that version. + +## Errors + +A `harness.error` frame has a `code` and a `requestId`. `message` is for people, so read `code` in your code. + +| `code` | Why | Does the stream close? | +| --- | --- | --- | +| `unauthorized` | `getUser` refused the user. | Yes. | +| `forbidden` | The user cannot read the thread, or lost access. On the socket, also when the host refuses one input. | Yes. When the host refuses one input, no. | +| `invalid_frame` | The frame is not valid JSON, has an unknown `type` or a bad `input`, comes before `harness.subscribe`, or is a second `harness.subscribe`. | No. | +| `unsupported_version` | The server does not speak the `v` that you sent. | Yes. | +| `slow_consumer` | The client read too slowly. | Yes. | +| `internal` | The server failed to read the events, or failed to apply one input. | When the read failed, yes: connect again from the last cursor. When one input failed, no. | + +On HTTP, a refused login or action before the stream starts is the `401` or `403` status of the request, with no frame. + +`createHarnessClient` reads these frames for you. After `slow_consumer`, `events()` and `hostEvents()` connect again. Any other code ends the read with a `HarnessProtocolError`, which has the `code` and the `message`. + +The `requestId` connects the error to its request: + +- On the socket and the stdio worker, it is the `requestId` of the frame that failed. A frame with no `requestId` gets an id from the server. On the socket, that id is the id of the upgrade request. +- On HTTP, the id is your `X-Request-Id` header when it is a short safe id: 1 to 128 letters, digits, `-`, `_`, `.`, or `:`. Else the server makes a new UUID. +- Every HTTP answer of the handler sends the id back in the `X-Request-Id` header. + +Log the `requestId` with your errors. Then you can find the request in the server logs. + +## Stream values + +A long stream passes through proxies and slow networks. The server keeps it open and protects itself with these values: + +| Value | Option | Default | What happens | +| --- | --- | --- | --- | +| Heartbeat | `heartbeatMs` | 20 s | SSE gets a `: ping` comment. The socket gets `{"type":"harness.ping"}`. | +| SSE queue | `sseQueueBytes` | 1 MiB | The server stops reading the log while this much data waits for the client. | +| Socket buffer | `socketBufferBytes` | 8 MiB | Above this `bufferedAmount`, the server stops sending. It checks again every 50 ms. | +| Slow close | `slowConsumerMs` | 30 s | If the client does not read for this long, the server sends `slow_consumer` and closes. | + +- `POST run` and `GET run` also get the `: ping` comment. SSE parsers skip comments. +- A slow client loses no events. The server reads them from the log when the client reads again. +- Set the idle timeout of your proxy to more than the heartbeat. +- A client can use the heartbeat to find a dead connection. `createHarnessClient` connects again when no bytes come for 60 s (its `idleTimeoutMs`). If you set `heartbeatMs` to 60 s or more, set a larger `idleTimeoutMs` on the client. +- The stdio worker has no heartbeat and does not use these values. + +To change a value, pass `stream` to the handler and to the socket. A value that you do not set keeps its default: + +```ts group=harness-protocol-server +// A proxy that closes idle streams after 15 s, and mobile clients that read slowly. +const stream = { heartbeatMs: 10_000, slowConsumerMs: 60_000 } + +export const slowNetworkHandler = createHarnessHandler({ host, harness: assistant, stream }) + +export function onSlowNetworkUpgrade(socket: WebSocketLike, request: Request) { + handleHarnessSocket({ host, harness: assistant, socket, request, stream }) +} +``` + +Each value must be a finite number above 0. If a value is not, `createHarnessHandler` and `handleHarnessSocket` throw at the start, and the error names the field. + +## Traces + +To join a turn to the trace of your app, send a W3C `traceparent`: + +- On HTTP, send the `traceparent` header on `POST control` or `POST run`. +- On the socket and the stdio worker, set `traceparent` on the `harness.input` frame. + +```ts +const response = await fetch('/api/harness/control', { + method: 'POST', + headers: { + 'content-type': 'application/json', + traceparent: '00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01', + }, + body: JSON.stringify({ + threadId: 'thread-1', + input: { op: 'prompt', message: 'Hello' }, + }), +}) +console.log(response.headers.get('X-Request-Id')) +``` + +What the harness does with it: + +- It reads the `traceparent` of a message input only: `prompt`, `steer`, `followUp`, and the turn of `POST run`. Other inputs ignore it. +- It drops a value that is not a valid W3C `traceparent`. +- It stores the value on the `harness.input` record of the log. When another server owns the thread, the value goes to that server with the input. +- When the input starts a turn, the server that runs the turn gives its `harness.operation` span a link to the value. A link is not a parent, so your trace UI shows the turn as a linked trace. + +To give the host a tracer, see [Trace runs and report operations](./deploy#trace-runs-and-report-operations). + +## Event catalog + +A harness adds its own events to the stream as AG-UI `CUSTOM` events. `name` is the event name, and `value` has the fields below. Ignore a name that you do not know. A later version can add names. + +In TypeScript, `HarnessEventValues` from `@tanstack/ai-harness` maps each name to its value type, and `HARNESS_EVENTS` has the names. `harness.plugin.warning` is the only name that is in neither. + +### Runs + +| `name` | `value` | +| --- | --- | +| `harness.operation.started` | `{ operationId, kind, agent? }`. `kind` is `chat`, `agent`, `command`, or `compact`. | +| `harness.operation.finished` | `{ operationId, status }`. `status` is `completed`, `failed`, `cancelled`, or `interrupted`. | +| `harness.operation.resumed` | `{ operationId, resumedFrom? }`. A host continued a run after another host stopped. | +| `harness.turn.retry` | `{ operationId, retries, error: { message, code?, retryAfterMs? }, continued }`. `continued` is true when the model continues a partial answer. | +| `harness.handoff` | `{ from, to, note?, runId, by, waitId? }`. `from` and `to` are an agent name, `null` for the main model, or `{ human: true }`. `by` is `tool`, `code`, `router`, or `operator`. A handoff to a person has the `waitId` of its wait. | +| `harness.usage` | `{ model, sender?, source?, label?, usage, total }`. `usage` is this model call, and `total` is the new total of the thread. `source` is `plugin:`, `router`, or `transfer`. A turn or an agent run has no `source`. | + +### Inputs + +| `name` | `value` | +| --- | --- | +| `harness.input.accepted` | `{ inputId, op }` | +| `harness.input.queued` | `{ inputId, delivery }`. The owner of the log stored the input, and it waits behind the running turn or the turns before it. `delivery` is `steer` or `queue`. | +| `harness.input.applied` | `{ inputId, operationId }` | +| `harness.input.rejected` | `{ inputId, reason }` | +| `harness.input.settled` | `{ inputId, operationId?, outcome, error? }`. `outcome` is `completed`, `aborted`, `interrupted`, or `failed`. Only `failed` has `error: { message, code? }`. | +| `harness.input.delivery` | `{ inputId, delivery }`. A `setDelivery` input moved a waiting input. | + +### Waits for a person + +| `name` | `value` | +| --- | --- | +| `harness.wait.opened` | `{ waitId, kind, runId, message?, expiresAt? }`. A wait opened in the log: a question, a `step.ask`, a handoff to a person, or a wait of a background agent run. `kind` is `approval`, `question`, `sign-in`, `client-tool`, or `human`. `expiresAt` is in epoch milliseconds. The interrupts of a turn come in its `RUN_FINISHED`. | +| `harness.wait.settled` | `{ waitId, outcome }`. `outcome` is `answered`, `cancelled`, or `expired`. | +| `harness.question` | `{ questionId, message, schema?, secret?, url? }`. Answer it with the `answer` input. | +| `harness.question.answered` | `{ questionId }` | +| `harness.auth_required` | `{ connector, url?, userCode? }`. A tool needs a sign-in. | + +### Session changes + +When you get one of these events, load the part of the session that it names again. + +| `name` | `value` | +| --- | --- | +| `harness.config.changed` | `{ key, value }` | +| `harness.settings.changed` | `{ settings }` | +| `harness.commands.changed` | `{}`. Load `GET describe` again. | +| `harness.reloaded` | `{ error? }`. The plugins started again. Load `GET describe` again. | +| `harness.reset` | `{ inputId, note? }`. The next turn starts a fresh model context. | +| `harness.revert` | `{ messageId }`. `messageId` is `null` when no revert stands. Load `GET transcript` again. | + +### Files, plugins, and hosts + +| `name` | `value` | +| --- | --- | +| `harness.media` | A media record: `{ id, threadId, kind, mimeType, name, size, createdAt, source }`. See [Send and show media](./media). | +| `harness.plugin.event` | `{ plugin, name, value }`. An event that a plugin sent. | +| `harness.plugin.warning` | `{ plugin, message }`. A plugin hook, a listener, or a write that no one waits for failed. The work goes on. | +| `harness.ownership.changed` | `{ epoch, reason }`. A new host owns the log, from its first write. `reason` is `taken_over` when the ownership of the old owner expired with no release (it stopped), else `acquired`. | +| `harness.stranded` | `{ operationId?, hostId? }`. The workspace that the log binds the thread to is gone, so the workspace tools fail. `operationId` is the run that tried it. `hostId` is the host of a folder. | +| `harness.transferred` | `{ transferId, from?, to? }`. `from` and `to` are `{ threadId }`. The conversation came from, or went to, another thread. | +| `harness.gap` | `{ from, resumeAt, reason }`. See [When events are missing](#when-events-are-missing). | + +## What you have now + +- A client that reads the session stream over SSE, a WebSocket, or a stdio worker. +- Inputs with a receipt for each one, and errors that name their request. +- Reconnects that continue from the last cursor, and a reload when events are missing. +- A stream that stays open through proxies. +- The name and the value of every `harness.*` event. + +Next: send your first prompt with the two `curl` commands at the top of this page, and watch the events arrive. diff --git a/docs/harness/provider-keys.md b/docs/harness/provider-keys.md index 399851f599..40324eacf8 100644 --- a/docs/harness/provider-keys.md +++ b/docs/harness/provider-keys.md @@ -37,7 +37,7 @@ const claude = keyedAdapter(anthropicByok, (key) => - Each provider package exports one from its `/byok` subpath, for example `anthropicByok`, `grokByok`, `falByok`, and `openrouterByok`. - `keyedAdapter` works for every adapter kind: text, image, speech, audio, and video. -Before each turn, the session calls your function with the key of the user of that session. +Before each turn, the session calls your function with the key of the user who sent that turn. ## 2. Add the providerKeys plugin @@ -72,7 +72,7 @@ export const assistant = defineHarness({ const host = createHarnessHost({ persistence: memoryPersistence() }) const session = await host.open(assistant, { threadId: 'ada-chat', - principal: { id: 'ada' }, + user: { id: 'ada' }, }) ``` @@ -88,15 +88,16 @@ const session = await host.open(assistant, { ```ts group=harness-provider-keys const stopAsking = session.onEvent('CUSTOM', async ({ event }) => { if (event.name !== 'harness.question') return - const [question] = session.snapshot().pendingQuestions + const [question] = session.snapshot().pendingWaits // question.secret is true. Read the key without showing it. - if (question) await session.answer({ questionId: question.questionId, value: 'sk-proj-...' }) + if (question) await session.answer({ questionId: question.waitId, value: 'sk-proj-...' }) }) -console.log(await session.command('connect:openai')) // Connected to OpenAI (key ...abcd). +console.log(await session.runCommand({ name: 'connect:openai' })) // Connected to OpenAI (key ...abcd). stopAsking() ``` +- The command asks with `step.ask`, so the log keeps the question as a [wait](./waits), and the user can answer it on any server. The log never keeps the key. - With `keyUrl`, the page to make a key opens first. The session sends a `harness.auth_required` event with the URL, and your app opens it. - `/disconnect:openai` deletes the key. - `/keys` shows where each key comes from: @@ -109,7 +110,7 @@ fal.ai: missing. Run /connect fal. OpenRouter: missing. Run /connect openrouter. ``` -In your own UI, read the same list from `session.snapshot().plugins['tanstack/provider-keys'].providers`. Each entry has an `id`, a `label`, and a `state`: `connected`, `env`, or `missing`. +In your own UI, read the providers from `session.snapshot().plugins['tanstack/provider-keys'].providers`. Each entry has an `id`, a `label`, and a `state`: `env` or `missing`. Every member of a thread reads this list, so it shows only the env vars that the session reads, not the keys of one user. A host with access reads no env vars without `allowServerEnvKeys: true`. To show the keys of the user, run `/keys`. ### Sign in with OpenRouter @@ -137,10 +138,10 @@ After `/connect:openai`, send the message again. ## Where the keys are stored -`/connect:` saves the key in the credential store of the host, as an `api_key` credential with the provider id. The store keeps it for the principal of the session: +`/connect:` saves the key in the credential store of the host, as an `api_key` credential with the provider id. The store keeps it for the user who runs the command: - CLI: pass the store to `runCli(harness, { credentials })`. -- Hosted app: your database or your secrets manager. Each signed-in user is a principal, so each user has their own keys. Pass the store to `createHarnessHost({ credentials })`. +- Hosted app: your database or your secrets manager. Each signed-in person is a user, so each user has their own keys. Pass the store to `createHarnessHost({ credentials })`. - No `credentials`: the keys stay in memory until the process stops. To write a store, see [Write a store for production](../advanced/server-keys#write-a-store-for-production). Your own routes can read the same keys with `storedProviderKeys`. @@ -153,9 +154,11 @@ The key that a user pastes can be a Claude OAuth token (`sk-ant-oat...`). `creat If the user has no saved key, the session reads the env var of the provider, for example `OPENAI_API_KEY`. A saved key wins over the env var. So you can develop with a `.env` file, and your users connect in the app. +On a host with [`access`](./access), the session does not read the env vars. Each user must connect their own key, so no user spends the key of your server. To let users fall back to the env var, set `allowServerEnvKeys: true` in `access`. + ## Use the key of the user in an agent -Your agent makes images with fal, so it needs the fal key of the user. Every agent gets `ctx.keys`. In a harness, `ctx.keys` reads the same keys as the turn: +Your agent makes images with fal, so it needs the fal key of the user. Every agent gets `ctx.keys`. In a harness, `ctx.keys` reads the keys of the user who started the agent: ```ts group=harness-provider-keys import { defineAgent } from '@tanstack/ai' @@ -182,7 +185,7 @@ Put the agent in `subagents.agents` of the harness, and the model calls it as a - `ctx.keys.get(provider)`: the key, or `null`. - `ctx.keys.require(provider)`: the key. If it is missing, the run stops with `harness.auth_required`, the same as a turn. - The children of the agent get the same keys. -- Plugins get the same keys as `ctx.keys` in `setup`. +- In a plugin, a tool reads the keys of its run as `ctx.context.run.keys`, and a command reads `run.keys`. See [Act as the user of a run](./plugins#act-as-the-user-of-a-run). Outside a harness, `ctx.keys` reads the env var. See [Keys inside agents](../advanced/byok#keys-inside-agents). @@ -190,12 +193,13 @@ Outside a harness, `ctx.keys` reads the env var. See [Keys inside agents](../adv - A key shows masked only: `...` and the last 4 characters. - The plugin never puts a key in an event, the transcript, or a command result. -- The answer to the key question does not go into the inbox of the session. -- Do not log a key. For error messages, use `maskKey` from `@tanstack/ai/byok`. +- The session replaces each key that it reads with `[redacted]` in the log, the events, the model context, and its errors. See [Keep tokens out of the log and the model](./auth#keep-tokens-out-of-the-log-and-the-model). +- The session log keeps the answer to the key question as `[secret]`. If another host owns the thread, the answer goes to it through the credential store. See [Secret answers on another host](./durable-sessions#secret-answers-on-another-host). +- Do not log a key. For error messages, use `maskKey` or `scrubSecrets` from `@tanstack/ai/byok`. ## What you have now - `/connect:` for OpenAI, Anthropic, xAI, fal, and OpenRouter in the harness that you ship, with no `.env` file. - One key for each user, in your credential store. -- The env var as a fallback while you develop. -- Agents and plugins that use the key of the user through `ctx.keys`. +- The env var as a fallback while you develop, and no fallback on a host with access unless you set `allowServerEnvKeys`. +- Agents and plugins that use the key of the user of each run. diff --git a/docs/harness/session-log.md b/docs/harness/session-log.md index 83ace6f17c..605651f659 100644 --- a/docs/harness/session-log.md +++ b/docs/harness/session-log.md @@ -16,34 +16,61 @@ Your framework keeps its own state next to the conversation: signals for the mod ## Append your own records -`session.append` adds records to the log of a [durable host](./durable-sessions), in one batch after the events that wait: +`session.append` adds records to the [session log](./durable-sessions), in one batch after the events that wait: ```ts group=harness-session-log import { createHarnessHost, defineHarness } from '@tanstack/ai-harness' -import { memoryLogStore, memoryPersistence } from '@tanstack/ai-persistence' +import { memoryPersistence } from '@tanstack/ai-persistence' import { openaiText } from '@tanstack/ai-openai' -const log = memoryLogStore() +const persistence = memoryPersistence() +const { log } = persistence.stores const assistant = defineHarness({ name: 'acme/assistant', adapter: openaiText('gpt-6.1-sol'), }) -const host = createHarnessHost({ - persistence: { stores: { log, runs: memoryPersistence().stores.runs } }, -}) +const host = createHarnessHost({ persistence }) const session = await host.open(assistant, { threadId: 'thread-1' }) -await session.append([{ type: 'app.signal', text: 'The build failed.' }]) +await session.appendRecords({ records: [{ type: 'app.signal', text: 'The build failed.' }] }) ``` -- A record is JSON with a `type`. Types that start with `harness.` belong to the harness, so `append` refuses them. -- `append` resolves when the records are in the log. -- To share one log between a parent and its child sessions, see [Share one log between sessions](./shared-logs). +- A record is JSON with a `type`. Types that start with `harness.` belong to the harness, so `appendRecords` refuses them. +- `appendRecords` resolves with a receipt when the records are in the log. When another host owns the thread, the receipt is `queued`, and the owner writes the records. See [One host drives a thread](./durable-sessions#one-host-drives-a-thread). +- Pass `inputId` to make the append safe to send again: `session.appendRecords({ records, inputId: 'signal-7' })`. A second append with the same id writes nothing. Without `inputId`, the append gets a new id. The receipt has the id, as the log keeps it. `user` sets who appends. The default is the user who opened the session. +- A record can belong to a child thread of the same log. See [One log for a session tree](./shared-logs). - In a tool, use `append` from [`durableTool`](./durable-tools), so the records land with the tool batch. +- A key or a token of the session in a record goes into the log as `[redacted]`. See [Keep tokens out of the log and the model](./auth#keep-tokens-out-of-the-log-and-the-model). + +### Keep records after a truncate + +With [`keepLogRecords: 'since-snapshot'`](./lifecycle#drop-the-records-before-a-snapshot), the store drops the records before each snapshot. A plugin can make the snapshot keep the last records of its own types with `keepRecords`, and read them back with `ctx.session.keptRecords(type)`: + +```ts group=harness-session-log +import { defineCommand, definePlugin } from '@tanstack/ai-harness' + +export const steps = definePlugin({ + name: 'acme/steps', + keepRecords: { 'acme/steps:step': 100 }, + setup: (ctx) => ({ + commands: { + steps: defineCommand({ + description: 'Count the kept steps', + run: async () => (await ctx.session.keptRecords('acme/steps:step')).length, + }), + }, + }), +}) +``` + +- The snapshot keeps the last 100 `acme/steps:step` records of each thread. +- `keptRecords` gives the kept records of that type of this thread, oldest first. It works before and after a truncate. +- [`snapshots()`](./snapshots#go-back-to-an-earlier-message) keeps its last 500 file steps this way, so `session.revert()` works after a truncate. +- The plugins of the harness of the root thread decide, because that harness folds the log. A type in two plugins keeps the larger count. ## Fold records into the model context -By default, the model does not see your records. Give the host a `project` function that folds a record into the messages. It gets one object, `{ messages, record }`, and returns the new list, or `undefined` for no change: +By default, the model does not see your records. Set `durability.addRecordsToModelContext` on the harness to fold a record into the messages. Its `record` function gets one object, `{ messages, record }`, and returns the new list, or `undefined` for no change: ```ts group=harness-session-log import type { ProjectOptions } from '@tanstack/ai-harness' @@ -58,43 +85,55 @@ const project: ProjectOptions = { }, } -const projectedHost = createHarnessHost({ - persistence: { stores: { log, runs: memoryPersistence().stores.runs } }, - project, +export const projected = defineHarness({ + name: 'acme/projected', + adapter: openaiText('gpt-6.1-sol'), + durability: { addRecordsToModelContext: project }, }) ``` - `record` must be pure: the same log always folds to the same context, live and after a restart. - The model sees the change at its next model call. A message that arrives during a tool call comes after the tool result. -- Change `version` when you change `record`. A fold checkpoint with another version is ignored. +- Change `version` when you change `record`. A log snapshot with another version is not used, so the host folds the log again from the first record. - For compaction, use `projectCompaction` from `@tanstack/ai-compaction`. See [Compact a harness session](./compaction). -- A chat middleware on a durable host can append records itself with `getLogRecords(ctx).append([...])` from `@tanstack/ai`. The records land in the log at once. A plugin's middleware reads the writer with `getLogRecords(ctx, { optional: true })`. A plugin cannot list `LogRecordsCapability` in `requires`. +- A chat middleware in a harness turn can append records itself with `getLogRecords(ctx).append([...])` from `@tanstack/ai`. The records land in the log at once. A plugin's middleware reads the writer with `getLogRecords(ctx, { optional: true })`. A plugin cannot list `LogRecordsCapability` in `requires`. ## Let a client add records -A webhook or a browser knows something that the model needs: the build failed, or a payment arrived. Let it add a record to the thread, and the next turn reads the record through `project`. +A webhook or a browser knows something that the model needs: the build failed, or a payment arrived. Let it add a record to the thread, and the next turn reads the record through `durability.addRecordsToModelContext`. -1. On the server, list the record types in `expose.records`. Serve the harness from the host that has `project`: +1. On the server, list the record types in `expose.records`, and set `durability.addRecordsToModelContext` on the same harness: ```ts group=harness-session-log -import { createHarnessHandler } from '@tanstack/ai-harness' +import { createHarnessHandler } from '@tanstack/ai-harness/server' export const watcher = defineHarness({ name: 'acme/build-watcher', adapter: openaiText('gpt-6.1-sol'), expose: { records: ['app.signal'] }, + durability: { addRecordsToModelContext: project }, +}) + +const watcherHost = createHarnessHost({ + persistence: memoryPersistence(), + access: { + getUser: ({ request }) => + request.headers.get('authorization') === `Bearer ${process.env.HARNESS_TOKEN}` + ? { id: 'ci-bot' } + : null, + // The CI bot is the only user, so it can use every thread. + rules: { isAllowed: ({ user }) => user.id === 'ci-bot' }, + }, }) +// Every instance signs media URLs with the same secret. +const mediaSecret = process.env.HARNESS_MEDIA_SECRET +if (!mediaSecret) throw new Error('Set HARNESS_MEDIA_SECRET.') + export const handler = createHarnessHandler({ - host: projectedHost, + host: watcherHost, harness: watcher, - authorize: (request) => - request.headers.get('authorization') === `Bearer ${process.env.HARNESS_TOKEN}` - ? { id: 'ci-bot' } - : null, - canAccess: () => true, - // A durable host signs media URLs with the same secret on every instance. - mediaSecret: process.env.HARNESS_MEDIA_SECRET, + media: { secret: mediaSecret }, }) ``` @@ -110,26 +149,40 @@ const client = createHarnessClient({ headers: { authorization: 'Bearer my-token' }, }) -await client.append([{ type: 'app.signal', text: 'The build failed.' }]) -await client.continue() +await client.appendRecords({ records: [{ type: 'app.signal', text: 'The build failed.' }] }) +await client.continueWithoutMessage() ``` - A record of another type is refused with `not_exposed`. So is a record with a `thread` field: a client writes only to its own thread. - Types that start with `harness.` belong to the harness. `defineHarness` throws if `expose.records` lists one. -- The host must be durable. On a host with no log, the receipt is `rejected`. - An MCP client adds records with the `append` tool. See [Use a harness from any MCP client](./mcp-server#3-use-the-tools). ## Read the log -The store is yours, so read your records back from it, for example to rebuild your state when a session opens: +The store is yours, so read your records back from it, for example to rebuild your state when a session opens. The store keeps each log under an id with its tenant, so build the id with `logIdOf`: ```ts group=harness-session-log -const entries = await log.read('thread-1') +import { logIdOf } from '@tanstack/ai-harness' + +// The default tenant. For a tenant, pass `tenantId` too: 'acme:thread-1'. +const entries = await log.read({ logId: logIdOf({ logId: 'thread-1' }) }) const signals = entries.filter((entry) => entry.record.type === 'app.signal') console.log(signals.map((entry) => entry.seq)) ``` -A reader outside a session that needs a `MessageStore`, for example `reconstructChat`, uses `logMessageStore`. It folds the log with the same `project`: +Each `harness.input` record has the `user` of its sender: its `id` and `tenantId`. Every kind of input has it when the sender is known, also a cancel, an answer, a delivery change, and a change of settings. + +Two harness records tell who answers a thread and where a conversation went. See [Hand off a conversation](./handoffs). + +- `harness.handoff`: another agent answers from now on. It has `from` and `to` (an agent name, `null` for the main model, or `{ human: true }` for a person), the `note`, the `runId`, and `by` (`'tool'`, `'code'`, `'router'`, or `'operator'`). A handoff to a person also has the `waitId` of its `human` wait. The last one is the active agent of the thread. +- `harness.transferred`: `host.transfer` moved the conversation. The log of the target has `from`, and the log of the source has `to`. Each is `{ harness, threadId }`. Both records have the same `transferId`, the `user`, and `at`. + +```ts group=harness-session-log +const handoffs = entries.filter((entry) => entry.record.type === 'harness.handoff') +console.log(handoffs.at(-1)?.record.to) +``` + +A reader outside a session that needs a `MessageStore`, for example `reconstructChat`, uses `logMessageStore`. It folds the log with the same `addRecordsToModelContext`. Pass `tenantId` to read the threads of a tenant: ```ts group=harness-session-log import { logMessageStore } from '@tanstack/ai-harness' @@ -138,14 +191,35 @@ const messages = logMessageStore({ store: log, project }) console.log(await messages.loadThread('thread-1')) ``` +## Find the runs in the log + +Your dashboard shows every agent run of a thread, also the runs that ended long ago. The log has one `harness.run.node` record for each run that the session admits: + +```ts group=harness-session-log +const records = (await log.read({ logId: logIdOf({ logId: 'thread-1' }) })).map( + (entry) => entry.record, +) +const agentNodes = records.filter( + (record) => record.type === 'harness.run.node' && record.kind === 'agent', +) +console.log(agentNodes.map((record) => [record.runId, record.agent, record.parentRunId])) +``` + +- A `harness.run.node` record has `runId`, `kind` (`turn`, `agent`, or `command`), and the `user` when it is known. An agent run also has `agent`, and `parentRunId`, `key`, `limits`, and `budget` when it has them. +- Read only these fields. The other fields of the record are bookkeeping of the session, and they can change. +- The tree also has `tool` runs. These come from `harness.run` records, not from `harness.run.node` records. +- The status, the result, and the error come from later records: `harness.run`, `harness.run.update`, and `harness.agent.result`. A session folds them into one tree. See [The run tree](./subagents#the-run-tree). +- Each `harness.usage` record has the `runId` of the run that made the model call. See [Usage of one run](./usage#usage-of-one-run). +- A `harness.state` record has a `key` and one change to its value. It is the shared state of [`ctx.harness.state`](./subagents#share-state-between-agents). + ## Tune how text streams into the log -A durable host merges streamed text and tool call arguments for `coalesceMs` (default 100) before it appends them. A message start or end, a tool call start or end, and your own records append at once. +A host merges streamed text and tool call arguments for `log.bufferStreamWriteMs` milliseconds (default 100) before it appends them. A message start or end, a tool call start or end, and your own records append at once. ```ts group=harness-session-log const smoothHost = createHarnessHost({ - persistence: { stores: { log, runs: memoryPersistence().stores.runs } }, - coalesceMs: 250, + persistence: memoryPersistence(), + log: { bufferStreamWriteMs: 250 }, }) ``` @@ -157,3 +231,4 @@ A larger value writes less often, and clients get text in larger parts. - A model context that folds them in, the same live and after a restart. - Records that a webhook or a browser adds, for the record types you expose. - A transcript that any reader can fold from the log. +- Every run of the thread, and the shared state of its agents, in records that you can read. diff --git a/docs/harness/sessions.md b/docs/harness/sessions.md index 5b5b014010..29a6c8701e 100644 --- a/docs/harness/sessions.md +++ b/docs/harness/sessions.md @@ -37,44 +37,50 @@ const host = createHarnessHost({ persistence: memoryPersistence() }) const session = await host.open(assistant, { threadId: 'chat-1', - principal: { id: 'ada' }, + user: { id: 'ada' }, }) await session.send('Plan a trip to Lisbon.') -console.log((await host.sessions.get('chat-1'))?.principal) // { id: 'ada' } +console.log((await host.sessions.get({ threadId: 'chat-1' }))?.members) // [{ userId: 'ada', role: 'owner' }] ``` +`host.open` gives you a handle. When you are done with it, release it with `await session.close()`. Or open it with `await using`, which releases it at the end of the block. Then the session can close when it goes idle. See [Lifecycle and retention](./lifecycle#open-and-release-a-session). + What an entry holds: -- `title`: the name of the chat. Set it with `host.sessions.rename`, or from a plugin with `ctx.session.updateEntry({ title })`. +- `title`: the name of the chat. Set it with `host.sessions.rename`, or from a plugin with `ctx.session.updateIndexEntry({ title })`. - `updatedAt`: the time of the last change. The list sorts by it. -- `principal`: the owner. This is the user who opened the thread first. -- `usage`: the token totals of the thread. -- `parentThreadId`: the parent thread of a child session, for example a subagent run. +- `members`: the users who can open the thread, each with a role. The user who opened the thread first is the member with the role `owner`. +- `tenantId`: the tenant of the thread. Without it, the thread is in the default tenant. See [Access and tenants](./access). +- `usage` and `parentThreadId`: the token totals of the thread, and the parent thread of a child session, for example a subagent run. -A durable host takes `sessions` next to `log`: +The host takes `sessions` next to `log`: ```ts group=harness-sessions import { memoryLogStore } from '@tanstack/ai-persistence' -const { runs, sessions } = memoryPersistence().stores +const { sessions } = memoryPersistence().stores const durableHost = createHarnessHost({ - persistence: { stores: { log: memoryLogStore(), runs, sessions } }, + persistence: { stores: { log: memoryLogStore(), sessions } }, }) ``` -Without `stores.sessions`, `list` is empty, `get` returns `undefined`, and `rename`, `update`, and `delete` write nothing. To write your own store, read the [`SessionIndexStore` contract](../persistence/store-reference#sessionindexstore). +- Without `stores.sessions`, `list` is empty, `get` returns `undefined`, and `rename` and `update` write nothing. +- The store needs `getVersioned` and `upsertIf`, else `createHarnessHost` throws. The host changes an entry with compare-and-set, so two hosts never lose each other's changes. +- To write your own store, read the [`SessionIndexStore` contract](../persistence/store-reference#sessionindexstore). + +Each method of `host.sessions` takes one object with a `threadId` or filters, and a `user`. The tenant of `user` picks the entries. On a host with `access`, the `user` is required, and the host checks each call. See [Access and tenants](./access). ## List the sessions of a user ```ts group=harness-sessions -const page = await host.sessions.list({ principal: { id: 'ada' }, limit: 20 }) +const page = await host.sessions.list({ user: { id: 'ada' }, limit: 20 }) for (const entry of page.entries) { console.log(entry.threadId, entry.title ?? 'New chat') } if (page.truncated) { const next = await host.sessions.list({ - principal: { id: 'ada' }, + user: { id: 'ada' }, limit: 20, cursor: page.cursor, }) @@ -85,13 +91,14 @@ if (page.truncated) { - The newest `updatedAt` comes first. - By default, the list has only top-level sessions. Pass `parentThreadId: 'chat-1'` to get the child sessions of `chat-1`. - With `limit`, a page that has more entries after it has `truncated: true` and a `cursor`. Pass the `cursor` to get the next page. -- If `principal` has a `tenantId`, the tenant of the entry must match too. +- A list reads one tenant: the tenant of `user`. Without a `tenantId`, the default tenant. +- Without host `access`, the list has every session of that tenant. With `membersOnly()`, it has only the sessions where `user` is a member. With your own `isAllowed`, it has the sessions that `isAllowed` lets `user` list. To find sessions, add filters. All filters must match, and `limit` and `cursor` page the filtered list: ```ts group=harness-sessions const found = await host.sessions.list({ - principal: { id: 'ada' }, + user: { id: 'ada' }, search: 'trip', harness: 'acme/assistant', metadata: { project: 'p-1' }, @@ -121,9 +128,10 @@ await fork.send('Plan a trip to Porto.') - `at: { through: id }` copies the messages up to and including `id`. - `at: { before: id }` copies the messages before `id`. A fork before the first message has no messages, but it keeps the settings. -- The host picks the new thread id. The new entry has the title of the old one plus ` (fork)`, and the same owner. A thread without a title gives a fork without a title. +- The host picks the new thread id. The new entry has the title of the old one plus ` (fork)`. A thread without a title gives a fork without a title. +- With a `user` and host `access`, the user who forks owns the fork. `members` picks who else is in it. See [Fork a shared thread](./shared-threads#6-fork-a-shared-thread). Without access and without `members`, the fork keeps the owner and the members of the old thread. - If the transcript has no message with that id, `fork` throws. -- If the index entry of the thread names another harness, `fork` throws. One host can run more than one harness. +- If the index entry of the thread names another harness, `fork` throws. One host can run more than one harness. To move a chat to another harness, use [`host.transfer`](./handoffs#5-move-a-conversation-to-another-harness). What a fork copies, and what it does not copy: [Fork and reset a thread](./fork-and-reset#fork-a-thread-at-a-message). @@ -135,14 +143,14 @@ A fork keeps the old chat. To go back in the same chat, revert it to a message: const [question] = await session.transcript() if (!question?.id) throw new Error('The chat has no messages.') -await session.revert(question.id) +await session.revert({ messageId: question.id }) // The transcript now ends at `question`. To bring the later messages back: await session.unrevert() ``` - While a revert stands, `transcript()` hides the messages after that message. The session keeps them. -- The next turn drops the hidden messages for good. A steer, a follow-up, and `continue` are turns too. -- The session saves the revert in the log, or in `stores.metadata` on a host without a log. A restart keeps it. +- The next turn drops the hidden messages for good. A steer, a follow-up, and `continueWithoutMessage` are turns too. +- The session saves the revert in the log. A restart keeps it. - Each change sends a `harness.revert` event. Read the transcript again when it arrives. - `revert` and `unrevert` refuse with `busy` when the session is not idle, or when another host runs the thread. `revert` refuses with `unknown_message` for an id that is not in the transcript. @@ -150,12 +158,14 @@ await session.unrevert() ```ts group=harness-sessions await host.sessions.rename({ threadId: forked.threadId, title: 'Porto trip' }) -await host.sessions.delete('chat-1') +await host.sessions.delete({ threadId: 'chat-1' }) ``` - `rename` resolves to the changed entry. For a thread with no entry, it resolves to `undefined`. -- `delete` removes the index entry only. The transcript, the log, the media, and the settings of the thread stay in their stores. -- After `delete`, the turns of the open session do not write the entry again. If the thread opens again after a restart, the host writes a new entry for it. +- `delete` closes the sessions of the thread and of its child threads, for every handle. Then it deletes their data: the log or the messages, the index entry, the media, the settings, and the usage. +- The next user who opens the same `threadId` starts empty. The host writes a new entry for the thread, with that user as the owner. + +For the full list of what `delete` removes, and what stays, see [Delete a thread for good](./lifecycle#delete-a-thread-for-good). ## Keep app data on a session @@ -192,11 +202,11 @@ await host.sessions.update({ ### Writes from several servers -A turn on one server writes the usage of the thread. At the same time, the app on another server can write the pins. Your store keeps both writes when it has `getVersioned` and `upsertIf`. Then the host runs your change function again on the new entry. `memoryPersistence()` has both. To add them to your own store, read [Keep the writes of several hosts](../persistence/store-reference#keep-the-writes-of-several-hosts). +A turn on one server writes the usage of the thread. At the same time, the app on another server can write the pins. The store keeps both writes with `getVersioned` and `upsertIf`: when another server wrote first, the host runs your change function again on the new entry. For the rules of your own store, read [Keep the writes of several hosts](../persistence/store-reference#keep-the-writes-of-several-hosts). ### Read and write the metadata in a plugin -Give the plugin the same schema. Then `ctx.session.entry()` and `ctx.session.updateEntry()` are typed, and the schema checks each write of the plugin: +Give the plugin the same schema. Then `ctx.session.indexEntry()` and `ctx.session.updateIndexEntry()` are typed, and the schema checks each write of the plugin: ```ts group=harness-sessions import { defineCommand, definePlugin } from '@tanstack/ai-harness' @@ -210,7 +220,7 @@ export const pins = definePlugin({ description: 'Unpin a file from this chat', input: z.object({ file: z.string() }), run: ({ file }) => - ctx.session.updateEntry((entry) => ({ + ctx.session.updateIndexEntry((entry) => ({ metadata: { pinnedFiles: (entry.metadata?.pinnedFiles ?? []).filter( (pinned) => pinned !== file, @@ -223,7 +233,7 @@ export const pins = definePlugin({ }) ``` -Add `pins` to the `plugins` of the harness. If the harness metadata does not fit the schema of a plugin, TypeScript shows an error. The turn hooks of the harness also get a typed `session.entry()`. +Add `pins` to the `plugins` of the harness. If the harness metadata does not fit the schema of a plugin, TypeScript shows an error. The turn hooks of the harness also get a typed `session.indexEntry()`. ## Show a live status @@ -245,8 +255,11 @@ void watch() - `status`: `running` (an operation runs), `waiting` (a question or an approval waits for an answer), or `idle`. `at` is the time of the change. - `session`: the host wrote an entry, for a new session, the end of a turn, a rename, an update, or a fork. `entry` is the new entry. - `session-deleted`: the host removed an entry. `entry` is the removed entry. +- `revoked`: `host.revoke` or `unshare` took the access of the user `userId` to `threadId`. See [Use your own access tables](./access#use-your-own-access-tables). + +Each host reports only the sessions that are open in it. For those sessions, the status also counts the turns and the questions that another host writes to the same log. A session with no status event is not open, so show it as idle. -Each host reports only the sessions that are open in it. A session with no status event is not open, so show it as idle. +A reader that does not keep up stops. When more than 1 000 changes wait for it, its loop ends. The first statuses do not count. Call `host.events()` again: the current statuses come first again. `host.close()` does not end the readers of `events()`. Abort the signal of each reader before you close the host. If you do not abort it, the loop waits forever: diff --git a/docs/harness/shared-logs.md b/docs/harness/shared-logs.md index 868f790455..7febf08be1 100644 --- a/docs/harness/shared-logs.md +++ b/docs/harness/shared-logs.md @@ -1,72 +1,69 @@ --- -title: Share one log between sessions +title: One log for a session tree id: harness-shared-logs order: 4 -description: 'Put a parent session and its child sessions in one durable log, append to several sessions at once, and keep state for the whole log.' +description: 'A root thread and its child threads (agent runs and subagents) write to one durable log. Append to several threads at once, and keep one state for the whole tree.' keywords: - tanstack ai - harness - shared log - - logId + - session tree - reduce - logState - - child sessions + - child threads --- -Your framework runs one agent instance with a parent conversation and child tasks. Each conversation in its own log means two problems. A record for the child and a record for the parent can land one without the other after a crash. And no one place holds the state of the whole instance. Open all the sessions on one log. They share the order of the log, its atomic appends, and one state fold. +Your framework runs one agent with a main conversation and child tasks. If each conversation has its own log, two things go wrong. A record for the child and a record for the parent can land one without the other after a crash. And no one place holds the state of the whole agent. A harness keeps the whole tree in one log, so the threads share its order, its atomic appends, and one state fold. -## Open two sessions on one log +## One log per session tree -Pass the same `logId` to `host.open`. Each `threadId` is one session: +Open the root thread. Each child thread that the harness starts writes to the log of that root: + +- An agent run and its follow-ups (`::`). +- A subagent run (`subagent:`). +- A session of another harness that runs as a child with [`harnessAgent`](./subagents#call-a-harness-as-a-child) (`/agent/`). ```ts group=harness-shared-logs import { createHarnessHost, defineHarness } from '@tanstack/ai-harness' -import { memoryLogStore, memoryPersistence } from '@tanstack/ai-persistence' +import { memoryPersistence } from '@tanstack/ai-persistence' import { openaiText } from '@tanstack/ai-openai' -const log = memoryLogStore() -const { runs, metadata } = memoryPersistence().stores +const persistence = memoryPersistence() const assistant = defineHarness({ name: 'acme/assistant', adapter: openaiText('gpt-6.1-sol'), }) -const host = createHarnessHost({ - persistence: { stores: { log, runs, metadata } }, -}) - -const parent = await host.open(assistant, { threadId: 'agent-1' }) -const child = await host.open(assistant, { - threadId: 'task-1', - logId: 'agent-1', -}) +const host = createHarnessHost({ persistence }) -console.log(parent.logId, child.logId) +const root = await host.open(assistant, { threadId: 'agent-1' }) ``` -- `logId` defaults to `threadId`. The parent above has the log id `agent-1`, the same as its thread id. -- Each session folds only its own transcript, inputs, and events from the log. The child never sees the messages of the parent. -- The sessions run turns at the same time. A slow parent turn does not block a child turn. -- `session.logId` tells you which log a session writes to. +- The log has the id of the root thread, in its tenant: `logIdOf({ tenantId, logId: 'agent-1' })`. +- Each thread folds only its own transcript, inputs, and events from the log. A child never sees the messages of the root. +- A child thread does not open. Open its root, and read the child with `session.agentTranscript({ subagentRunId })`. `host.open` throws for the thread id of a `harnessAgent` child. +- `host.observe` also reads a child thread by its own thread id. It finds the root through the session index, so it needs `stores.sessions`. `memoryPersistence()` has it. -## Write to several sessions in one append +## Write to several threads in one append -`session.append` stamps the records with this session. A record with `thread: ''` belongs to that session of the same log. The whole append is all or nothing: +`session.append` adds records to the root thread. A record with `thread: ''` belongs to that thread of the same log. The whole append is all or nothing: ```ts group=harness-shared-logs -await parent.append([ - { type: 'app.child_created', thread: 'task-1', task: 'research' }, - { type: 'app.child_linked', child: 'task-1' }, -]) +await root.appendRecords({ + records: [ + { type: 'app.child_created', thread: 'task-1', task: 'research' }, + { type: 'app.child_linked', child: 'task-1' }, + ], +}) ``` -Here the child record and the link in the parent land together, or not at all. +Here the child record and the link in the root land together, or not at all. -- `thread` must be a string. Any other value makes `append` throw. -- The session whose thread id equals the log id (the parent above) writes no `thread` field. A record without `thread` belongs to it. +- `thread` must be a string. Any other value makes `appendRecords` throw. +- A record without `thread` belongs to the root thread. -## Keep state for the whole log +## Keep state for the whole tree -Give the host a `reduce` option. It folds every host record of the log, from all sessions, in log order. Then `host.logState` reads the result: +Give the harness a `durability.logState` option. It folds every host record of the log, from the root and from each child thread, in log order. Then `session.logState()` reads the result: ```ts group=harness-shared-logs import type { ReduceOptions } from '@tanstack/ai-harness' @@ -84,45 +81,52 @@ const reduce: ReduceOptions = { : state, } -const trackedHost = createHarnessHost({ - persistence: { stores: { log, runs, metadata } }, - reduce, +const tracker = defineHarness({ + name: 'acme/tracker', + adapter: openaiText('gpt-6.1-sol'), + durability: { logState: reduce }, +}) +const tracked = await host.open(tracker, { threadId: 'agent-2' }) +await tracked.appendRecords({ + records: [{ type: 'app.child_created', thread: 'task-2', task: 'research' }], }) -const tracked = await trackedHost.open(assistant, { threadId: 'agent-2' }) -await tracked.append([ - { type: 'app.child_created', thread: 'task-2', task: 'research' }, -]) -console.log(trackedHost.logState('agent-2')?.tasks) +console.log((await tracked.logState())?.tasks) ``` - `record` must be pure. It returns the next state, or the same state for no change. -- `host.logState(logId)` works for a log that has an open session in this host. Otherwise it returns `undefined`. -- The type of the state flows from `initial` and `record`. Here `logState` returns `Children | undefined`. -- Change `version` when you change `record`. A fold checkpoint with another version is ignored. -- `project` still works. It folds a host record into the messages of its own session only. +- The type of the state flows from `initial` and `record`. Here `logState()` resolves to `Children | undefined`. A harness without `logState` gets `undefined`. +- The fold comes from the log, so a session on another machine gets the same state. +- Change `version` when you change `record`. A log snapshot with another version is not used, so the host folds the log again from the first record. +- `durability.addRecordsToModelContext` also works with `logState`. It folds a host record into the messages of its own thread only. +- Every harness that opens the same log must pass the same `addRecordsToModelContext` and `logState` objects. When a harness with other ones opens a log that is open, `host.open` throws, and so do the reads of `host.observe`. -## One writer per log +A read route that opens no session gets the same state from a view: + +```ts group=harness-shared-logs +const view = await host.observe(tracker, { threadId: 'agent-2' }) +console.log((await view.logState())?.tasks) +``` -Only one host writes a log at a time. When another host writes to it: +## One writer per log -1. Every session of that log in this host stops. Their next `append` throws a log conflict. -2. The next `host.open` for that log folds the log again and gives you a new session. +Only one host writes a log at a time. When another host takes the log, the sessions of that log on this host stay open and read on. Their next write takes the log back, when no other host owns it. See [One host drives a thread](./durable-sessions#one-host-drives-a-thread). -One thread can have one open session per log. A second harness on the same thread and log throws. +One root thread has one open session per host. A second harness on the same thread throws. -To read one thread of a shared log outside a session, pass `logId` to `logMessageStore`: +To read a root thread outside a session, for example for `reconstructChat`, use `logMessageStore`: ```ts group=harness-shared-logs import { logMessageStore } from '@tanstack/ai-harness' -const messages = logMessageStore({ store: log, logId: 'agent-1' }) -console.log(await messages.loadThread('task-1')) +const messages = logMessageStore({ store: persistence.stores.log }) +console.log(await messages.loadThread('agent-1')) ``` +Each thread id that you pass is the root of a tree. To read a child thread, use `host.observe`. + ## What you have now -- Many sessions in one durable log, each with its own transcript. -- Appends that reach several sessions and land all together or not at all. -- One state for the whole log, from `reduce`, that you read with `host.logState`. -- A clear rule for conflicts: one writer, and a fresh fold on the next open. +- One durable log for a root thread and all of its child threads, each with its own transcript. +- Appends that reach several threads and land all together or not at all. +- One state for the whole tree, from `durability.logState`, that you read with `session.logState()` on any machine. diff --git a/docs/harness/shared-threads.md b/docs/harness/shared-threads.md index 6da9613c88..00d43fcfc3 100644 --- a/docs/harness/shared-threads.md +++ b/docs/harness/shared-threads.md @@ -2,28 +2,26 @@ title: Share a thread between people id: harness-shared-threads order: 7 -description: "Let several people write in one thread. Each message runs as its sender, with the credentials and the provider key of that person." +description: "Let a team use one thread. Share it with members and viewers, and run each message as its sender, with the credentials and the provider key of that person." keywords: - tanstack ai - harness - shared thread - - principal + - membersOnly + - share - tenantId - canJoin --- -Several people write in one thread: a team in one chat, or a customer and a support agent. A message from Bob must run as Bob. His GitHub token and his provider key must do the work, not those of the person who opened the thread first. The harness runs each message as the person who sent it. +Several people work in one thread: a team in one chat, or a customer and a support agent. Only the people you invite can see it. A message from Bob must run as Bob: his GitHub token and his provider key do the work, not those of the person who opened the thread. `membersOnly()` gives each thread an owner and a list of members, and the harness runs each message as the person who sent it. -## 1. Return the sender from `authorize` +## 1. Turn on members -`authorize` decides who sends each request. Each message runs as the principal that it returns. If your users belong to organizations, add `tenantId`: +Use `membersOnly()` as the `rules` of the host `access`. [Access and tenants](./access) explains `getUser`: ```ts group=harness-shared-threads -import { - createHarnessHandler, - createHarnessHost, - defineHarness, -} from '@tanstack/ai-harness' +import { createHarnessHost, defineHarness, membersOnly } from '@tanstack/ai-harness' +import { createHarnessHandler } from '@tanstack/ai-harness/server' import { memoryPersistence } from '@tanstack/ai-persistence' import { openaiText } from '@tanstack/ai-openai' @@ -32,35 +30,85 @@ export const assistant = defineHarness({ adapter: openaiText('gpt-6.1-sol'), }) -const host = createHarnessHost({ persistence: memoryPersistence() }) - -// Your own lookups: a session token to a user, and a thread to its members. +// Your own login check: a session token to a user. const users = new Map([ ['Bearer token-ada', { id: 'ada', tenantId: 'acme' }], ['Bearer token-bob', { id: 'bob', tenantId: 'acme' }], ]) -const members = new Map([['team-thread', ['ada', 'bob']]]) -export const handler = createHarnessHandler({ - host, - harness: assistant, - authorize: (request) => users.get(request.headers.get('authorization') ?? '') ?? null, - canAccess: (principal, threadId) => - members.get(threadId)?.includes(principal.id) ?? false, +export const host = createHarnessHost({ + persistence: memoryPersistence(), + access: { + getUser: ({ request }) => users.get(request.headers.get('authorization') ?? '') ?? null, + rules: membersOnly(), + }, +}) + +export const handler = createHarnessHandler({ host, harness: assistant }) +``` + +The user who opens a new thread is its owner. Nobody else can open it until the owner shares it. What each role can do: + +| Action | Owner | Member | Operator | Viewer | +| --- | --- | --- | --- | --- | +| Read the thread and see it in a list | Yes | Yes | Yes | Yes | +| Send messages, run commands, start agents | Yes | Yes | No | No | +| Answer and cancel | Every run | Only the runs they started | No | No | +| Answer a `human` wait, and reply as an [operator](./handoffs#reply-as-the-operator) | No | No | Yes | No | +| Fork | Yes | Yes | No | No | +| Configure, share, delete, transfer | Yes | No | No | No | + +The user who waits for a person must not act as that person, so the owner does not get `operator`. To give it to other roles, pass `membersOnly({ operator: { roles: ['owner', 'operator'] } })`. + +A user who is not a member can do nothing. A child thread of a subagent uses the members of its root thread. + +## 2. Share the thread + +The owner adds a member from your server code: + +```ts group=harness-shared-threads +const ada = { id: 'ada', tenantId: 'acme' } + +await host.open(assistant, { threadId: 'team-thread', user: ada }) +await host.sessions.share({ + threadId: 'team-thread', + userId: 'bob', + role: 'member', + user: ada, +}) +``` + +From the browser, the owner calls `client.sessions.share`: + +```ts group=harness-shared-threads-invite +import { createHarnessClient } from '@tanstack/ai-harness/client' +import type { assistant } from './harness' + +const client = createHarnessClient({ + url: '/api/harness', + threadId: 'team-thread', + headers: { authorization: 'Bearer token-ada' }, }) + +const entry = await client.sessions.share({ threadId: 'team-thread', userId: 'bob', role: 'member' }) +console.log(entry.members) // [{ userId: 'ada', role: 'owner' }, { userId: 'bob', role: 'member' }] + +await client.sessions.unshare({ threadId: 'team-thread', userId: 'bob' }) ``` -You pass nothing else. The sender of each request is used everywhere: +- `share` adds a member, or changes the role of a member. It resolves to the changed entry. +- `unshare` removes a member: `host.sessions.unshare({ threadId, userId, user })` on the server, or `client.sessions.unshare({ threadId, userId })` from the browser. It resolves to the changed entry. +- Only the owner can share. Another user gets `AccessDeniedError`, and the handler answers `403`. +- The owner cannot be removed. -- The log keeps the sender of each message. After a restart, a message runs again as its sender. -- The tools of a turn get the credentials of the sender, in `ctx.credentials`. -- The model call uses the provider key of the sender, in `ctx.keys`. So the sender pays for the call. -- In a plugin, `ctx.session.principal` is the sender of the running turn. Outside a turn, it is the person who opened the session first. -- Commands, for example `connect:github`, run as their sender. So a sign-in belongs to the person who signed in. A turn that waits for a sign-in continues only when its own sender signs in. +When `unshare` removes a user: -`handleHarnessSocket` works the same way. Every input on the socket runs as the `principal` that you give it. Context that a client sends with a message is client data, and it never changes the sender. See [Send context with a message](./inputs#send-context-with-a-message). +- The queued and running work that the user started stops, on every host. The log keeps who cancelled it. The schedules of the user on the thread are removed. +- The streams of the user on the thread close. A socket gets a `forbidden` error frame. A running `events()` read of a handle or a view ends with `AccessDeniedError`. +- The open handles and views of the user throw `AccessDeniedError`. +- The signed media URLs of the user stop working. -## 2. Connect each person +## 3. Connect each person Each person opens the same thread with their own token. This snippet shows two clients. In a real app, each person has one client in their own browser: @@ -85,19 +133,48 @@ await bob.send('List my open pull requests.') Bob sends his message while the turn of Ada runs. His message waits. Then it runs as its own turn, as Bob. -On the server, give the sender to `send` yourself. `resolve` and `command` take `principal` too. Without it, the message runs as the person who opened the session: +On the server, open one handle for each person. A handle acts only as its user: ```ts group=harness-shared-threads -const session = await host.open(assistant, { threadId: 'team-thread' }) -const ada = { id: 'ada', tenantId: 'acme' } +const bob = { id: 'bob', tenantId: 'acme' } +const bobSession = await host.open(assistant, { threadId: 'team-thread', user: bob }) -session.send('Summarize the open issues.', { principal: ada }) -session.command('connect:github', undefined, { principal: ada }) +await bobSession.send('Summarize the open issues.') ``` -A command acts for the person who runs it. In its `run` context, `session.principal` is that person, and `session.send` runs as that person. +## 4. Each message runs as its sender + +Each message, command, and agent runs as the person who started it: + +- The log keeps the sender of each input, also of a cancel, an answer, and a change of settings. After a restart, a message runs again as its sender. +- Tools and middleware read the run as `ctx.context.run`. It has the user, the credentials, and the provider keys of the sender. +- The model call uses the provider key of the sender. So the sender pays for the call. +- An answer does not change the sender. When the owner approves a tool in the turn of Bob, the rest of that turn still runs as Bob. +- A command runs as the person who runs it. So a sign-in, for example `connect:github`, belongs to the person who signed in. +- A background agent runs as the person who started it, also after the turn ends. The wake turn after the agent runs as that person too. + +A tool reads the credentials of the user of its own run: + +```ts group=harness-shared-threads +import { toolDefinition } from '@tanstack/ai' +import type { HarnessRunContext } from '@tanstack/ai-harness' + +export const myIssues = toolDefinition({ + name: 'my_linear_issues', + description: 'List the Linear issues of the user', +}).server(async (_args, ctx) => { + const credential = await ctx.context.run.credentials.require('linear') + const key = credential.type === 'oauth' ? credential.accessToken : credential.value + const response = await fetch('https://api.linear.app/graphql', { + method: 'POST', + headers: { authorization: key, 'content-type': 'application/json' }, + body: JSON.stringify({ query: '{ viewer { assignedIssues { nodes { title } } } }' }), + }) + return response.json() +}) +``` -If you write a command that saves a sign-in or a key, save it with the `credentials` of the second `run` argument. Those credentials belong to the person who runs the command. The `ctx.credentials` of `setup` belong to the sender of the running turn: +A command that saves a sign-in or a key saves it with the `credentials` of its run context. Those credentials belong to the person who runs the command: ```ts group=harness-shared-threads import { defineCommand, definePlugin } from '@tanstack/ai-harness' @@ -123,35 +200,62 @@ export const linearKey = definePlugin({ }) ``` -## 3. Keep other people out of a running turn +The `ctx.credentials` of a plugin `setup` have no user, so each call throws. See [Act as the user of a run](./plugins#act-as-the-user-of-a-run). + +Context that a client sends with a message is client data, and it never changes the sender. See [Send context with a message](./inputs#send-context-with-a-message). + +## 5. Keep other people out of a running turn A message with `busy: 'steer'` can join the turn that runs now. A joined message runs with the credentials of the sender of that turn. So without `turn.canJoin`, a waiting message joins the running turn only when the same person sent both: - A message from another person waits. It runs as its own turn after the running one, as its sender. - The waiting messages join as a prefix, in the order they arrived. So a message that waits also holds back the messages that wait after it. -Set `canJoin` to choose your own rule. It gets the sender of the waiting message as `principal`, and the sender of the running turn as `turnPrincipal`. This harness keeps the default rule, and it also lets your build bot join any turn: +Set `canJoin` to choose your own rule. It gets the sender of the waiting message as `user`, and the sender of the running turn as `turnUser`. This harness keeps the default rule, and it also lets your build bot join any turn: ```ts group=harness-shared-threads export const team = defineHarness({ name: 'acme/team', adapter: openaiText('gpt-6.1-sol'), turn: { - canJoin: ({ principal, turnPrincipal }) => - principal?.id === 'build-bot' || - (principal?.id === turnPrincipal?.id && - principal?.tenantId === turnPrincipal?.tenantId), + canJoin: ({ user, turnUser }) => + user?.id === 'build-bot' || + (user?.id === turnUser?.id && + user?.tenantId === turnUser?.tenantId), }, }) ``` -A background agent that a turn starts keeps the provider key of the sender of that turn. It keeps the key also after the turn ends. The wake turn after the agent runs as the person who started the agent. +## 6. Fork a shared thread + +Bob wants to try another path without changing the thread of the team. He forks it. The user who forks owns the fork: + +```ts group=harness-shared-threads +const [first] = await bobSession.transcript() +if (!first?.id) throw new Error('The thread has no messages.') + +const fork = await host.sessions.fork({ + harness: assistant, + threadId: 'team-thread', + at: { through: first.id }, + members: 'all', + user: bob, +}) +console.log(fork.members[0]) // { userId: 'bob', role: 'owner' } +``` + +`members` picks who else is in the fork: + +- `'self'`: only the user who forks. +- `'all'`: the members of the old thread keep their roles. The old owner becomes a `member`. -One limit applies to code that runs after its turn ended, for example a tool of a background agent. That code reads `ctx.credentials` and `ctx.session.principal` of the turn that runs at that time. In an agent, use `ctx.keys`. In a command, use the `credentials` of its `run` context. +Without `members`, the fork uses `fork.members` of `membersOnly()`, and the default is `'self'`. To change it for every fork, pass `membersOnly({ fork: { members: 'all' } })`. -## 4. Keep organizations apart +## 7. Keep organizations apart -The credential scope of a turn has the `userId` and the `tenantId` of the sender. So one person in two organizations has two sets of credentials. The memory store of `memoryPersistence()` keeps them apart. In your own credential store, put `scope.tenantId` in the key: +The tenant of a user picks the thread. One `threadId` in two tenants is two threads, so Acme and Globex never see each other's work. See [Keep tenants apart](./access#keep-tenants-apart). + +The credential scope of a run has the `userId` and the `tenantId` of its user. So one person in two organizations has two sets of credentials. The memory store of `memoryPersistence()` keeps them apart. In your own credential store, put `scope.tenantId` in the key: ```ts group=harness-shared-threads import type { Scope } from '@tanstack/ai' @@ -183,15 +287,79 @@ export async function shareKey( } ``` -- A turn reads the credential of its sender first. If the sender has none, the turn reads the credential of the tenant. -- A save from a turn goes to the sender only. This includes `connect:` and `ctx.credentials.set`. The credential of the tenant does not change. -- `ctx.credentials.list()` shows the credentials of the sender and of the tenant. For the same id, the credential of the sender wins. -- Use an API key for a credential of the tenant. If the turn refreshes an OAuth token of the tenant, the new token goes to the sender only. +- A run reads the credential of its user first. If the user has none, the run reads the credential of the tenant. +- A save from a run goes to its user only. This includes `connect:` and `run.credentials.set`. The credential of the tenant does not change. +- `run.credentials.list()` shows the credentials of the user and of the tenant. For the same id, the credential of the user wins. +- An OAuth token of the tenant works too. A refresh saves the new token for the tenant, so every user of the tenant gets it. See [Refresh a token](./auth#refresh-a-token). + +## 8. Show who is here + +Ada and Bob work in one thread. Each of them wants to see who else has the thread open now. The `presence` hooks of the host tell you when a client stream starts and when it ends. + +On the server, keep the open streams of each thread, and serve the list: + +```ts group=harness-shared-threads +// The open streams on this server: thread id, then connection id, then user id. +const online = new Map>() + +export const presenceHost = createHarnessHost({ + persistence: memoryPersistence(), + access: { + getUser: ({ request }) => users.get(request.headers.get('authorization') ?? '') ?? null, + rules: membersOnly(), + }, + presence: { + onConnect: ({ threadId, connectionId, user }) => { + const streams = online.get(threadId) ?? new Map() + streams.set(connectionId, user?.id ?? 'guest') + online.set(threadId, streams) + }, + onDisconnect: ({ threadId, connectionId }) => { + online.get(threadId)?.delete(connectionId) + }, + }, +}) + +// GET /api/who?threadId=team-thread: the users who have the thread open. +export async function whoIsHere(request: Request) { + const user = users.get(request.headers.get('authorization') ?? '') + const threadId = new URL(request.url).searchParams.get('threadId') ?? '' + // Only a user who can read the thread sees the list. Another user gets AccessDeniedError. + const entry = user && (await presenceHost.sessions.get({ threadId, user })) + if (!entry) return new Response('Not found', { status: 404 }) + return Response.json([...new Set(online.get(threadId)?.values())]) +} +``` + +In the browser, show the list: + +```ts group=harness-shared-threads-presence-client +export async function showWhoIsHere(list: HTMLElement) { + const response = await fetch('/api/who?threadId=team-thread', { + headers: { authorization: 'Bearer token-ada' }, + }) + if (!response.ok) return + const userIds: unknown = await response.json() + if (Array.isArray(userIds)) list.textContent = `Here now: ${userIds.join(', ')}` +} +``` + +Call `showWhoIsHere` when the page opens. To update it at once, publish a change from the hooks with your own pub/sub, and call it again when the change comes. + +- Each stream is one connection with its own `connectionId`. Two tabs of Bob are two connections, so the route counts users, not streams. +- `GET events` and the socket of `handleHarnessSocket` count. `POST run` streams and reads from your own code do not. +- `resumed` is `true` for a stream that goes on from a cursor, for example a reconnect. +- `onDisconnect` runs once for each `onConnect`. Its `reason` is `closed`, `revoked` (a revoke or `unshare`), or `error`. `error` includes a stream that the server closes because the client reads too slowly. +- Each server reports only its own streams. When more than one server runs, keep the streams in a shared store, for example Redis, in place of the `Map`. + +The host does not wait for a hook. A hook that throws or rejects only logs a warning, and the stream goes on. ## What you have now +- Threads that only their members can see, with an owner, members, and viewers. - Each message runs as the person who sent it, also after a restart. - Tools and model calls use the credentials and the provider key of the sender. - A message from another person waits for its own turn. -- Sign-ins and keys belong to the person who saved them. -- The credentials of two organizations stay apart, and one key can serve a whole organization. +- Forks that the forker owns, with the members you pick. +- The threads and credentials of two organizations stay apart. +- A list of who has the thread open now. diff --git a/docs/harness/skills.md b/docs/harness/skills.md index 89a399d56e..ebaa7c34a2 100644 --- a/docs/harness/skills.md +++ b/docs/harness/skills.md @@ -37,14 +37,15 @@ import { defineHarness } from '@tanstack/ai-harness' import { openaiText } from '@tanstack/ai-openai' import { skills } from '@tanstack/ai-skills/harness' +// One time, outside `plugins`, so the sessions share the folder watchers. +const agentSkills = skills({ + dirs: ['./.agents/skills', join(homedir(), '.acme', 'skills')], +}) + export const assistant = defineHarness({ name: 'acme/assistant', adapter: openaiText('gpt-6.1-sol'), - plugins: () => [ - skills({ - dirs: ['./.agents/skills', join(homedir(), '.acme', 'skills')], - }), - ], + plugins: () => [agentSkills], }) ``` @@ -73,7 +74,7 @@ export const cliSkills = skills({ ## Skills that change while the session runs -The plugin watches the folders, and it lists them again at the start of each turn: +The plugin watches the folders, and the commands follow them: - A new `SKILL.md` adds its command. - A removed skill takes its command away. @@ -82,6 +83,10 @@ The plugin watches the folders, and it lists them again at the start of each tur Each change sends a `harness.commands.changed` event, so a UI can show the new command list at once. +The plugin watches each folder once for each host. All sessions of the harness on that host share the watchers, so many open sessions still use one watcher for each folder. A change only tells the sessions to update. The plugin reads the folders again at the start of each turn. + +If the plugin cannot watch a folder, it logs one warning with the folder and the reason. An example is a Linux system with no free file watchers. Changes in that folder still show at the next turn. + ## Many skills The model's list has one line for each skill: the name and the first sentence of the description. When the list is longer than `maxCatalogTokens` (default 4000), the skills of the earlier folders stay, and the rest leave the model's list. A skill that the user calls with its command always stays in it. diff --git a/docs/harness/snapshots.md b/docs/harness/snapshots.md index cf8ab2425f..7ff300bf3d 100644 --- a/docs/harness/snapshots.md +++ b/docs/harness/snapshots.md @@ -23,13 +23,11 @@ import { createHarnessHost, defineHarness } from '@tanstack/ai-harness' import { permissions } from '@tanstack/ai-harness/plugins' import { snapshots, workspaceTools } from '@tanstack/ai-harness/plugins/coding' import { openaiText } from '@tanstack/ai-openai' -import { memoryLogStore, memoryPersistence } from '@tanstack/ai-persistence' +import { memoryPersistence } from '@tanstack/ai-persistence' const root = process.cwd() -const history = snapshots({ - root, - dataDir: join(homedir(), '.acme-coder', 'snapshots'), -}) +const dataDir = join(homedir(), '.acme-coder', 'snapshots') +const history = snapshots({ root, dataDir }) const coder = defineHarness({ name: 'acme/coder', @@ -37,45 +35,52 @@ const coder = defineHarness({ plugins: () => [permissions({ root }), workspaceTools({ root }), history], }) -const log = memoryLogStore() -const host = createHarnessHost({ - persistence: { stores: { log, runs: memoryPersistence().stores.runs } }, -}) +const persistence = memoryPersistence() +const { log } = persistence.stores +const host = createHarnessHost({ persistence }) const session = await host.open(coder, { threadId: 'fix-login' }) ``` - `root` is the workspace folder, an absolute path. -- `dataDir` is a folder for the snapshot repositories, an absolute path. Put it outside `root`. Each workspace gets its own repository in it. +- `dataDir` is a folder for the snapshot repositories, an absolute path. Put it outside `root`. Each thread of each tenant gets its own repository in it. - The plugin needs the `git` command. `/undo` and `/redo` need git 2.26 or later. - Your project does not need to be a git repository, and the plugin never uses the `.git` folder of your project. +- With [`workspaceTools({ workspace: { bind } })`](./coding-workspaces) before it, each thread keeps its snapshots in its own workspace. Leave out `root`. ## 2. Undo and redo a turn 1. Send a prompt that changes files: `await session.send('Fix the login form.')`. -2. Run `await session.command('undo')`. It returns the number of files that it put back, for example `Undid the last turn. Files restored: 3.` +2. Run `await session.runCommand({ name: 'undo' })`. It returns the number of files that it put back, for example `Undid the last turn. Files restored: 3.` 3. Look at the files that the turn changed: they are as they were before the turn. The turn is gone from the conversation too. -4. To bring back the files and the turn, run `await session.command('redo')`. It returns `Redid the last turn.` +4. To bring back the files and the turn, run `await session.runCommand({ name: 'redo' })`. It returns `Redid the last turn.` In a terminal app, the user types `/undo` and `/redo`. See [Build a coding agent](./coding-agent). - `/undo` puts back each file that the turn changed in `root`. This includes the edits of the file tools and the changes of `bash` commands. - `/undo` and `/redo` do not touch other files. This is also true for a file that you add or edit after the turn. -- If you edit a file that the turn changed, `/undo` and `/redo` still write over it, and your edit is lost. -- The snapshots skip the files that `.gitignore` names, like `node_modules`. `/undo` does not touch them. +- The snapshots skip the files that `.gitignore` names, like `node_modules`, and `.env` files. `/undo` does not touch them. - A new turn clears `/redo`. -In your own UI, add `expose: { commands: ['undo', 'redo'] }` to `defineHarness`. A client can run only the commands that the harness exposes. Then call `client.command('undo')`. The result text arrives as a `harness.command.result` event. +When someone else edits the same files: + +- If you edit a file that the turn changed, `/undo` and `/redo` still write over it, and your edit is lost. +- A step keeps only the files that its own run changed. So when another agent or session edits a file during the turn, `/undo` keeps that edit. +- A `bash` command does not tell which files it changed. So its step keeps every file that changed while the command ran. + +In your own UI, add `expose: { commands: ['undo', 'redo'] }` to `defineHarness`. A client can run only the commands that the harness exposes. Then call `client.runCommand({ name: 'undo' })`. The result text arrives as a `harness.command.result` event. ## See what a step changed Each model step that changed files leaves a step: `{ from, to, files, toolCallIds }`. `from` and `to` are the ids of two snapshots. `files` lists the changed paths, and `toolCallIds` lists the tool calls of the step. `history.diff(from, to)` gives the changed files and a unified diff. -On a [durable host](./durable-sessions), like the host above, the steps are `tanstack/snapshots:step` records in the session log: +The steps are `tanstack/snapshots:step` records in the [session log](./session-log): ```ts group=harness-snapshots +import { logIdOf } from '@tanstack/ai-harness' + await session.send('Fix the login form.') -for (const { record } of await log.read('fix-login')) { +for (const { record } of await log.read({ logId: logIdOf({ logId: 'fix-login' }) })) { if (record.type !== 'tanstack/snapshots:step') continue if (typeof record.from !== 'string' || typeof record.to !== 'string') continue const { files, patch } = await history.diff(record.from, record.to) @@ -85,7 +90,7 @@ for (const { record } of await log.read('fix-login')) { ``` - Without `to`, `diff` compares with the files now. `history.diff(record.to)` shows what changed after that step. -- On a host without a log, the steps are a list in the plugin state, `snapshot().plugins['tanstack/snapshots']`. +- `diff` finds a snapshot only in the repository of a session that is open. - A step that changed no file is not saved. ## Go back to an earlier message @@ -95,12 +100,13 @@ for (const { record } of await log.read('fix-login')) { ```ts group=harness-snapshots const messages = await session.transcript() const firstAnswer = messages.find((message) => message.role === 'assistant') -if (firstAnswer?.id) await session.revert(firstAnswer.id) +if (firstAnswer?.id) await session.revert({ messageId: firstAnswer.id }) // Bring back the files and the messages. await session.unrevert() ``` +- A revert uses the last 500 steps of the thread. The log snapshot keeps them, so a revert also works with [`keepLogRecords: 'since-snapshot'`](./lifecycle#drop-the-records-before-a-snapshot). - A revert touches only the files that the tool calls after that message changed. Each file goes back to its state before the first of these calls. A file that did not exist then is removed. - Other files stay as they are. Your own edits between the hidden turns stay too. - `unrevert()` touches only the same files. It puts them back as they were when the revert started, with your edits. @@ -109,7 +115,7 @@ await session.unrevert() - The next prompt drops the hidden messages for good. Then `unrevert()` does not change the files. - The revert is saved, so `unrevert()` still works after a restart. -In your own UI, call `client.revert(messageId)` and `client.unrevert()`. The client needs `'undo'` in `expose.commands`. More about reverts: [Go back to an earlier message](./sessions#go-back-to-an-earlier-message). +In your own UI, call `client.revert({ messageId })` and `client.unrevert()`. The client needs `'undo'` in `expose.commands`. More about reverts: [Go back to an earlier message](./sessions#go-back-to-an-earlier-message). ## In a sandbox @@ -131,14 +137,40 @@ export const sandboxHistory = snapshots({ Git then runs in the sandbox, and `dataDir` is a path in the sandbox. The sandbox image needs `git`. +## On more than one server + +The undo point and the steps are in the session log. So when another server takes the thread, `/undo`, `/redo`, and a revert work there too, if that server reaches the same snapshot repository: + +- In a sandbox, the repository is in the sandbox. A server that uses the same sandbox reaches it. See [A sandbox](./coding-workspaces#a-sandbox). +- On the host, give every server the same `dataDir` on a shared disk. Or pin each thread to the server that has its folder with `ownership: { hostId }`. See [A folder on one server](./coding-workspaces#a-folder-on-one-server). + +## Remove the snapshots of a thread + +[`host.sessions.delete`](./lifecycle#delete-a-thread-for-good) removes the snapshot repository of the thread for you. This needs two things: + +- The thread has an index entry. +- This host knows the harness: it opened a session of it before, or it has the harness in its [`harnesses`](./lifecycle#delete-a-thread-for-good) option. + +In other cases, call `purgeSnapshots` with the options of the plugin and the thread: + +```ts group=harness-snapshots +import { purgeSnapshots } from '@tanstack/ai-harness/plugins/coding' + +await purgeSnapshots({ root, dataDir }, { tenantId: 'acme', threadId: 'fix-login' }) +``` + +- Call it after the sessions of the thread close. See [Lifecycle and retention](./lifecycle). +- Leave out `tenantId` for a thread in the default tenant. +- It waits for the git commands of that repository in this process. +- It resolves also when there is no repository. + ## Known limits - One level of undo. A second `/undo` answers `Nothing to undo.` After a `/redo`, `/undo` works again. - `/undo` and `/redo` work only when the session is idle. During a turn, they answer `Wait until the turn ends, then try again.` - While a revert stands, `/undo` and `/redo` answer `A revert stands. Run unrevert, or send a prompt, first.` -- The undo point is kept in memory. After a restart, or when the session opens again, `/undo` works again after the next turn. - If you sent a message while the turn ran, `/undo` removes the conversation from that message on. The files still go back to the start of the turn. -- Without `git`, the plugin prints `snapshots: git was not found. Snapshots are off.` once, and adds no commands. `diff()` throws. The turns run as usual. +- Without `git`, the plugin sends a `tanstack/snapshots:failed` event (`SnapshotFailed`) with `snapshots: git was not found. Snapshots are off.`, and adds no commands. `diff()` throws. The turns run as usual. A snapshot that fails for another reason sends the same event, and the turn goes on. ## What you have now diff --git a/docs/harness/subagents.md b/docs/harness/subagents.md index 8d637a4f03..3e5debf6fa 100644 --- a/docs/harness/subagents.md +++ b/docs/harness/subagents.md @@ -2,7 +2,7 @@ title: Run agents from a harness id: harness-subagents order: 14 -description: "Start typed agents from plugins, message them while they run, route a turn to an agent, and call a whole harness as a child." +description: "Start typed agents from plugins and agent code, in parallel or in series, with limits and budgets. Message them while they run, share state, route a turn to an agent, and call a whole harness as a child." keywords: - tanstack ai - harness @@ -12,7 +12,7 @@ keywords: - limits --- -A review command needs an explorer to collect facts, then a reviewer to judge them. A research task needs three explorers at once. In a harness, plugins and commands start agents from code, and a whole harness can be a child of another one. A router can also send a whole turn to an agent. +A review command needs an explorer to collect facts, then a reviewer to judge them. A research task needs three explorers at once. In a harness, plugins, commands, and agents start agents from code, and a whole harness can be a child of another one. A router can also send a whole turn to an agent. Each run knows the run that started it, its user, its limits, and its budget, and the session log keeps all of it. ## Run agents from a plugin @@ -45,12 +45,10 @@ export const review = definePlugin({ description: 'Review a topic', input: z.object({ topic: z.string() }), run: async ({ topic }) => { - const [auth, sessions] = await ctx.agents.group({ onFailure: 'cancel-siblings' }, (group) => - Promise.all([ - group.run(explorer, { task: `${topic}: auth checks` }), - group.run(explorer, { task: `${topic}: session expiry` }), - ]), - ) + const [auth, sessions] = await ctx.agents.group({ onFailure: 'cancel-siblings' }, (g) => [ + g.start(explorer, { task: `${topic}: auth checks` }), + g.start(explorer, { task: `${topic}: session expiry` }), + ]) return `${auth}\n\n${sessions}` }, }), @@ -59,20 +57,69 @@ export const review = definePlugin({ }) ``` -- `ctx.agents.run(agent, input)` runs one agent and resolves with its result. -- `ctx.agents.start(agent, input, { wake: true })` runs it in the background and starts a turn when it finishes or fails. -- `ctx.agents.start(agent, input, { principal })` runs it as that user. Without `principal`, it runs as the user of the current turn. The wake turn runs as the same user. -- `ctx.agents.group(options, body)` runs several. With `onFailure: 'cancel-siblings'`, one failure cancels the others. With `'collect'`, use `group.runSettled` to get every result or error. Every child settles before `group` returns. +- `ctx.agents.run(agent, input, options?)` runs one agent. Await it to get the result. It is an `AgentRun`, so you can also cancel it or send it a message. +- `ctx.agents.start(agent, input, { startTurnWhenDone: true })` runs it in the background and starts a turn when it finishes or fails. +- `ctx.agents.start(agent, input, { user })` runs it as that user. The wake turn runs as the same user. +- Without `user`, a plugin with `lifetime: 'turn'` starts it as the user of its turn. A session plugin starts it as the user who opened the session. To start it for the user of a tool call, pass `user: ctx.context.run.user`. +- In a tool that an agent run calls, `ctx.agents` starts the agent as a child of that run. It runs for the user of that run, and counts in its tree and budget. It takes no `maxConcurrent` slot, so the run can await it. +- When tools of two runs run at the same time, the plugin cannot tell which run calls it. Then pass `parent: ctx.context.run` to name the run. +- A `parent` that names no running agent run of this session throws: an unknown id, a run that ended, an older run of a chain, or a run that another host runs. A running turn is fine: in a tool of the main model, `parent: ctx.context.run` starts a root run. +- Pass `parent: null` to start a root run instead. It takes a `maxConcurrent` slot. +- `ctx.agents.group(options, (g) => runs)` runs several. See [Run agents in parallel or in series](#run-agents-in-parallel-or-in-series). -The turn that `wake: true` starts sends the model a user message from the harness, for example `Background agent pricer finished: ...`. The user did not write it. So the transcript marks it with `metadata.harness.wake: true`, and a view from `createSessionView` does not show it. In your own chat UI, skip the user messages with this mark. +The input type comes from the `inputSchema` of the agent, and the result type comes from its output. A wrong input is a type error. An agent name instead of a definition gives a result of type `unknown`. `ctx.agents` in a plugin and `ctx.harness.agents` in agent code have the same type, `AgentActions`. + +The turn that `startTurnWhenDone: true` starts sends the model a user message from the harness, for example `Background agent pricer finished: ...`. The user did not write it. So the transcript marks it with `metadata.harness.wake: true`, and a view from `createSessionView` does not show it. In your own chat UI, skip the user messages with this mark. To track usage or apply a policy in each of these runs, see [Middleware in every agent](./plugins#middleware-in-every-agent). The harness saves the images, audio, and video that these agents make. See [Media that agents make](./media#media-that-agents-make). +## Run agents in parallel or in series + +A research agent has ten topics. It can explore them all at once, but your provider allows only two requests at a time. Run the explorers as a group with a `concurrency`: + +```ts group=harness-subagents +const researcher = defineAgent({ + name: 'researcher', + description: 'Explores each topic, two at a time', + inputSchema: z.object({ topics: z.array(z.string()) }), + run: async (ctx) => { + if (!ctx.harness) throw new Error('This agent must run in a harness.') + const notes = await ctx.harness.agents.group({ concurrency: 2 }, (g) => + ctx.input.topics.map((topic) => g.start(explorer, { task: topic })), + ) + return notes.join('\n\n') + }, +}) +``` + +The body starts each child with `g.start` and returns the runs. `group` resolves to their results, in the same order. Every child that the body started settles before `group` returns. + +- `concurrency`: how many children run at the same time. The others wait with the status `queued`, and use no budget until they start. The default is all of them. +- `onFailure: 'cancel-siblings'` (default): the first failure cancels the other children, also the ones that wait. Then `group` rejects with that error. +- `onFailure: 'collect'`: every child runs to its end. Each result is an `AgentSettled` entry: `{ ok: true, value }` or `{ ok: false, error }`. +- `key`: the name of the group in the run that starts it. See [Keep a background agent going after a crash](#keep-a-background-agent-going-after-a-crash). + +To run agents in series, await each `run`. The next agent starts after the one before it ends: + +```ts group=harness-subagents +const checker = defineAgent({ + name: 'checker', + description: 'Explores a topic, then checks the notes', + inputSchema: z.object({ topic: z.string() }), + run: async (ctx) => { + if (!ctx.harness) throw new Error('This agent must run in a harness.') + const { agents } = ctx.harness + const notes = await agents.run(explorer, { task: ctx.input.topic }) + return await agents.run(explorer, { task: `Find the errors in these notes:\n${notes}` }) + }, +}) +``` + ## Keep a background agent going after a crash -A background agent can work for a long time. When its host stops, for example in a deploy, the next host fails the run. Start it with `resume: true`, and the next host runs it again instead: +A background agent can work for a long time. When its host stops, for example in a deploy, the next host fails the run. Start it with `runAgainAfterCrash: true`, and the next host runs it again instead. A background child that the model starts runs again by default (see [Let the model run agents in the background](#let-the-model-run-agents-in-the-background)): ```ts group=harness-subagents // Your own services. @@ -88,6 +135,7 @@ const migrator = defineAgent({ description: 'Moves the data of one customer to the new region', inputSchema: z.object({ customer: z.string() }), run: async (ctx) => { + if (!ctx.harness) throw new Error('This agent must run in a harness.') const { step } = ctx.harness // Each side effect is a step. A finished step does not run again. const exportId = await step.do('export', () => startExport(ctx.input.customer)) @@ -109,7 +157,7 @@ export const migrations = definePlugin({ description: 'Move one customer', input: z.object({ customer: z.string() }), run: ({ customer }) => { - ctx.agents.start(migrator, { customer }, { resume: true, wake: true }) + ctx.agents.start(migrator, { customer }, { runAgainAfterCrash: true, startTurnWhenDone: true }) return `Moving ${customer}.` }, }), @@ -118,19 +166,53 @@ export const migrations = definePlugin({ }) ``` -1. Give the host a session log (`stores.log`). See [Durable sessions](./durable-sessions). Without a log, `start` with `resume: true` throws. +1. Keep the session log in a store that survives a restart. See [Durable sessions](./durable-sessions). 2. Put each side effect in `ctx.harness.step.do(name, fn)`. The agent code runs again from its start, and a finished step returns its stored value. 3. Give each step a name that is the same on every run, and return a JSON value from `fn`. -`ctx.harness` is on the context of every agent run that a session starts. Outside such a run, `step.do` runs `fn` each time. +`ctx.harness` gives the agent `step`, `agents`, and `state`. A harness gives it to every agent run in a session: + +- A run that a session starts from code gets the full context. A background child that the model starts gets it too. +- An agent that the model calls as a tool, and an agent that a router picks, get `step`, `agents`, and `state`. Their steps keep their values while the turn waits for the agent. See [When an agent tool waits](#when-an-agent-tool-waits). +- An agent in a plain `chat()` has no harness, so `ctx.harness` is `undefined`. Check it before you use it, as the samples on this page do. What the next host does: - It runs the agent again with the same input. Its `ctx.chat` calls continue from the saved transcript, and its finished tool calls do not run again. - A step runs at least once. A host can stop after `fn` ran and before the log stored its value. Then the next run calls `fn` again. -- After `durability.maxAttempts` runs (default 10), the run ends `failed` with the code `attempts_exhausted`. +- After `durability.maxAttempts` runs (default 10), the run ends `failed` with the code `attempts_exhausted`. To set the limit for one run, pass `limits: { maxAttempts }`. See [Give one run a budget and a time limit](#give-one-run-a-budget-and-a-time-limit). - `host.close()` cancels the run. A crash, a kill, or `host.close({ recoverable: true })` leaves the run for the next host. +### Children of a run that runs again + +The agent code runs again from its start, so it calls `start`, `run`, and `group` again. The harness does not start those children a second time: + +- A child that started before the host stopped is the same child. Its run gives the result that the log stored, or it goes on if it did not end. +- Only the children that did not start yet start now. +- The harness finds each child by its `key`. A call with no `key` is found by its place among the calls with no `key`: the order of the `start` and `run` calls, the order of the `group` calls, and the order of the children in each group. +- When the log has another agent at that place or key, the start throws. +- A background child that outlives its parent keeps its steps and finished tool calls. + +If the order of the `start`, `run`, or `group` calls can change between runs, give each call a `key`. The children in one group always keep their order: + +```ts group=harness-subagents +const auditor = defineAgent({ + name: 'auditor', + description: 'Explores the services, then writes a summary', + inputSchema: z.object({ services: z.array(z.string()) }), + run: async (ctx) => { + if (!ctx.harness) throw new Error('This agent must run in a harness.') + const { agents } = ctx.harness + const notes = await agents.group({ key: 'services' }, (g) => + ctx.input.services.map((service) => g.start(explorer, { task: service })), + ) + return await agents.run(explorer, { task: `Summarize:\n${notes.join('\n')}` }, { key: 'summary' }) + }, +}) +``` + +This works only for a run that started with `runAgainAfterCrash: true`, because only that run runs again on the next host. + ## Message a running agent A background agent drafts a post for ten minutes. Halfway through, the user sees that it cites blogs, not papers. Or the draft is done and they want one more change. Send the run a message, and the agent uses it. @@ -177,6 +259,12 @@ For a follow-up, `receipt.operationId` is the id of the new run. While the run b A steer joins only a run of the same sender. A steer from another sender runs as a follow-up for that sender. As a result, no message runs with the credentials of another person. +Who sends: + +- `run.send` and `run.cancel` act as the user of the run, not as the user who opened the session. +- To send as another user, pass `user`: `run.send('Add a title.', { user })`. +- `session.agents.drafter.run(input, { user })` and `start` run the agent as that user. + An agent written as plain code, with no `ctx.chat` call, makes no model call. It gets a steer only as a follow-up. To message a run from the browser, see [Send a message to a running agent](./connect#send-a-message-to-a-running-agent). @@ -201,24 +289,36 @@ if (session.agentRun(run.id)) { Each item of `agentRuns()` has these fields: -- `operationId` and `agent`: the run and the name of its agent. -- `status`: `running`, `queued` (a follow-up that waits for the run before it), `completed`, `failed`, or `cancelled`. -- `parentRunId`: the run whose agent code started this run, if there is one. -- `principal`: the user that the run runs for. +- `runId` (the same as `operationId`) and `agent`: the run and the name of its agent. `kind` is always `'agent'`. +- `subagentRunId`: the id of the run as a child of the thread, the same as `run.subagentRunId`. A run that runs again after a host stop keeps it. A run in the log that did not start yet has none. +- `status`: `running`, `queued`, `waiting`, `completed`, `failed`, or `cancelled`. A `queued` run did not start yet: it waits for a free slot, or it is a follow-up that waits for the run before it. A `waiting` run waits for an answer, see [When a background agent waits](#when-a-background-agent-waits). +- `parentRunId`: the run that started this run, if there is one. That is the agent run whose code started it, or the turn whose tool started it. +- `user`: the user that the run runs for. `budget`: the budget of the run, if it has one. +- `result` of a run that completed, and `error` (`{ message, code? }`) of a run that failed. + +`agentRuns()` reads the run tree of the session. See [The run tree](#the-run-tree). -On a durable host (a host with `stores.log`), these three methods also read the log. After a restart, they still find each run, and a follow-up continues a run that ended before the restart. See [Durable sessions](./durable-sessions). +These three methods also read the log. After a restart, they still find each run, and a follow-up continues a run that ended before the restart. See [Durable sessions](./durable-sessions). -A run and all of its follow-ups keep their messages in one thread: `::`. So each follow-up reads the full history of the runs before it. +A run and all of its follow-ups keep their messages in one thread: `::`. So each follow-up reads the full history of the runs before it. To read those messages, pass the run id or its `subagentRunId`: + +```ts group=harness-subagents-messages +const history = await session.agentTranscript({ runId: run.id }) +const same = await session.agentTranscript({ subagentRunId: run.subagentRunId }) +``` + +The list starts with the messages that the first run started from. See [Show what a child wrote after a reload](#show-what-a-child-wrote-after-a-reload). ### Start agents from agent code -An agent can start other agents in the background with `ctx.harness.agents.start(agent, input, options)`. It returns a run that you can await, message, and cancel: +An agent can start other agents in the background with `ctx.harness.agents.start(agent, input, options?)`. It returns a run that you can await, message, and cancel. `run` and `group` work here too, as in [Run agents in parallel or in series](#run-agents-in-parallel-or-in-series): ```ts group=harness-subagents-messages const editor = defineAgent({ name: 'editor', description: 'Starts two drafts and joins them', run: async (ctx) => { + if (!ctx.harness) throw new Error('This agent must run in a harness.') const { agents } = ctx.harness const intro = agents.start(drafter) const outro = agents.start(drafter) @@ -228,22 +328,55 @@ const editor = defineAgent({ }) ``` -- The agent must run in a session: from `session.agents`, from `ctx.agents` in a plugin, or as a background child. Outside a session run, `start` throws. -- Each child counts against the tree budget of the harness: `maxDepth`, `maxConcurrent`, and `maxCalls` in `subagents.limits`. Over the budget, `start` throws and no child starts. See [Stay within limits](#stay-within-limits). -- Each child runs for the sender of its parent run. Its `agentRuns()` item has the `parentRunId` of that run. +- The agent must run in a session. Outside a session, `start` throws. +- Each child counts against the limits of its tree. Over a limit, `start` throws and no child starts. See [Stay within limits](#stay-within-limits). +- Each child gets the user, the keys, and the signal of its parent run. See [What a child gets from its parent](#what-a-child-gets-from-its-parent). - A cancel of a run stops its children first, then the run. - A cancel of a follow-up that did not start yet removes it. It never runs. +### What a child gets from its parent + +A child run acts for the same person as the run that started it, and it stops when that run is stopped: + +- The child runs for the user of its parent run, with that user's credentials and provider keys. Its `agentRuns()` item has the `parentRunId` of the parent. +- A cancel, a timeout, or a budget stop of the parent also stops the child. +- A parent that completes leaves its background children running. +- A host stop for a deploy (`host.close({ recoverable: true })`) gives the child back for the next host. It does not cancel the child. +- A cancel of a turn also cancels the background runs that the turn started. + +A background run that a turn starts keeps the plugins of that turn, for example the `agentMiddleware` of a plugin with `lifetime: 'turn'`. The turn plugins are cleaned up when the turn and every run that it started have settled, or when the session closes. + ### Messages after a host stop -This applies only to a durable host. When the host stops during a run, the messages that wait for the run are in the log: +When the host stops during a run, the messages that wait for the run are in the log: -- If the run started with `resume: true`, the next host runs it again. The resumed run gets the messages that waited for it. -- If the run started without `resume`, the waiting messages settle `aborted`. A steer that already joined the run settles `failed`, with the run. +- If the run started with `runAgainAfterCrash: true`, the next host runs it again. The resumed run gets the messages that waited for it. +- If the run started without `runAgainAfterCrash`, the waiting messages settle `aborted`. A steer that already joined the run settles `failed`, with the run. + +### When a background agent waits + +A background run can need a person, for example an approval of [`permissions()`](./permissions) before it edits a file. The run then stops with status `waiting`. Its wait is in `session.snapshot().pendingWaits`, with the `runId` of the run: + +```ts group=harness-subagents-messages +const [wait] = session.snapshot().pendingWaits.filter( + (open) => open.runId === run.id, +) +if (wait) { + await session.resolve({ + resume: [{ interruptId: wait.waitId, status: 'resolved', payload: 'once' }], + }) +} +``` + +- The run continues after every wait of the run has an answer. +- The log keeps the wait, so no host stays busy. An answer on any host, also after a restart, continues the run. +- A run that started with `runAgainAfterCrash: true` can also ask with `ctx.harness.step.ask`. See [Wait for a person](./waits). +- The `question` tool belongs to the lead turn. Agent runs do not get it. +- An agent that the model calls as a tool inside a turn waits with the turn. See [When an agent tool waits](#when-an-agent-tool-waits). ### Known limits -- A run that another host runs is in `agentRuns()`, but it takes no message. `send` with `to` gets `not_running`. +- A run that another host runs is in `agentRuns()`. `send` with `to` gives the message to that host's run. A run that no host runs gets `not_running`. - A follow-up waits for its run. If that run had already ended when the host stopped, the follow-up settles `aborted`. ## Let the model run agents in the background @@ -291,6 +424,15 @@ For the tool input in plain `chat()`, see [The tool input](../chat/subagents#the When a background child ends, the session starts a new turn. Its message starts with `Background agent researcher finished:` or `Background agent researcher failed:`. While the child runs, it is in `session.agentRuns()`. +When the host stops while a background child runs, or before the child starts, the next host continues the child with the same `subagentRunId`, as a start with `runAgainAfterCrash: true` does: + +- The agent code runs again from its start. +- `ctx.chat` continues from the saved transcript. A tool call that finished does not run again. A tool call that ran at the stop runs again. +- `ctx.harness.step.do` keeps its values, so a finished step does not run again. +- The plugins with `lifetime: 'turn'` of the turn that started the child are gone. + +To fail a child that ran at the stop instead, set `durability: { resumeBackgroundAgents: false }` in `defineHarness`. A child that did not start yet still starts, because none of its code ran. + With a session index, each child gets an entry with its `parentThreadId` and `parentToolCallId`. See [Turn on the session index](./sessions#turn-on-the-session-index). The `subagents` option of a harness has no `binding` and no `keys`, because the session binds its own provider keys. @@ -301,9 +443,97 @@ Known limits: - Only the turns of the session can start a background child. A child that calls an agent tool with `background` gets the tool error `background needs a host that runs children in the background.` - A call with `background` and `sessionId` gets the tool error `background cannot continue a sessionId.` +## When an agent tool waits + +The main model calls a `writer` agent as a tool, and the writer wants to save a file. [`permissions()`](./permissions) asks the user first. The turn stops for that approval, and the writer goes on after the answer: + +```ts group=harness-subagents-waits +import { writeFile } from 'node:fs/promises' +import { defineAgent, toolDefinition } from '@tanstack/ai' +import { createHarnessHost, defineHarness } from '@tanstack/ai-harness' +import { permissions } from '@tanstack/ai-harness/plugins' +import { openaiText } from '@tanstack/ai-openai' +import { memoryPersistence } from '@tanstack/ai-persistence' +import { z } from 'zod' + +const saveFile = toolDefinition({ + name: 'save_file', + description: 'Write a file', + inputSchema: z.object({ path: z.string(), text: z.string() }), +}).server(async ({ path, text }) => { + await writeFile(path, text) + return `Saved ${path}.` +}) + +const writer = defineAgent({ + name: 'writer', + description: 'Writes files', + run: (ctx) => + ctx.chat({ adapter: openaiText('gpt-6.1-sol'), tools: [saveFile], stream: false }), +}) + +export const studio = defineHarness({ + name: 'acme/studio', + adapter: openaiText('gpt-6.1-sol'), + subagents: { agents: [writer] }, + plugins: () => [permissions({ rules: [{ tool: 'save_file', decision: 'ask' }] })], +}) + +const host = createHarnessHost({ persistence: memoryPersistence() }) +const session = await host.open(studio, { threadId: 'user-1-thread' }) +await session.send('Write a short README.') + +const waits = session.snapshot().pendingWaits +for (const wait of waits) console.log(wait.kind, wait.agent) // approval writer + +await session.resolve({ + resume: waits.map((wait) => ({ + interruptId: wait.waitId, + status: 'resolved', + payload: 'once', + })), +}) +``` + +- Each wait of the agent has `turn: true`, the `agent` name, and the `subagentRunId` of the agent run. For a nested agent, `agent` is the agent that waits. +- One `resolve` answers every wait of the turn. The agent goes on from where it stopped, and the main model gets its result as the tool result. +- The other waits of the agent work the same way: a sign-in, an interrupt of `defineInterrupt`, a client tool, and `ctx.harness.step.ask`. +- The log keeps the stopped agent. An answer on any host, also after a restart, continues it. +- With `durability: { agentToolWaits: false }`, an agent tool does not wait. A call of it that needs an approval gets a tool error, and `step.ask` throws. + +### Ask from agent code + +Agent code asks the user with `ctx.harness.step.ask`. The question is a wait of the turn, and you answer it with `resolve`: + +```ts group=harness-subagents-waits +const namer = defineAgent({ + name: 'namer', + description: 'Names a project', + run: async (ctx) => { + if (!ctx.harness) throw new Error('This agent must run in a harness.') + const { step } = ctx.harness + const audience = await step.ask({ + name: 'audience', + message: 'Who reads the project?', + schema: z.string(), + }) + return step.do('name', () => `A name for ${audience}`) + }, +}) +``` + +The `resolve` runs the agent code again from its start. An answered `step.ask` returns its answer, and a finished `step.do` returns its stored value. + +Known limits: + +- A `step.ask` with `secret: true` throws here. Ask for a secret in a [durable tool](./durable-tools) or a background run. +- `expiresAt` of a `step.ask` has no effect here. +- The wait does not get the `url` of a `step.ask`. Put the link in `message`. +- A crash while the agent runs (not while it waits) gives the main model a tool error. The agent does not run again. + ## Show what a child wrote after a reload -Your UI shows each child's text and tool calls while they stream. Then the user reloads the page. The thread's transcript has only the agent tool call and its result, because each child keeps its own messages. `session.agentTranscript(subagentRunId)` gives you those messages back. +Your UI shows each child's text and tool calls while they stream. Then the user reloads the page. The thread's transcript has only the agent tool call and its result, because each child keeps its own messages. `session.agentTranscript({ subagentRunId })` gives you those messages back. Each agent tool result names its child with `subagentRunId`. Read the result, then ask for the child: @@ -336,7 +566,7 @@ for (const message of await session.transcript()) { if (!('subagentRunId' in result)) continue if (typeof result.subagentRunId !== 'string') continue - const child = await session.agentTranscript(result.subagentRunId) + const child = await session.agentTranscript({ subagentRunId: result.subagentRunId }) console.log(child.map((item) => item.content)) } ``` @@ -347,6 +577,7 @@ What you get back: - Each message has the child's card in `metadata.tanstack.subagent`: its `name` and its `status` (`running`, `finished`, `error`, or `suspended`). - A child that failed before it wrote anything gives you one empty message. It holds only the card, so you can still show that it failed. - A nested child works the same way. Pass its own `subagentRunId`. +- A background child (`background: true`) and an agent run that code starts give the messages of their chain thread. See [Find a run by its id](#find-a-run-by-its-id). For those, `{ runId }` works too. In the browser, `client.agentTranscript` reads the same messages through [your handler](./connect): @@ -361,17 +592,19 @@ const client = createHarnessClient({ }) export async function childMessages(subagentRunId: string) { - return client.agentTranscript(subagentRunId) + return client.agentTranscript({ subagentRunId }) } ``` +The handler route is `GET .../agent-transcript?threadId=&subagentRunId=`, or `&runId=` for a run id. + You get an empty list when the id is unknown, or when the child belongs to another thread. The id can come from a prompt, so a session never reads the children of another thread. A [fork](./fork-and-reset) copies the agent results, but their children stay with the thread that ran them. So in the fork, `agentTranscript` gives an empty list for them. ## Route a turn to an agent Your harness has a writer and a pricer, but the main model answers every turn. Add `routing.router` to send a turn to the agent that can answer it. The router runs at the start of each new turn: a new message, a follow-up, or the wake turn of a background agent. -```ts +```ts group=harness-routing import { chat, decide, defineAgent, subagentRoute } from '@tanstack/ai' import { defineHarness } from '@tanstack/ai-harness' import { openaiText } from '@tanstack/ai-openai' @@ -429,15 +662,87 @@ export const studio = defineHarness({ The router can also return a pick of its own: `'main'`, a name, a list of names, `{ names, order }`, or `{ steps }`. Anywhere a name goes, `{ name, input }` can go too, for example `{ name: 'pricer', input: { vendor: 'Acme' } }`. If a picked agent has an `inputSchema` and no input, the turn fails with `Agent "pricer" needs input. Return { name: 'pricer', input } from the router.` See [Route, or let the model pick](../chat/subagents#route-or-let-the-model-pick) for each pick. -The router also gets `session`, the `input` of the turn as the user sent it, `operationId`, `inputId`, and `abortSignal`. Next to `router`, `routing` takes `order`, `strategy`, `limits`, and `sandbox`. They work the same as in `subagents`. +The router also gets `session`, the `input` of the turn as the user sent it, `operationId`, `inputId`, `abortSignal`, `activeAgent`, and `model`. Next to `router`, `routing` takes `order`, `limits`, `sandbox`, `afterAgentsAnswer`, `allowHandoffToHuman`, and `giveModelsHandoffTool`. The first three work the same as in `subagents`. `giveModelsHandoffTool: false` gives no model the `handoff` tool. See [Hand off a conversation](./handoffs). + +### Count the model calls of the router + +Your router asks a model which agent fits. A plain `chat()` call does not count in `session.usage()`, and it keeps running after the user cancels the turn. Call the model with `model.chat` instead: + +```ts group=harness-routing +export const helpdesk = defineHarness({ + name: 'acme/helpdesk', + adapter: openaiText('gpt-6.1-sol'), + agents: [writer, pricer], + routing: { + router: async ({ model, messages }) => { + const { text } = await model.chat({ + label: 'pick', + options: { + messages, + systemPrompts: ['Answer "writer" for a blog post, else "main". Answer one word.'], + }, + }) + return text.trim() === 'writer' ? 'writer' : 'main' + }, + }, +}) +``` + +- `model.chat` takes the same options as [`ctx.model.chat`](./plugins#call-a-model-from-a-plugin) of a plugin. Without `adapter`, it uses the thread setting `model`, else the harness `adapter`. +- The call uses the keys of the sender of the turn. A cancel of the turn stops it. +- Its usage counts in the turn, and in the session under `router`. See [Usage of the router and transfers](./usage#usage-of-the-router-and-transfers). What the turn does with the pick: - `'main'`: the main model answers, with its `subagents` as usual. - Agent names: only those agents run. The harness does not call the main model. The turn text is their answer as the transcript keeps it, for example `pricer:\n...`. -- `strategy: 'handoff'`: the agents run first. Then the main model answers in the same turn. It reads their text and keeps its subagent tools. +- `afterAgentsAnswer: 'main-answers'`: the agents run first. Then the main model answers in the same turn. It reads their text and keeps its subagent tools. With the default, `afterAgentsAnswer: 'end-turn'`, the turn ends after the agents. - An interrupt in a routed agent ends the turn. `session.resolve` continues the same plan, and it does not call the router again. -- A steer that you send during a routed turn waits. It runs as its own turn, and the router picks for that turn too. + +A routed agent works like the main model in a turn: + +- A steer that you send during the turn joins the next model call of the picked agent. +- If the agents already made their last model call, they run again with the steer, in the same turn. This late join does not call the router. +- `turn.beforeFinish` and `turn.onModelError` run for the routed agents too. +- Each answer of an agent stays in the transcript as its own message, also when the agent runs again or hands off to another agent. + +Some limits: + +- An agent that makes no model call, for example plain code in `run`, gets a steer only as a late join. +- When several agents run in parallel, the first one takes the steer. +- With `afterAgentsAnswer: 'main-answers'`, the main model takes no steer. A steer sent during its part runs as its own turn, and the router picks for that turn too. +- After `session.resolve` continues a plan, the agents do not run again for a late join or a turn hook. + +### Keep the agent that answers + +A user works with the writer for five turns. The router does not have to pick the writer again each time. The log keeps the active agent, and the router reads it as `activeAgent`: + +```ts group=harness-routing +export const desk = defineHarness({ + name: 'acme/desk', + adapter: openaiText('gpt-6.1-sol'), + agents: [writer, pricer], + routing: { + router: ({ activeAgent, input }) => { + if (input === 'Thanks, that is all.') return 'main' + if (activeAgent === undefined && typeof input === 'string' && input.includes('blog')) { + return { handoff: 'writer', note: 'The user wants a blog post.' } + } + return undefined + }, + }, +}) +``` + +How each pick changes the active agent: + +- `undefined`: the active agent answers. With no active agent, the main model answers. +- One agent name: that agent answers, and it becomes the active agent. +- `'main'`: the main model answers, and no agent is active. +- `{ handoff, note }`: the log keeps the note in a `harness.handoff` record. Then the target answers, and it becomes the active agent. `handoff: 'main'` clears it. +- `{ name, input }`, a list, `{ names }`, or `{ steps }`: those agents answer. The active agent does not change. + +Agents can also hand the turn to each other, or to a person. See [Hand off a conversation](./handoffs). ## Call a harness as a child @@ -461,6 +766,20 @@ export const lead = defineHarness({ The tool name comes from the harness name, with characters other than letters, digits, `_`, and `-` changed to `_`. Pass `name` to pick another. +In a session, the child is a run of the session tree. It opens a session of the child harness on its own thread, `/agent/`, in the log of the root thread. The child gets these from the run that calls it: + +- The user, with that user's credentials and provider keys. +- The budget of the parent run. The model calls of the child count against it. + +What you see in the parent session: + +- The model calls of the child count in `session.usage()` and in the budget of the parent. +- A wait of the child, for example a sign-in or an approval, is in `snapshot().pendingWaits` of the root session. Answer it there, and the child goes on in the parent run. The receipt names the parent run, so a `POST run` resume streams that run to its end. A wait that expires lets the child go on by itself. Every host sees the wait, and so does `host.observe`, because it is in the log of the root thread. An answer on another host goes to the host that runs the child. A child that waits keeps its parent run running. If the host of the child stops, no host runs the child again. The next host that owns the log ends the waits of that child as `cancelled`. +- The events of the child (text, reasoning, tool calls, and custom events) reach the parent stream with `parentRunId` in `metadata.tanstack`. A last `RUN_FINISHED` has the total usage of the child. +- The child session closes, and its plugins are cleaned up, when the run settles. + +Outside a session, for example in a plain `chat()`, `harnessAgent` runs the child harness on the host that you pass as `host`. Without `host`, it makes a host with a memory log the first time it runs. + ## Stay within limits A harness always limits its children. Without `subagents.limits`, it uses a depth of 2, 3 children at once, and 12 children per tree. Agents started from code count against the same limits: @@ -476,12 +795,97 @@ export const careful = defineHarness({ }) ``` -See [Limit the tree](../chat/subagents#limit-the-tree) for what each limit does. +See [Limit the tree](../chat/subagents#limit-the-tree) for what each limit does. In a harness session, they count like this: + +- `maxDepth` and `maxCalls` count per tree. An agent that a session, a plugin, or a wake starts is the root of its own tree. +- `maxConcurrent` counts the root agent runs of the whole session, and the children of each run. +- Over `maxConcurrent`, a root start waits with the status `queued`. It starts when a slot is free. A run that waits for a person keeps its slot. A plugin start in a tool of an agent run is a child of that run, not a root, unless it passes `parent: null`. +- Over a limit, a child start throws, and no child starts. A child of a `group` with `concurrency` waits in its group instead. +- A start that fails before its run starts, for example with an unknown agent or an input that is not valid, gives its call back. + +The log keeps the counts, so a restart does not reset them. A `queued` start also survives a host stop. The next host keeps its run id and its place in the order, and starts it when a slot is free. A start that was cancelled before the stop ends `cancelled`. + +## Give one run a budget and a time limit + +A drafter can loop on a hard task and use a lot of tokens. Give the run a budget and a time limit. When it goes over, the run fails, and the rest of the session goes on: + +```ts group=harness-subagents-messages +const limited = session.agents.drafter.start(undefined, { + limits: { timeoutMs: 5 * 60_000, maxAttempts: 3 }, + budget: { maxTokens: 50_000 }, + onModelError: { retries: 2 }, +}) + +try { + await limited +} catch { + const info = session.agentRuns().find((item) => item.runId === limited.id) + console.log(info?.error?.code) // 'budget' or 'timeout' +} +``` + +Every way to start an agent takes these options: `start`, `run`, and `g.start`, on `session.agents`, `ctx.agents`, and `ctx.harness.agents`. + +- `limits.timeoutMs`: the run fails with the code `timeout` after this time. A run again on another host keeps the first deadline. The deadline is never later than the deadline of the parent run, or `durability.timeoutMs`. +- `limits.maxAttempts`: the attempts of this run. It replaces `durability.maxAttempts` for this run. +- `budget.maxTokens` and `budget.maxCost`: the model use of the run and of all its children. The check runs after each model call, so a run stops after the first model call over the budget. Then it fails with the code `budget`. A run with `runAgainAfterCrash: true` that runs again after a host stop also counts the model use of the runs before. +- `onModelError`: what to do when a model call of the run fails before it streams output. Only that model call runs again, so a tool that ran before does not run twice. One model call runs at most 3 times. A tool that throws, or an error in the agent code, does not count. + +A budget holds for the whole tree under the run: + +- A child never has more budget than its parent has left. A child start under a parent with no budget left throws. +- `maxCost` counts the cost that the providers report. A call with no reported cost adds nothing to it. See [Track usage and cost](./usage). + +`onModelError` takes two forms: + +- `{ retries: 1 | 2, transientOnly? }`: retry up to `retries` times, after a short backoff or the `retryAfterMs` of the error (up to 5 s). With `transientOnly` (default `true`), only an error that `isTransientModelError` finds transient retries. The log keeps this form, so a run that another host takes over or resumes retries too. +- A function that returns `'retry'` or `'fail'`. The log cannot keep a function. So a run that another host takes over runs without it, and that host logs one warning. + +## The run tree + +The log keeps every run of a session as a node of one tree. A node has the type `RunNode`: + +- `runId` and `kind`: `turn`, `agent`, `tool`, or `command`. +- `status`: `queued`, `running`, `waiting`, `completed`, `failed`, or `cancelled`. +- `parentRunId`: the run that started it. A root run has none. +- `agent`, `user`, `key`, `limits`, and `budget`: what the run was started with. +- `result` and `error`: how the run ended. + +`session.agentRuns()` lists the agent nodes. The tree comes from the log, so a second host lists the same runs, also after a restart. To read every node yourself, see [Find the runs in the log](./session-log#find-the-runs-in-the-log). + +## Share state between agents + +A crawler starts many agents, and each one must skip the pages that another agent already read. Give them a shared state with `ctx.harness.state`: + +```ts group=harness-subagents +const reader = defineAgent({ + name: 'reader', + description: 'Reads one page, once per session', + inputSchema: z.object({ url: z.string() }), + run: async (ctx) => { + if (!ctx.harness) throw new Error('This agent must run in a harness.') + const seen = ctx.harness.state>({ key: 'seen-urls', initial: [] }) + if ((await seen.get()).includes(ctx.input.url)) return 'Already read.' + await seen.update((urls) => [...urls, ctx.input.url]) + return ctx.chat({ + adapter: openaiText('gpt-6.1-sol'), + messages: [{ role: 'user', content: `Summarize ${ctx.input.url}` }], + stream: false, + }) + }, +}) +``` + +- Every agent of one session that asks for the same `key` gets the same value. Before the first update, `get` gives `initial`. +- `get` and `update` work like [plugin state](./plugins#keep-state). The updates of one key run one at a time, and the value is JSON. +- The log keeps the value. It survives a restart, and a [fork](./fork-and-reset) copies it. +- Clients see each change in the AG-UI `STATE_SNAPSHOT` event, under `shared`. Plugin state is under `plugins`. +- An agent in a plain `chat()` has no `ctx.harness`, so it has no shared state. ## What you have now -- Commands and plugins that start typed agents, alone or in groups, and message them while they run. -- Background agents that continue on the next host after a crash, with steps that do not run twice. -- A tool for each agent that continues a child or starts it in the background. -- A router that sends each turn to the right agent, and gives each agent its input. -- Harnesses that call other harnesses as tools, in a tree that stays within limits. +- Plugins, commands, and agents that start typed agents, in parallel or in series, and message them while they run. +- Background agents that continue on the next host after a crash, with steps and children that do not run twice. +- Runs with their own budget and time limit, in a tree that stays within limits. +- A router that sends each turn to the right agent, and a routed agent that takes steers like the main model. +- Harnesses that call other harnesses as child runs, and agents that share state by key. diff --git a/docs/harness/thread-settings.md b/docs/harness/thread-settings.md index 9de666b134..e64cd982b3 100644 --- a/docs/harness/thread-settings.md +++ b/docs/harness/thread-settings.md @@ -6,13 +6,13 @@ description: "Store the model, reasoning, instructions, tools, plugins, and work keywords: - tanstack ai - harness - - settings - - configure + - threadSettings + - updateThreadSettings - model picker - cwd --- -Each conversation in your app has its own choices: a faster model for one chat, extra instructions for another, a folder for a coding task. These choices must stay after a reload and a restart. `session.configure()` stores them with the thread, and every next turn uses them. +Each conversation in your app has its own choices: a faster model for one chat, extra instructions for another, a folder for a coding task. These choices must stay after a reload and a restart. `session.updateThreadSettings()` stores them with the thread, and every next turn uses them. ## 1. Name the models a thread can pick @@ -40,19 +40,21 @@ const host = createHarnessHost({ persistence: memoryPersistence() }) ```ts group=harness-thread-settings const session = await host.open(assistant, { threadId: 'review-thread' }) -const receipt = await session.configure({ - model: 'strong', - reasoning: 'high', - instructions: 'Answer in French.', +const receipt = await session.updateThreadSettings({ + settings: { + model: 'strong', + reasoning: 'high', + instructions: 'Answer in French.', + }, }) console.log(receipt.status) // 'accepted' -console.log(session.settings()) // { model: 'strong', reasoning: 'high', instructions: 'Answer in French.' } +console.log(session.threadSettings()) // { model: 'strong', reasoning: 'high', instructions: 'Answer in French.' } ``` -The settings apply from the next turn. A field that you leave out stays as it is. Set a field to `null` to clear it: +The settings apply from the next turn. A field that you leave out stays as it is. `updateThreadSettings` merges your fields into the latest stored settings, also when another host changed them. Set a field to `null` to clear it: ```ts group=harness-thread-settings -await session.configure({ instructions: null }) +await session.updateThreadSettings({ settings: { instructions: null } }) ``` The six fields: @@ -66,9 +68,9 @@ The six fields: | `plugins` | `{ remove: [names] }`: the turns leave out what those plugins give. | | `cwd` | The working folder of the thread. See [The working folder](#the-working-folder). | -An unknown field, model, or plugin, or a bad value, gets a receipt with `status: 'rejected'` and the reason. `stores.metadata` keeps the settings, so they stay after a restart. Without a metadata store, they stay in memory. A [fork](./fork-and-reset) copies them. +An unknown field, model, or plugin, or a bad value, gets a receipt with `status: 'rejected'` and the reason. The settings are in the log, so a durable log store keeps them after a restart. A [fork](./fork-and-reset) copies them. -`configure` is an input, so the log keeps who changed what. Each change sends a `harness.settings.changed` event with the new `settings`. +`updateThreadSettings` stores an input, so the log keeps who changed what. Each change sends a `harness.settings.changed` event with the new `settings`. A session on another host reads the change too, so its next turn uses the new settings. A client can change only the fields in `expose.settings` of the harness. See [Choose what clients can change](./connect#choose-what-clients-can-change). @@ -81,7 +83,7 @@ const { models, settings } = session.describe() console.log(models) // ['fast', 'strong'] console.log(settings.model) // 'strong' -await session.configure({ model: 'fast' }) +await session.updateThreadSettings({ settings: { model: 'fast' } }) ``` ## What wins @@ -99,17 +101,18 @@ For `reasoning`, the `overrides` of the prompt win, then the stored setting, the - A `tools` list keeps only the tools with those names. `{ remove }` drops the tools with those names. Either way, the tools of the prompt `overrides` stay. - The harness applies `tools` before the plugins prepare the tool list (`prepareTools`). So a plugin that wraps tools sees only the tools that a thread keeps. -- `plugins: { remove }` leaves the tools, prompts, middleware, and model pick of those plugins out of the turn. Plugins that run once per session stay mounted. -- The agents and subagents that a removed plugin gives stay available. +- `plugins: { remove }` leaves everything that those plugins give out of the thread: tools, prompts, middleware, the model pick, agents, subagents, commands, and the items that they add for other plugins. For example, a thread without `agents()` gets no subagent tools, no `/agent` command, and no permission rules of its agent profiles. The thread also cannot run an agent of a removed plugin by its name. +- Plugins that run once per session stay mounted, but they give the thread nothing. +- If a plugin that stays (of any lifetime) or a harness `middleware` entry needs a capability of a removed plugin, the setting is rejected. ## The working folder -`cwd` is the working folder of the thread, for the plugins whose tools work in a folder. A plugin reads it at each call with `ctx.session.settings().cwd`, so a change applies from the next turn: +`cwd` is the working folder of the thread, for the plugins whose tools work in a folder. A plugin reads it at each call with `ctx.session.threadSettings().cwd`, so a change applies from the next turn: ```ts group=harness-thread-settings const task = await host.open(assistant, { threadId: 'fix-login' }) -await task.configure({ cwd: 'web-app' }) -console.log(task.settings().cwd) // 'web-app' +await task.updateThreadSettings({ settings: { cwd: 'web-app' } }) +console.log(task.threadSettings().cwd) // 'web-app' ``` When `cwd` changes, the model gets a short note before its next call. So it stops using the paths of the old folder: @@ -119,7 +122,7 @@ When `cwd` changes, the model gets a short note before its next call. So it stop - The same `cwd` again adds no note. - A thread with no messages yet gets no note. -The note is an assistant message in the transcript, so it stays after a restart. A change while a turn runs adds the note before the next turn. On a durable host (one with `stores.log`), the log keeps that waiting note, and the next host adds it once after a crash. Without `stores.log`, a crash loses a note that waits. +The note is an assistant message in the transcript, so it stays after a restart. A change while a turn runs adds the note before the next turn. The log keeps that waiting note, and the next host adds it once after a crash. ## What you have now diff --git a/docs/harness/tool-runs.md b/docs/harness/tool-runs.md index 8895278d43..59d93398f8 100644 --- a/docs/harness/tool-runs.md +++ b/docs/harness/tool-runs.md @@ -6,7 +6,7 @@ description: "Run one harness tool from a schedule, a webhook, or any server cod keywords: - tanstack ai - harness - - tool + - runTool - schedule - webhook - out-of-band @@ -17,7 +17,7 @@ A cron job or a webhook needs to run one tool, not hold a conversation. You want ## Run a tool from server code -Call `session.tool(name, args?)`. A schedule or a webhook handler calls it. +Call `session.runTool({ name, args })`. A schedule or a webhook handler calls it. ```ts group=harness-tool-runs import { createHarnessHost, defineHarness } from '@tanstack/ai-harness' @@ -43,7 +43,7 @@ const host = createHarnessHost({ persistence: memoryPersistence() }) const session = await host.open(dashboard, { threadId: 'ops-thread' }) // From a cron job, every hour: -const run = session.tool('fetchStats', { window: '1h' }) +const run = session.runTool({ name: 'fetchStats', args: { window: '1h' } }) const { text } = await run ``` @@ -53,18 +53,18 @@ const { text } = await run - While an approval waits, `session.tool` is refused with `pending_interrupts`. Answer the approval first. - Pass `inputId` to make the run idempotent, as [`send`](./inputs) does. -TypeScript types `args` from the input schema of a harness tool. `session.tool('fetchStats', { window: 1 })` does not compile. A plugin tool takes any `args`, and its input schema checks them when the tool runs. +TypeScript types `args` from the input schema of a harness tool. `session.runTool({ name: 'fetchStats', args: { window: 1 } })` does not compile. A plugin tool takes any `args`, and its input schema checks them when the tool runs. ## Tag a run with its trigger Pass `meta` to record why the tool ran, for example the schedule or the webhook event. The harness sends it as a `tanstack.injection` custom event with the tool call id. ```ts group=harness-tool-runs -const fromWebhook = session.tool( - 'fetchStats', - { window: '24h' }, - { meta: { trigger: 'webhook', deliveryId: 'whk-99' } }, -) +const fromWebhook = session.runTool({ + name: 'fetchStats', + args: { window: '24h' }, + meta: { trigger: 'webhook', deliveryId: 'whk-99' }, +}) await fromWebhook ``` @@ -91,7 +91,7 @@ This works for a model call and for `session.tool`. Your own `context` data stay ## Let a client run a tool 1. Add the tool to `expose.tools`. TypeScript suggests the names in `tools`, and `defineHarness` throws for any other name. -2. Run it from the client with `client.tool(name, args?, options?)`. `args` is typed from the tool's input schema. +2. Run it from the client with `client.runTool({ name, args, meta, inputId })`. `args` is typed from the tool's input schema. ```ts group=harness-tool-runs-client import { createHarnessClient } from '@tanstack/ai-harness/client' @@ -103,11 +103,11 @@ const client = createHarnessClient({ headers: { authorization: 'Bearer my-token' }, }) -const receipt = await client.tool( - 'fetchStats', - { window: '1h' }, - { inputId: 'hourly-stats-09' }, -) +const receipt = await client.runTool({ + name: 'fetchStats', + args: { window: '1h' }, + inputId: 'hourly-stats-09', +}) ``` - The receipt has your `inputId`. It is `queued` while another turn runs. diff --git a/docs/harness/turn-control.md b/docs/harness/turn-control.md index e121275f88..e82e456f95 100644 --- a/docs/harness/turn-control.md +++ b/docs/harness/turn-control.md @@ -55,7 +55,7 @@ const assistant = defineHarness({ - When the return changes the transcript, the model runs again. The hook runs again at the next stop. - Always check `messages`. A hook that returns the same reminder every time never stops. - `maxFinishCycles` (default 32) is the limit. A hook that still asks for more at the limit fails the turn. -- `records` are host records in the same append. They need a durable host. See [Append your own records](./session-log#append-your-own-records). +- `records` are host records in the same append. See [Append your own records](./session-log#append-your-own-records). ### Remind the model without saving the reminder @@ -83,7 +83,7 @@ const reminded = defineHarness({ - The transcript, the session log, and the message store never keep them. - A return with only `ephemeral` messages also sends the model back to work, and it counts for `maxFinishCycles`. - `onJoin` can return `ephemeral` messages too. They go to the model call of the join. -- A reminder that you send in your own loop, one turn at a time, does not count for `maxFinishCycles`. Pass it as `ephemeral` to `send` or `continue`. Each model call of that turn gets it, and no store keeps it. See [Give the model a note for one turn](./inputs#give-the-model-a-note-for-one-turn). +- A reminder that you send in your own loop, one turn at a time, does not count for `maxFinishCycles`. Pass it as `ephemeral` to `send` or `continueWithoutMessage`. Each model call of that turn gets it, and no store keeps it. See [Give the model a note for one turn](./inputs#give-the-model-a-note-for-one-turn). ## Retry model errors @@ -186,14 +186,18 @@ A message with `busy: 'steer'` waits while a turn runs. Before each model call, The join stops at the first message that: -- has an abort request (`session.cancel(operationId)` on it), or +- has an abort request (`session.cancel({ operationId })` on it), or - `turn.canJoin` refuses. That message and the ones after it wait. `canJoin` runs again before each model call and at each final answer, so it can run more than once for one message. Keep it a quick check with no side effects. What still waits when the turn ends runs as its own turn, before other queued turns. A cancelled one settles `aborted` and does not run. -A joined message runs with the credentials of the sender of the turn. So without `turn.canJoin`, a waiting message joins the running turn only when the same person sent both. A message from another person waits and runs as its own turn after the running one, as its sender. Joins are a prefix, so it also holds back the messages that wait after it. Set `canJoin` to choose your own rule. Compare `principal` with `turnPrincipal` to keep one person's message out of another person's turn. See [Keep other people out of a running turn](./shared-threads#3-keep-other-people-out-of-a-running-turn). +A `session.runTool()` turn takes no messages. It runs no model, so nothing could answer them. A message that arrives while it runs waits, and then runs as its own turn. See [Run a tool without a model](./tool-runs). -`onJoin` runs before the model call that the joined messages reach. It can return messages, which come after the joined ones, and records, which land in the same append as the join. Records need a durable host. +A turn that `routing` sends to root agents takes messages, the same as the main model. The root agents get them at each model call. + +A joined message runs with the credentials of the sender of the turn. So without `turn.canJoin`, a waiting message joins the running turn only when the same person sent both. A message from another person waits and runs as its own turn after the running one, as its sender. Joins are a prefix, so it also holds back the messages that wait after it. Set `canJoin` to choose your own rule. Compare `user` with `turnUser` to keep one person's message out of another person's turn. See [Keep other people out of a running turn](./shared-threads#5-keep-other-people-out-of-a-running-turn). + +`onJoin` runs before the model call that the joined messages reach. It can return messages, which come after the joined ones, and records, which land in the same append as the join. - If `onJoin` throws, the turn fails, and those messages run as their own turns. - If you cancel the turn while `onJoin` runs, the hook adds nothing to it, and those messages run as their own turns. @@ -308,7 +312,7 @@ Every model call of the turn uses the overrides: - each step of the tool loop - each `onModelError` retry - each `beforeFinish` cycle -- the main model after a [`routing`](./subagents#route-a-turn-to-an-agent) handoff +- the main model after the routed agents, with [`routing.afterAgentsAnswer: 'main-answers'`](./subagents#route-a-turn-to-an-agent) - the extra model call for steer messages that arrive at the final answer The `promptCache` of the overrides also reaches the agents that the model calls as tools in this turn. @@ -316,17 +320,17 @@ The `promptCache` of the overrides also reaches the agents that the model calls ### Tools for one turn - A tool name that the turn already has fails the turn, the same as two tools with one name. -- A [durable tool](./durable-tools) gets `step` and `append` on a durable host. +- A [durable tool](./durable-tools) gets `step` and `append`. - A middleware `onConfig` that returns a new tool list does not drop these tools. The harness adds them back. ### Where the overrides live -The session keeps the overrides in memory only. The session log does not store them. +The session log keeps the JSON overrides: `model`, `modelOptions`, `reasoning`, and `promptCache`. `adapter` and `tools` are objects, so they stay in memory only. - A queued prompt keeps its overrides until its turn runs. - A steer that joins a running turn uses the overrides of that turn. The session ignores the overrides of the steer. - A `send(text, { mode: 'steer', overrides })` that does not join runs as its own turn, with its own overrides. -- After a restart, recovery runs an unfinished turn with the harness settings. To give it overrides again, return them from `durability.recover`. See [Decide recovery yourself](./durable-sessions#decide-recovery-yourself). +- After a restart, recovery runs an unfinished turn with its JSON overrides. It runs without `adapter` and `tools`, with a warning. To give them again, return them from `durability.recover`. See [Decide recovery yourself](./durable-sessions#decide-recovery-yourself). - A second prompt with the same `inputId` and message is a duplicate, also when its overrides are different. To keep a model or instructions for every turn of a thread, also after a restart, store them with `session.configure`. See [Store settings per thread](./thread-settings). diff --git a/docs/harness/usage.md b/docs/harness/usage.md index f206d819e2..77be2c88fe 100644 --- a/docs/harness/usage.md +++ b/docs/harness/usage.md @@ -2,7 +2,7 @@ title: Track usage and cost id: harness-usage order: 7 -description: "Read the tokens and the cost of a thread, in total, by model, and by sender. Show a live total, stop a person who is over budget, and add /usage with your own prices." +description: "Read the tokens and the cost of a thread, in total, by model, by sender, by run, and by plugin. Show a live total, stop a person who is over budget, and add /usage with your own prices." keywords: - tanstack ai - harness @@ -29,7 +29,7 @@ export const assistant = defineHarness({ const host = createHarnessHost({ persistence: memoryPersistence() }) const session = await host.open(assistant, { threadId: 'team-thread', - principal: { id: 'ada' }, + user: { id: 'ada' }, }) await session.send('Summarize the open tickets.') @@ -39,11 +39,12 @@ console.log(total.calls, total.totalTokens, total.cost) console.log(byModel['openai/gpt-6.1-sol']?.totalTokens, bySender.ada?.cost) ``` -The totals have three groups. `snapshot().usage` has the same data: +The totals have four groups. `snapshot().usage` has the same data: - `total`: every model call of the thread. - `byModel`: the calls of each model, keyed by `provider/model`, for example `openai/gpt-6.1-sol`. -- `bySender`: the calls of each person, keyed by the principal id. It has only the calls of a known sender, so its sum can be less than `total`. +- `bySender`: the calls of each person, keyed by the user id. It has only the calls of a known sender, so its sum can be less than `total`. +- `bySource`: the model calls that are not a turn or an agent run, keyed by their source: `plugin:`, `router`, or `transfer`. See [Usage of plugins](#usage-of-plugins). Each entry has `calls`, `promptTokens`, `completionTokens`, `totalTokens`, `cachedTokens`, and `cacheWriteTokens`. If a provider reported a cost, the entry also has `cost`. The harness adds the costs that the providers report, in their currency. It uses no price list of its own. @@ -54,19 +55,82 @@ At the end of each call, the provider reports its usage. These calls count: - The model calls of each turn, for the sender of that turn. - The subagents that a turn starts, for the same sender. - Background agents, for the person who started the agent. +- A harness that runs as a child with [`harnessAgent`](./subagents#call-a-harness-as-a-child), for the same person. +- The model calls of plugins with `ctx.model.chat`, for the user of their run. +- The model calls of the router with `model.chat`, for the sender of the turn. +- The summary of a transfer with `messages: 'summary'`, for the person who made the transfer, in the source thread. A call that fails before it ends does not count. Each message runs as its sender, so in a [shared thread](./shared-threads), `bySender` shows what each person used. ## Where the totals live -The totals stay with the thread: - -- A durable host (with `stores.log`) keeps one `harness.usage` record per call in the log. The log keeps the totals after a restart. -- Another host with `stores.metadata` keeps the totals there. -- A host with neither keeps them in memory. +The totals stay with the thread. The host keeps one `harness.usage` record per call in the log, so a durable log store keeps the totals after a restart. A [fork of a thread](./fork-and-reset) starts at zero. +## Usage of one run + +One agent run used much more than the others, and you want to know which one. Each `harness.usage` record in the log has the `runId` of the run that made the call: a turn, or an agent run. Add them up per run: + +```ts group=harness-usage +import { logIdOf } from '@tanstack/ai-harness' +import type { LogStore } from '@tanstack/ai-persistence' + +// The tokens of the model calls that each run made itself. +export async function tokensByRun(log: LogStore, threadId: string) { + const tokens = new Map() + for (const { record } of await log.read({ logId: logIdOf({ logId: threadId }) })) { + if (record.type !== 'harness.usage' || typeof record.runId !== 'string') continue + const { usage } = record + if (typeof usage !== 'object' || usage === null || !('totalTokens' in usage)) continue + if (typeof usage.totalTokens !== 'number') continue + tokens.set(record.runId, (tokens.get(record.runId) ?? 0) + usage.totalTokens) + } + return tokens +} +``` + +- A call also counts for each run above its run in the [run tree](./subagents#the-run-tree). So the use of a run includes the use of its children. +- When an agent run ends, its run record gets `usage`: the tokens of the run and of all its children. It is in the `harness.run.update` record of the end. +- To stop a run that uses too much, give it a `budget`. See [Give one run a budget and a time limit](./subagents#give-one-run-a-budget-and-a-time-limit). + +## Usage of plugins + +A goal judge or a title writer calls a model next to the turn, and you want to see what these calls cost. A plugin that calls its model with [`ctx.model.chat`](./plugins#call-a-model-from-a-plugin) counts each call under its own name: + +```ts group=harness-usage +import { title } from '@tanstack/ai-harness/plugins' + +export const titled = defineHarness({ + name: 'acme/titled', + adapter: openaiText('gpt-6.1-sol'), + plugins: () => [title({ adapter: openaiText('gpt-6-luna') })], +}) + +const titledSession = await host.open(titled, { threadId: 'titled-thread' }) +await titledSession.send('Plan a trip to Rome.') + +const { bySource } = titledSession.usage() +console.log(bySource['plugin:tanstack/title']?.totalTokens) +``` + +- The title call runs next to the turn, so its usage can arrive a moment after the turn ends. +- The calls of a plugin also count in `total`, `byModel`, and `bySender`, and in the use of their run. +- The calls of turns and agent runs have no source, so they are not in `bySource`. +- Each `harness.usage` record of a plugin call has `source`, and the `label` of the call when it has one. The `goal()` judge has the label `judge`, `title()` has `title`, and `/compact` has `compact`. + +## Usage of the router and transfers + +Two more kinds of calls run outside a turn model call. Each one has its own source in `bySource`: + +- `router`: the calls of [`model.chat` in the router](./subagents#count-the-model-calls-of-the-router). They also count in the use of the turn. +- `transfer`: the summary call of a [transfer](./handoffs) with `messages: 'summary'`. Its record has the label `summary`. It counts in the source thread, also when the transfer fails later. + +```ts group=harness-usage +const { router, transfer } = session.usage().bySource +console.log(router?.totalTokens, transfer?.totalTokens) +``` + ## Show a live total Want a token counter that ticks up while the chat runs? `session.onUsage` calls you after each model call, with the new thread totals: @@ -80,7 +144,7 @@ const stopCounting = session.onUsage(({ total }) => { stopCounting() ``` -You also get the `model`, the `sender` when it is known, and the `usage` of that one call. Reading raw events? Each call is a `harness.usage` event (`HARNESS_EVENTS.usage`). +You also get the `model`, the `sender` when it is known, and the `usage` of that one call. A call of a plugin or of the router also has its `source`, and its `label` when it has one. Reading raw events? Each call is a `harness.usage` event (`HARNESS_EVENTS.usage`). ## Stop a person who is over budget @@ -90,12 +154,12 @@ Read the totals of the sender before you send a prompt. This route refuses a pro const BUDGET_USD = 5 export async function send(threadId: string, userId: string, text: string) { - const thread = await host.open(assistant, { threadId }) + const thread = await host.open(assistant, { threadId, user: { id: userId } }) const spent = thread.usage().bySender[userId]?.cost ?? 0 if (spent >= BUDGET_USD) { return { error: 'You used your budget for this thread.' } } - return thread.send(text, { principal: { id: userId } }).receipt + return thread.send(text).receipt } ``` @@ -106,7 +170,7 @@ Many providers report no cost, so `cost` can stay empty. For those, compare the With `stores.sessions`, each [session index](./sessions) entry has a `usage` field. The host writes the token counts of `session.usage()` at the end of each turn. `turns` is the number of model calls. ```ts group=harness-usage -const page = await host.sessions.list({ principal: { id: 'ada' }, limit: 20 }) +const page = await host.sessions.list({ limit: 20 }) for (const entry of page.entries) { console.log(entry.title ?? 'New chat', entry.usage?.totalTokens ?? 0) } @@ -140,7 +204,7 @@ export const priced = defineHarness({ }) ``` -`session.command('usage')` reads `session.usage()` and shows the total. When the thread used more than one model, it also shows a line for each model: +`session.runCommand({ name: 'usage' })` reads `session.usage()` and shows the total. When the thread used more than one model, it also shows a line for each model: ```text 3 model calls, 41200 input tokens (30000 cache read, 0 cache write), 2100 output tokens, 43300 total. Cost: $0.0432. @@ -166,7 +230,8 @@ Limits of the plugin: ## What you have now -- The tokens and the cost of a thread, in total, by model, and by sender. +- The tokens and the cost of a thread, in total, by model, by sender, and by plugin. +- The tokens of each run, with the use of its children in the run above them. - Totals that stay after a restart. - A live total from the `harness.usage` event. - A budget check before each prompt. diff --git a/docs/harness/waits.md b/docs/harness/waits.md new file mode 100644 index 0000000000..c34bacb379 --- /dev/null +++ b/docs/harness/waits.md @@ -0,0 +1,161 @@ +--- +title: Wait for a person +id: harness-waits +order: 5 +description: "Stop a run until a person approves a tool call, answers a question, or signs in. The session log keeps the wait, so any server can take the answer, also after a restart, and the wait can expire." +keywords: + - tanstack ai + - harness + - waits + - approvals + - step.ask + - human in the loop + - expiresAt +--- + +Your agent needs a person before it can go on: a yes for a refund, the amount, a sign-in. A person can take a minute or a day. In that time the server can restart, and the answer can reach another instance. In a harness, each of these is a wait. The session log keeps it, so no process holds it, any instance can take the answer, and it can expire. + +## 1. Ask from a tool + +Call `step.ask` in a [durable tool](./durable-tools). This tool asks for the amount of a refund, then pays it once: + +```ts group=harness-waits +import { toolDefinition } from '@tanstack/ai' +import { createHarnessHost, defineHarness, durableTool } from '@tanstack/ai-harness' +import { openaiText } from '@tanstack/ai-openai' +import { memoryPersistence } from '@tanstack/ai-persistence' +import { z } from 'zod' + +declare const payments: { + refund: (orderId: string, amount: number) => Promise<{ id: string }> +} + +const refund = durableTool( + toolDefinition({ + name: 'refund', + description: 'Refund an order', + inputSchema: z.object({ orderId: z.string() }), + }), + async ({ orderId }, { step }) => { + const amount = await step.ask({ + name: 'amount', + message: `How much do we refund for ${orderId}?`, + schema: z.number().positive(), + expiresAt: Date.now() + 24 * 60 * 60 * 1000, + }) + const paid = await step.do(`refund:${orderId}`, () => + payments.refund(orderId, amount), + ) + return { refundId: paid.id } + }, +) + +const support = defineHarness({ + name: 'acme/support', + adapter: openaiText('gpt-6.1-sol'), + tools: [refund], +}) +``` + +- `name` identifies the question in this call. Use the same name on every run of the code. +- The session checks the answer against `schema`. Here `amount` is a `number`. +- `expiresAt` ends the wait if nobody answers in time. Then `step.ask` rejects. Without it, the wait has no time limit. +- `secret: true` asks for a key or a password. The log never keeps that answer. + +When the host stops while the tool waits, the next host runs the tool again from the top. `step.ask` then returns the logged answer, or waits again. A finished `step.do` returns its stored value, so the refund runs once. + +## 2. Find the wait + +`session.waitFor` resolves with the first wait of a run: + +```ts group=harness-waits +const host = createHarnessHost({ persistence: memoryPersistence() }) +const session = await host.open(support, { threadId: 'ticket-7' }) + +const turn = session.send('Refund order A-17.') +const wait = await session.waitForRun({ + runId: turn.id, + signal: AbortSignal.timeout(60_000), +}) +console.log(wait.kind, wait.message) // question How much do we refund for A-17? +``` + +- Without `runId`, it resolves with the first wait of the session. +- `session.snapshot().pendingWaits` lists every open wait, oldest first. A wait that a chat run stopped for has its `interrupt`. A wait of the stopped turn has `turn: true`. +- A wait of an agent that the model calls as a tool has the `agent` name and the `subagentRunId` of that agent run. See [When an agent tool waits](./subagents#when-an-agent-tool-waits). + +Each wait has a `kind`: + +| Kind | The run waits for | +| --- | --- | +| `approval` | A yes or a no for a tool call. | +| `question` | An answer that fits the `schema` of the wait: from `step.ask`, the `question` tool, or an MCP server. | +| `sign-in` | The user signs in to a connector. | +| `client-tool` | The output of a tool that runs in the client. | +| `human` | A person who takes over, from an interrupt of `defineInterrupt`. | + +`waitKind(interrupt)` and `isApproval(interrupt)` give the kind of the `interrupt` of a wait too. Import them from `@tanstack/ai-harness`. + +## 3. Answer it + +```ts group=harness-waits +const hasInterrupt = session + .snapshot() + .pendingWaits.some( + (open) => open.waitId === wait.waitId && open.interrupt !== undefined, + ) + +if (hasInterrupt) { + await session.resolve({ + resume: [{ interruptId: wait.waitId, status: 'resolved', payload: true }], + }) +} else { + await session.answer({ questionId: wait.waitId, value: 40 }) +} +await turn +``` + +The tool pays 40 and the turn finishes. Two inputs answer waits: + +- `answer`: a wait with no `interrupt`: a `step.ask` question (of a durable tool, a command, or a background agent run), a question of an MCP server, and the `human` wait of a handoff. It checks the answer against the `schema` of the wait, and keeps a secret answer out of the log. +- `resolve`: a wait with an `interrupt`: a wait that a turn stopped for (an approval, the `question` tool, a client tool, a sign-in, a wait of an [agent tool](./subagents#when-an-agent-tool-waits)), and a wait of a [background agent run](./subagents#when-a-background-agent-waits). One `resolve` answers every wait of the stopped turn, or waits of one background run. A `resolve` of a wait with no `interrupt` is refused with `use_answer`. + +Both are inputs. An answer that reaches another instance waits in the log until the owner takes it. `host.submit` stores an answer for a thread that has no open session. + +## How a wait ends + +Each wait ends with one outcome: + +- `answered`: someone answered it. +- `cancelled`: the run stopped, or the `signal` of the question aborted. +- `expired`: `expiresAt` passed with no answer. + +Only a `sign-in` wait has a default time limit: 15 minutes. The other kinds wait until someone answers, unless you set `expiresAt`. An interrupt of a turn with an `expiresAt` also ends on time. + +`session.waitFor` and `session.settled` take a `signal` too. It stops your own wait for the session. It does not end the wait of the run. + +## Where a run can wait + +A run waits for a person only at these points: + +1. Before a tool call. [`permissions()`](./permissions) opens one approval wait for each call that asks. +2. The `question` tool. The model asks the user to pick an option. +3. `step.ask` in a durable tool, in a command, in an agent run that started with `runAgainAfterCrash: true`, or in an agent that the model calls as a tool. +4. Live waits: an MCP server that asks during a tool call (elicitation), and an OAuth sign-in. The host that runs the call keeps the thread until the answer comes. See [Answer a question from a server](./mcp#answer-a-question-from-a-server). + +`ctx.session.ask` in your own plugin throws an error that names these points. A command that called `step.ask` runs again from the top after a host stop, so put each side effect before the ask in `step.do`. + +## Who can answer + +On a host with [`access`](./access), its `rules` decide each answer: + +- An answer is the `answer` action. With [`membersOnly()`](./shared-threads), the owner of the session and the user who started the run have it. +- A `human` wait takes only the `operator` action. With `membersOnly()`, only members with the role `operator` have it. The owner does not. To change that, pass `membersOnly({ operator: { roles } })`. See [Shared threads](./shared-threads). + +## What you have now + +- A refund that waits for a person, on any instance, and pays once. +- One list of everything that waits, with a kind for each item. +- Waits that expire on time, and that a restart does not lose. + +To show waits in your own screen, see [Build your own UI](./custom-ui). diff --git a/docs/harness/workers.md b/docs/harness/workers.md new file mode 100644 index 0000000000..5fab236e5e --- /dev/null +++ b/docs/harness/workers.md @@ -0,0 +1,172 @@ +--- +title: Run background work on workers +id: harness-workers +order: 4 +description: "Run harness threads that no user has open: inputs that a server submits, schedules, tools that wait for hours, and the work that a deploy cut. A worker takes each thread from the session log." +keywords: + - tanstack ai + - harness + - worker + - createHarnessWorker + - submit + - schedule + - step.sleep + - deploy +--- + +Some work has no user who waits for it: a report that a cron job asks for, a reminder in one day, a tool that waits for a payment, or a turn that a deploy cut. Nobody opens these threads, so nothing runs them. A worker finds each thread in the session log that has work and no live host, takes it, runs the work, and gives it back. + +Every host on this page shares one log store. In production, that is your [`LogStore`](../persistence/store-reference#logstore). Here, `memoryLogState()` gives two hosts one memory log, the same as two servers on one database. + +## 1. Start a worker + +Make a host for the worker, and start the worker on it: + +```ts group=harness-workers +import { + createHarnessHost, + createHarnessWorker, + defineHarness, +} from '@tanstack/ai-harness' +import { memoryLogState, memoryLogStore } from '@tanstack/ai-persistence' +import { openaiText } from '@tanstack/ai-openai' + +const assistant = defineHarness({ + name: 'acme/assistant', + adapter: openaiText('gpt-6.1-sol'), +}) + +// One log store for every host. Swap in your own store. +const state = memoryLogState() +const hostOn = () => + createHarnessHost({ persistence: { stores: { log: memoryLogStore({ state }) } } }) + +const workerHost = hostOn() +const worker = createHarnessWorker({ host: workerHost, harnesses: [assistant] }) +``` + +- `harnesses`: the harnesses whose threads the worker runs, matched by name. +- `concurrency`: at most how many threads the worker runs at the same time. The default is 8. +- `pollMs`: how long the worker waits after a look that found no work. The wait doubles after each empty look, up to 30 seconds. The default is 1 000. +- `signal`: an `AbortSignal` that stops the worker, the same as `stop()`. + +Run as many workers as you want, on as many machines. Each thread goes to one of them. Each thread opens in its own tenant, with no user. + +If a thread fails to open, the worker logs the error and skips that thread for a time. The time doubles after each failure in a row, up to 5 minutes. The other threads still run. + +## 2. Submit an input from any server + +A cron route or a webhook wants the agent to do a job, but it must not wait for the turn. `host.submit` stores the input in the log and runs nothing: + +```ts group=harness-workers +const apiHost = hostOn() + +const receipt = await apiHost.submit({ + harness: assistant, + threadId: 'weekly-report', + input: { op: 'prompt', message: 'Write the weekly report.' }, + inputId: 'weekly-report-2026-42', +}) +console.log(receipt.status) // 'queued' +``` + +- The input waits until a host owns the thread. That is the host that runs the thread now, or a worker. +- Pass `inputId` so that a retry is safe. The same id with the same input is a duplicate. The same id with another input gets `rejected` with `conflict`. +- `input` is any input that runs on the owner: a prompt, a steer, a follow-up, an answer, a cancel, a command, and more. +- On a host with [`access`](./access), pass `user`. The input needs the same action as on a handle, else `submit` throws `AccessDeniedError` and stores nothing. +- An answer or a cancel is checked with the run that it targets, from the log. A cancel must name its `operationId`. A run that the log does not have gets `rejected` with `unknown_run`. To drop an input that waits, use `cancelInput`. + +## 3. Run an input later + +`session.schedule` runs an input at a time that you pick. The log keeps the schedule, so it runs once, also after a restart or on another machine: + +```ts group=harness-workers +const session = await apiHost.open(assistant, { threadId: 'daily-summary' }) + +const { scheduleId } = await session.schedule({ + at: Date.now() + 60 * 60 * 1000, + input: { op: 'prompt', message: 'Write the daily summary.' }, +}) +await session.close() +``` + +- While a host has the thread open, that host runs the input at `at`. +- When no host has the thread open, the log store wakes a worker at `at`. +- A time that passed runs at once. +- Pass `scheduleId` to pick the id. A second call with the id of a schedule that waits writes nothing. +- `user` is who sends the input when it runs. The default is the user of the open. + +To stop a schedule that did not run yet: + +```ts group=harness-workers +const later = await apiHost.open(assistant, { threadId: 'daily-summary' }) +await later.cancelSchedule({ scheduleId }) +await later.close() +``` + +A schedule that ran, or an unknown id, changes nothing. While another live host owns the thread, `schedule` and `cancelSchedule` throw. + +## 4. Wait inside a tool + +A tool sends a reminder, waits one day, and then sends a follow-up. `step.sleep` in a [durable tool](./durable-tools) waits, and the log keeps the time it wakes: + +```ts group=harness-workers +import { toolDefinition } from '@tanstack/ai' +import { durableTool } from '@tanstack/ai-harness' +import { z } from 'zod' + +declare const mail: { + send: (message: { to: string; text: string }) => Promise +} + +export const remind = durableTool( + toolDefinition({ + name: 'remind', + description: 'Remind a customer, and follow up one day later', + inputSchema: z.object({ to: z.string() }), + }), + async ({ to }, { step }) => { + await step.do('send-reminder', () => mail.send({ to, text: 'Your invoice is due.' })) + await step.sleep({ name: 'wait-a-day', ms: 24 * 60 * 60 * 1000 }) + await step.do('send-follow-up', () => mail.send({ to, text: 'Your invoice is late.' })) + return 'Sent.' + }, +) +``` + +- A sleep of 30 seconds or less waits in the process. The host keeps the thread while it waits. +- A longer sleep on a thread that only a worker runs gives the thread back to the log store with a wake. The worker slot is free, and `worker.stop()` does not wait for the sleep. On a thread that your app has open, the sleep waits in the process. +- At the wake time, or when the host stops, a worker runs the tool call again. The finished steps return their stored values, and the sleep waits only for the time that is left. +- `name` is a step name. Use the same name on every run, and not the name of a `do` step of the same call. +- Agent code has the same `step.sleep` on `ctx.harness.step`, in a run started with `runAgainAfterCrash: true`. See [Keep a background agent going after a crash](./subagents#keep-a-background-agent-going-after-a-crash). + +## 5. Restart for a deploy + +Stop a server with `recoverable: true`. Its running work stops, and the host gives each busy thread back with a wake, so a worker takes it at once: + +```ts group=harness-workers +process.on('SIGTERM', async () => { + await apiHost.close({ recoverable: true }) + process.exit(0) +}) +``` + +A worker host stops the same way. To let its threads end their work first, call `worker.stop()`. It stops the looks for work, and waits until the threads that the worker runs close: + +```ts group=harness-workers +process.on('SIGTERM', async () => { + await worker.stop() + await workerHost.close() +}) +``` + +To stop the work at once, close the host with `recoverable: true` instead. Another worker runs that work again. + +## What you have now + +- A pool of workers that runs every thread with work and no live host. +- Inputs that any server submits, and inputs that run at a time you pick. +- Tools that wait for hours, and continue on another machine after a stop. +- Deploys that hand the cut work to a worker at once. + +A client reads these threads as any other thread. See [Connect clients](./connect). diff --git a/docs/persistence/build-your-own-adapter.md b/docs/persistence/build-your-own-adapter.md index 1c3b63a7e4..cfeff46423 100644 --- a/docs/persistence/build-your-own-adapter.md +++ b/docs/persistence/build-your-own-adapter.md @@ -189,7 +189,7 @@ The optional run methods are `listByThread`, `listByParentRun`, and `skipMethods`. An omitted `listByParentRun` needs no entry: the subagent checks skip on their own. -The session stores (`inbox`, `log`, `workClaims`, `sessions`) are +The session stores (`log` and `sessions`) are opt-in too. Their cases run only when your persistence has the store, so a backend without them needs no `skip` entry. See the [session stores](./store-reference#session-stores) for their rules. diff --git a/docs/persistence/store-reference.md b/docs/persistence/store-reference.md index d753de5a65..9a24b3c8d8 100644 --- a/docs/persistence/store-reference.md +++ b/docs/persistence/store-reference.md @@ -16,10 +16,7 @@ there is no separate enable list. | `generationRuns` | Generation run status and result metadata, keyed by its own `runId`. | `withGenerationPersistence`, required | | `artifacts` | File metadata. Needs `blobs`. | `withGenerationPersistence`, portable snapshots | | `blobs` | File bytes. Needs `artifacts`. | `withGenerationPersistence`, portable snapshots | -| `inbox` | Inputs a host accepted, one entry per input id. | Your host code | -| `log` | Append-only session log per thread. | Your host code | -| `leases` | Turn leases from a system that already has them. | Your host code | -| `workClaims` | Claims on threads that have pending work. | Your host code | +| `log` | Append-only session log, with ownership, an input lane, and snapshots. | A harness host, required | | `sessions` | One index entry per thread, for a list of sessions. | `withPersistence` (subagent children), your host code | Named groupings of the chat stores (`ChatTranscriptStores`, `ChatPersistenceStores`, @@ -596,279 +593,455 @@ skips `blobs` altogether instead. ## Session stores A host that keeps long-lived sessions needs more than a transcript. It must remember -the inputs it accepted, the keys a user saved, which host works on which thread, and a -list of sessions to show. The session stores hold that data. +the inputs it accepted, the state of each thread, which host works on which thread, +and a list of sessions to show. Two stores hold that data: -All of them are opt-in. `withPersistence` uses only `sessions`, to index subagent -children. The other stores are for your host code. +- `log`: one append-only log per session tree. A harness host needs it. +- `sessions`: one index entry per thread. `withPersistence` also uses it, to index + subagent children. -## InboxStore - -A client sends an input, and the host must not lose it in a crash or run it twice. -Write the input to the inbox before you answer the client. `InboxStore` keeps one -entry per `inputId`. - -```ts -type InboxStatus = 'pending' | 'applied' | 'rejected' - -interface InboxEntry { - inputId: string // the idempotency key - threadId: string - principal?: { id: string; tenantId?: string } // who sent it - input: unknown // stored as-is - status: InboxStatus - createdAt: number // epoch ms - expiresAt?: number - operationId?: string // the operation that applied it - reason?: string // why it was rejected -} - -interface InboxStore { - append(entry: Omit): Promise - listPending(threadId: string): Promise> - markApplied(entry: { threadId: string; inputId: string; operationId: string }): Promise - markRejected(entry: { threadId: string; inputId: string; reason: string }): Promise - get(entry: { threadId: string; inputId: string }): Promise - delete(entry: { threadId: string; inputId: string }): Promise -} -``` - -Rules for an implementation: - -- `inputId` is unique across all threads. Let the client send a UUID. -- `append` stores a new entry as `'pending'`. For an `inputId` that is already - stored, it returns the stored entry and ignores the new payload. -- `listPending` returns the pending entries of one thread, oldest `createdAt` first. -- `markApplied` and `markRejected` do nothing for an unknown `inputId`. -- `get` returns `null` for an unknown `inputId`, and for an `inputId` of another - thread. -- `delete` removes the entry. It does nothing for an unknown `inputId`, or for - an `inputId` of another thread. - -In SQL, make `input_id` the primary key and write with `INSERT ... ON CONFLICT DO -NOTHING`, then read the row back. +A harness host keeps the data of each tenant apart. In `log`, `metadata`, and the +media artifacts, it writes the id of the root thread as +`logIdOf({ tenantId, logId: threadId })` from `@tanstack/ai-harness`: `acme:thread-1`, +or `:thread-1` in the default tenant. Keep it as an opaque string. The session index +is the one store that keeps `tenantId` and `threadId` as two fields. ## LogStore -Some hosts keep each thread's events, transcript, inputs, and tool steps in one -append-only log, so that a turn survives a crash. `LogStore` stores that log. +A host keeps the events, the transcript, the inputs, the runs, the interrupts, the +settings, the plugin state, and the tool steps of a session tree in one append-only +log. The root thread and its child threads (agent runs and subagents) share it. With the log, a turn survives a crash. When several machines +serve the sessions, the log also records which host runs each session, and it keeps +the inputs that any machine accepts. `LogStore` stores all of this. + +The types, `NO_OWNER_EPOCH`, and the three errors come from `@tanstack/ai-persistence`: ```ts +const NO_OWNER_EPOCH = 0 // the epoch of a log that no host ever acquired + interface LogRecord { type: string [key: string]: unknown } interface LogEntry { - seq: number + seq: number // the first record of a log is at 1 + record: LogRecord +} + +interface LogSnapshot { + seq: number // the last position that `state` includes + version: string // the version of the `state` shape, yours to name + state: unknown // JSON +} + +type OfferResult = 'offered' | 'duplicate' | 'conflict' | 'deleted' + +interface OfferedInput { + inputId: string record: LogRecord + offeredAt: number // epoch ms, from the store clock +} + +interface OrphanedLog { + logId: string + harness: string // the name of the setup that runs the log + reason: 'expired' | 'offered' | 'wake' } interface LogStore { + // Records append(batch: { - threadId: string + logId: string seq: number + epoch: number records: ReadonlyArray + consumes?: ReadonlyArray // input ids to consume in the same write }): Promise - read( - threadId: string, - options?: { after?: number; limit?: number }, - ): Promise> - subscribe(options: { threadId: string; listener: () => void }): () => void + read(query: { + logId: string + after?: number + limit?: number + }): Promise> + subscribe(watch: { logId: string; listener: () => void }): () => void + + // Ownership + acquireOwnership(claim: { + logId: string + harness: string + ownerId: string + ttlMs: number + affinity?: string + }): Promise<{ epoch: number; expired?: boolean } | null> + renewOwnership(claim: { + logId: string + ownerId: string + epoch: number + ttlMs: number + }): Promise + releaseOwnership(claim: { + logId: string + ownerId: string + epoch: number + wakeInMs?: number + }): Promise + pinToWorkers(pin: { + logId: string + epoch: number + affinity: string | null + }): Promise + markHostAlive(beat: { hostId: string; ttlMs: number }): Promise + listLogsWithoutOwner(query: { + limit: number + harnesses?: ReadonlyArray + affinity?: string + }): Promise> + + // Input lane + offerInput(input: { + logId: string + harness: string + inputId: string + record: LogRecord + }): Promise + listOfferedInputs(query: { logId: string }): Promise> + + // Snapshots and lifecycle + getSnapshot(query: { logId: string }): Promise + putSnapshot( + snapshot: { logId: string; epoch: number } & LogSnapshot, + ): Promise + truncate(cut: { + logId: string + epoch: number + beforeSeq: number + }): Promise + deleteLog(target: { logId: string }): Promise } ``` -- `append` writes the batch at `seq`, `seq + 1`, and so on. The first record of a - thread is at `1`. -- The batch is atomic. Write all of it or none of it, in one transaction. -- `seq` must be the next free position. For a taken position or a gap, reject with - `LogConflictError` from `@tanstack/ai-persistence`, and write nothing. This is how - a writer finds a second writer on the same thread. -- An empty batch writes nothing. -- `read` returns the entries after `after`, in `seq` order, at most `limit`. A thread - with no records returns `[]`. Return copies, so a caller cannot change the log. -- `subscribe` calls `listener` after each append that the store can see. A store that - cannot see appends from other processes can poll, or call only for its own appends. - -In SQL, a primary key on `(thread_id, seq)` gives the conflict rule. Insert the batch -in one transaction, and map the key violation to `LogConflictError`: - -```sql -CREATE TABLE session_log ( - thread_id TEXT NOT NULL, - seq INTEGER NOT NULL, - record TEXT NOT NULL, - PRIMARY KEY (thread_id, seq) -); -``` - -A key violation covers a taken position. A gap needs one more check in the same -transaction: `seq` must equal `MAX(seq) + 1` for the thread, or `1`. - -Each record is JSON. Store it as text or as a JSON column. The store does not read -the records. - -`memoryLogStore()` is the reference store. It is not part of `memoryPersistence()`, -so add it yourself: +`memoryLogStore()` is the reference store. `memoryPersistence()` includes one. To +run two store instances on one memory log, as two machines on one database, share a +`memoryLogState()`: ```ts -import { - defineAIPersistence, - memoryLogStore, - memoryPersistence, -} from '@tanstack/ai-persistence' - -const persistence = defineAIPersistence({ - stores: { ...memoryPersistence().stores, log: memoryLogStore() }, -}) +import { memoryLogState, memoryLogStore } from '@tanstack/ai-persistence' + +const state = memoryLogState() +const first = memoryLogStore({ state }) +const second = memoryLogStore({ state }) ``` -## LeaseStore +### Fence the writes with the epoch -Your system can already have leases: a job queue, or a submission table that says -which worker owns a job. `LeaseStore` lets a host use them for each attempt of a turn. +Two machines can work on one log. Host A owns the log, and then it stops for a +while: a long garbage collection pause, or a lost network. Its ownership expires, +and host B acquires the log. When host A continues, it writes again. Without a +fence, the log gets the records of two owners. -A host does four things with `leases`: +The epoch is the fence. Each `acquireOwnership` that starts a new ownership adds 1 to the +epoch of the log. The owner sends that epoch with each write, and the store refuses +an old epoch. So the late write of host A rejects with `OwnershipLostError`, and the +log does not change. -- It takes a lease when a turn attempt starts. -- It renews the lease while the attempt runs. -- It releases the lease after the attempt ends. -- After a crash, it calls `isAlive` to find attempts that nobody runs any more. +`append`, `putSnapshot`, `truncate`, and `pinToWorkers` are fenced. They check the +epoch before anything else: -```ts -interface TurnLeaseKey { - threadId: string - inputId: string - operationId: string // the operation that runs the attempt, also the AG-UI run id - attempt: number // 1 for the first run of the input, then 1 more after each crash -} - -interface TurnLease extends TurnLeaseKey { - ownerId: string // the host that runs the attempt - expiresAt: number // epoch ms -} +- On a log that a host acquired, `epoch` must be the current epoch. A released log + refuses every epoch. +- On a log that no host acquired, `append` needs `NO_OWNER_EPOCH`. A writer that never + calls `acquireOwnership` writes with it. `putSnapshot`, `truncate`, and `pinToWorkers` need + an owner, so they refuse `NO_OWNER_EPOCH`. +- A deleted log refuses every epoch until the next `acquireOwnership`. +- A call with any other epoch rejects with `OwnershipLostError`, and the store writes + nothing. -interface LeaseStore { - acquire(lease: TurnLease): Promise - renew(lease: TurnLease): Promise - release(lease: TurnLease): Promise - isAlive(key: TurnLeaseKey): Promise -} -``` +### Rules for records + +`append` writes `records` at `seq`, `seq + 1`, and so on. The first record of a log +is at 1. `append` checks in this order, and the first check that fails rejects: + +1. The fence. +2. `seq` is the next free position: the last position plus 1, or 1 for a new log. + Else it rejects with `LogConflictError`. +3. Each id in `consumes` is a pending offer. Else it rejects with a `TypeError`. +4. Each record can become JSON. Else it rejects with the error of the JSON copy. + +The batch is atomic. The store writes all the records and consumes all of +`consumes`, or it changes nothing. An empty batch with no `consumes` resolves. It +writes nothing and checks nothing. + +`read` gives the log back in order: + +- It returns the entries with `seq` more than `after` (default 0), in `seq` order, at + most `limit`. +- `limit: 0` returns `[]`. A log with no records returns `[]`. +- If a truncate dropped the record at `after + 1`, `read` rejects with + `LogTruncatedError`. +- The entries are copies. A change to them does not change the log. + +`subscribe` wakes a host when the log changes: + +- The store calls the listener after each successful `append`, each `offerInput` that + returns `'offered'`, and each `deleteLog` of the log. +- The calls come from every store instance on the same backend. That is how one + machine hears the writes of another. +- A store that cannot push must poll inside `subscribe`. +- If a listener throws, the store logs the error with `console.error`. The write + still succeeds. + +### Rules for ownership + +Each log has one ownership record: `harness`, `ownerId`, the epoch, the expiry time, +a released flag, and an optional affinity and wake time. The epoch never goes down. +It stays after `releaseOwnership` and after `deleteLog`. The store clock decides each expiry, +so callers pass `ttlMs` and `wakeInMs`, never a timestamp. A `ttlMs` of 0 or less, +or a `wakeInMs` less than 0, rejects with `RangeError`. + +`acquireOwnership` gives the log to `ownerId` for `ttlMs` milliseconds: + +- It returns `null` when another owner holds a live record: not released, and not + expired. +- It returns `null` when the log has an affinity that is not `claim.affinity`, and + the host of that affinity is alive. +- For the owner of a live record, it extends the expiry and returns the same epoch. +- Else it adds 1 to the epoch, stores `harness`, clears the wake time, opens a + deleted log again, and returns `{ epoch }`. +- When the record had another owner that did not release, it returns + `{ epoch, expired: true }`. That owner stopped. The harness then sends + `harness.ownership.changed` with `reason: taken_over`. + +For the other ownership methods: + +- `renewOwnership` returns `true` only for the current epoch and owner of a log that is not + released. In that case, it sets the expiry to `ttlMs` from now. A renew after the + expiry also succeeds if no other host acquired the log in between. +- `releaseOwnership` with the current epoch and owner marks the log released. With + `wakeInMs`, it also sets the wake time to `wakeInMs` from now. Any other `releaseOwnership` + does nothing. +- `pinToWorkers` pins the log to the workers with that affinity, or unpins it with + `null`. Only the owner pins: it needs the current epoch of an acquired log. +- `markHostAlive` marks the host `hostId` alive for `ttlMs` from now. `hostId` is the + affinity of its pins. A pin counts only while its host is alive, so the logs of a + host that stopped go to other hosts. A host that is alive keeps its pins, also on + the logs that it released. + +`listLogsWithoutOwner` finds the logs that have work and no live owner. A worker sweep +([`createHarnessWorker`](../harness/workers) or `host.resumePending`) calls it, then +calls `acquireOwnership` for each log. A log is listed for the first reason that +applies: + +1. `'expired'`: the log is not released, and its expiry is due. +2. `'offered'`: the log has pending offers, and no live owner. +3. `'wake'`: the log is released, and its wake time is due. + +Then: + +- With `harnesses`, the store lists only the logs with one of those harness names. +- A log with an affinity is listed only when the query has the same `affinity`, or + when the host of the affinity is not alive. +- The log that is due first comes first, and the store lists at most `limit` logs. + `limit: 0` returns `[]`. + +### Rules for the input lane + +A client can send an input to any machine, but only the owner of the log runs it. +`offerInput` keeps the input as a pending offer. The owner consumes it with +`append({ consumes })`, in the same write as the records of the input. + +- If `deleteLog` deleted the log and no host acquired it since, `offerInput` returns + `'deleted'` and stores nothing. +- If the `inputId` is new, `offerInput` stores a pending offer with `offeredAt` from the + store clock. It calls the listeners and returns `'offered'`. +- The harness of the owner names the log for `listLogsWithoutOwner`. A log with no owner + keeps the `harness` of its first offer. A later offer does not change it. +- If the `inputId` was offered before (pending or consumed), `offerInput` changes nothing. + It returns `'duplicate'` when the JSON of the record is equal, else `'conflict'`. +- If the record cannot become JSON, `offerInput` rejects and stores nothing. +- Offers belong to one log. The same `inputId` on two logs gives two offers. +- `listOfferedInputs` returns the pending offers in offer order. + +### Rules for snapshots, truncate, and delete + +A long log is slow to read from the start. A snapshot saves the state at one +position. Then a reader loads the snapshot and reads only the records after it. +With a snapshot, the owner can also drop the old records. + +- `putSnapshot` needs the owner, and it replaces the previous snapshot. If `seq` is + after the last position, or before `firstSeq - 1`, it rejects with `RangeError`. + `firstSeq` is the first kept position, so an earlier snapshot needs records that + a truncate dropped. +- `getSnapshot` returns the latest snapshot, or `null` when the log has none. +- `truncate` needs the owner. It drops the records with `seq` less than `beforeSeq`. + If the log has no snapshot with a `seq` of `beforeSeq - 1` or more, it rejects + with `RangeError`. +- After a truncate, the positions keep counting, so the next `append` is still at the + last position plus 1. A `read` from before the cut rejects with + `LogTruncatedError`, and its `firstSeq` is the first kept position. +- `deleteLog` removes the records, the offers, the snapshot, the truncation point, + the affinity, and the wake time, and calls the listeners. The epoch counter stays. + Every fenced write fails until the next `acquireOwnership`, also with `NO_OWNER_EPOCH`. So a + late write cannot bring a deleted transcript back. An `offerInput` returns `'deleted'` + and stores nothing. + +### In SQL + +Run each method that writes in one transaction. Three tables hold the records, the +ownership, and the offers: + +- `log_records`: the primary key `(log_id, seq)` refuses a taken position. +- `log_owner`: one row per log, with the epoch, the last position (`head`), and the + first kept position. `append`, `acquireOwnership`, and `deleteLog` insert the row with + `epoch` 0 when it is missing. Then one compare-and-set on `epoch` and `released` + fences every log: acquired, never acquired, and deleted. +- `log_offers`: the unique key `(log_id, input_id)` finds a duplicate offer. -Rules for an implementation: +```sql +CREATE TABLE log_records ( + log_id TEXT NOT NULL, + seq BIGINT NOT NULL, + record TEXT NOT NULL, + PRIMARY KEY (log_id, seq) +); -- `isAlive` returns `true` while another host still runs the attempt. That means a - lease exists and has not expired. -- `renew` and `release` for a lease you do not hold can do nothing. -- Key a lease by input id and attempt. Two attempts of one input are two leases. +CREATE TABLE log_owner ( + log_id TEXT PRIMARY KEY, + epoch BIGINT NOT NULL DEFAULT 0, -- 0 is NO_OWNER_EPOCH. It never goes down. + head BIGINT NOT NULL DEFAULT 0, -- the last position + first_seq BIGINT NOT NULL DEFAULT 1, -- the first kept position, after a truncate + harness TEXT, + owner_id TEXT, + expires_at TIMESTAMPTZ, + released BOOLEAN NOT NULL DEFAULT FALSE, + deleted BOOLEAN NOT NULL DEFAULT FALSE, -- set by deleteLog, cleared by acquire + affinity TEXT, + wake_at TIMESTAMPTZ +); -`memoryLeaseStore()` is the reference store. It is not part of `memoryPersistence()`, so add it yourself. This store does the same with a `Map`: +CREATE TABLE log_hosts ( + host_id TEXT PRIMARY KEY, + expires_at TIMESTAMPTZ NOT NULL -- the heartbeat expires +); -```ts -import type { - LeaseStore, - TurnLease, - TurnLeaseKey, -} from '@tanstack/ai-persistence' - -function createMemoryLeases(): LeaseStore { - const leases = new Map() - const keyOf = (key: TurnLeaseKey) => `${key.inputId}:${key.attempt}` - - return { - acquire: async (lease) => { - leases.set(keyOf(lease), lease) - }, - renew: async (lease) => { - leases.set(keyOf(lease), lease) - }, - release: async (lease) => { - leases.delete(keyOf(lease)) - }, - isAlive: async (key) => { - const lease = leases.get(keyOf(key)) - return lease !== undefined && lease.expiresAt >= Date.now() - }, - } -} +CREATE TABLE log_offers ( + id BIGSERIAL PRIMARY KEY, -- the offer order + log_id TEXT NOT NULL, + input_id TEXT NOT NULL, + harness TEXT NOT NULL, + record TEXT NOT NULL, + offered_at TIMESTAMPTZ NOT NULL DEFAULT now(), + consumed BOOLEAN NOT NULL DEFAULT FALSE, + UNIQUE (log_id, input_id) +); ``` -The conformance suite has no `leases` cases. The rules depend on the system that -issues the leases. +`append` does the fence, the position check, and the consume in the same +transaction as the insert: -## WorkClaimStore +```sql +BEGIN; +INSERT INTO log_owner (log_id) VALUES (:log_id) ON CONFLICT DO NOTHING; + +-- The fence and the position in one compare-and-set. +UPDATE log_owner SET head = :seq + :count - 1 +WHERE log_id = :log_id AND epoch = :epoch AND NOT released AND head = :seq - 1; +-- No row changed: roll back. Read the row, then reject with +-- OwnershipLostError (epoch or released) or LogConflictError (head). + +-- One row for each record of the batch. +INSERT INTO log_records (log_id, seq, record) VALUES (:log_id, :seq, :record); + +UPDATE log_offers SET consumed = TRUE +WHERE log_id = :log_id AND input_id = ANY(:consumes) AND NOT consumed; +-- Fewer rows changed than ids in :consumes: roll back, and reject with TypeError. +COMMIT; +``` -A host can stop while a thread still has work. Then another host must find that -thread and finish the work. A host claims a thread while the thread has work, and -renews the claim. When the host stops, the claim expires. Another host's sweep finds -the thread with `listExpired` and takes it over with `claim`. +`acquireOwnership` adds the row too, so a host can take a new log before its first record. +It reads the row first, because the result needs the old owner: -```ts -interface WorkClaimStore { - claim(entry: { - threadId: string - harness: string // the name of the setup that runs the thread - ownerId: string // the host that holds the claim - until: number // epoch ms - }): Promise - release(claim: { threadId: string; ownerId: string }): Promise - get(threadId: string): Promise<{ ownerId: string; until: number } | null> - listExpired(options: { - now: number // epoch ms - limit?: number - }): Promise> -} +```sql +BEGIN; +INSERT INTO log_owner (log_id) VALUES (:log_id) ON CONFLICT DO NOTHING; + +SELECT owner_id, released, NOT released AND expires_at > now() AS live +FROM log_owner WHERE log_id = :log_id FOR UPDATE; +-- Live, and owner_id is not :owner_id: roll back, and return null. +-- A live pin of another host (see below): roll back, and return null. +-- Live, and owner_id is :owner_id: set expires_at, and return { epoch }. + +UPDATE log_owner +SET epoch = epoch + 1, owner_id = :owner_id, harness = :harness, + expires_at = now() + :ttl_ms * interval '1 millisecond', + released = FALSE, deleted = FALSE, wake_at = NULL +WHERE log_id = :log_id +RETURNING epoch; +-- expired: true when the row that you read had an owner_id that is not +-- :owner_id, and released was FALSE. +COMMIT; ``` -Rules for an implementation: - -- `claim` is atomic. Two callers never both get `true` for one thread at one moment. -- `claim` returns `true` and sets `until` when nobody holds the thread, when - `ownerId` already holds it (a renewal), or when the claim expired (a takeover). -- `claim` returns `false` while another owner holds a claim that has not expired. -- `release` removes the claim only for its owner. For another `ownerId`, it does - nothing. -- `get` returns the owner and the `until` of the claim, or `null` when no one holds - the thread. It can return a claim that expired. -- `listExpired` returns the claims that expired before `now`, oldest first, at most - `limit`. Without `limit`, it returns all of them. +`deleted = FALSE` opens a deleted log again. For the other methods: -In SQL, one row per thread and a conditional upsert give the atomic rule. The `WHERE` -clause refuses a live claim of another owner: +- `pinToWorkers`, `putSnapshot`, and `truncate` need the owner. Lock the row with + `SELECT 1 FROM log_owner WHERE log_id = :log_id AND epoch = :epoch AND epoch > 0 + AND NOT released FOR UPDATE`. No row: reject with `OwnershipLostError`. +- `offerInput` first reads `deleted` with `SELECT deleted FROM log_owner WHERE log_id = + :log_id FOR SHARE`. `TRUE`: return `'deleted'`, and insert nothing. +- In `listLogsWithoutOwner`, the harness of a log is `log_owner.harness`. When it is `NULL`, + use the `harness` of the first offer of the log (the lowest `id`). +- Use the database clock (`now()`) for each expiry, wake time, and `offeredAt`. The + clocks of the hosts can disagree. +- `markHostAlive` is one upsert into `log_hosts`. In `acquireOwnership` and `listLogsWithoutOwner`, an + affinity counts only when its row has `expires_at > now()`: ```sql -CREATE TABLE work_claims ( - thread_id TEXT PRIMARY KEY, - harness TEXT NOT NULL, - owner_id TEXT NOT NULL, - until INTEGER NOT NULL -); - -INSERT INTO work_claims (thread_id, harness, owner_id, until) -VALUES (:thread_id, :harness, :owner_id, :until) -ON CONFLICT (thread_id) DO UPDATE - SET harness = excluded.harness, owner_id = excluded.owner_id, until = excluded.until - WHERE work_claims.owner_id = excluded.owner_id - OR work_claims.until <= :now; +INSERT INTO log_hosts (host_id, expires_at) +VALUES (:host_id, now() + :ttl_ms * interval '1 millisecond') +ON CONFLICT (host_id) DO UPDATE SET expires_at = EXCLUDED.expires_at; + +-- The pin of a log that `acquireOwnership` and `listLogsWithoutOwner` respect. +SELECT o.affinity FROM log_owner o +JOIN log_hosts h ON h.host_id = o.affinity AND h.expires_at > now() +WHERE o.log_id = :log_id; ``` -The claim is `true` when the statement wrote one row. An index on `until` keeps -`listExpired` fast. +- Keep the snapshot in one more table, with one row per log. +- `deleteLog` deletes the rows of the log in the other tables. It inserts the + `log_owner` row when it is missing, and sets `head = 0`, `first_seq = 1`, + `released = TRUE`, `deleted = TRUE`, and no `wake_at` or `affinity`. It keeps + `epoch`. The released row refuses every write, also with `NO_OWNER_EPOCH`, and + `deleted` refuses every offer, until the next `acquireOwnership`. That owner starts a new + log at position 1. +- Each record is JSON. Store it as text or as a JSON column. The store does not read + the records. -`memoryPersistence()` includes a memory `workClaims` store. It does not survive a -restart. +Your store is done when the +[conformance suite](#check-the-session-stores-with-the-conformance-suite) passes +with `twin`. ## SessionIndexStore Your app shows a list of sessions: the title, the owner, the time of the last change, and the token totals. Reading every transcript to build that list is slow. -`SessionIndexStore` keeps one small entry per thread. The index holds no messages. -The data of each thread stays in the stores that hold it. +`SessionIndexStore` keeps one small entry per tenant and thread. The index holds no +messages. The data of each thread stays in the stores that hold it. ```ts +interface SessionMember { + userId: string + role: 'owner' | 'member' | 'viewer' | 'operator' +} + +interface SessionIndexKey { + tenantId?: string // without it, the default tenant + threadId: string +} + interface SessionIndexEntry { + tenantId?: string // with `threadId`, the key of the entry threadId: string + members: Array // the users who can open the thread. The owner has the role `owner`. Empty for a subagent child harness?: string // the name of the setup that runs the thread title?: string parentThreadId?: string // the parent thread. A subagent child's own threadId is `subagent:` @@ -876,7 +1049,6 @@ interface SessionIndexEntry { createdAt: number // epoch ms, the first open updatedAt: number // epoch ms, the last change pinned?: boolean - principal?: { id: string; tenantId?: string } // the owner usage?: { turns: number // model calls promptTokens: number @@ -890,10 +1062,11 @@ interface SessionIndexEntry { } interface SessionIndexListOptions { + tenantId?: string // one tenant per list. Without it, the default tenant + memberId?: string // only the entries with this user in `members` limit?: number cursor?: string parentThreadId?: string | null - principal?: { id: string; tenantId?: string } search?: string harness?: string metadata?: Record @@ -906,81 +1079,94 @@ interface SessionIndexPage { } interface SessionIndexStore { - upsert(entry: SessionIndexEntry): Promise - get(threadId: string): Promise - list(options?: SessionIndexListOptions): Promise - delete(threadId: string): Promise - // Optional: compare-and-set. Add both, or neither. - getVersioned?( - threadId: string, + get(key: SessionIndexKey): Promise + getVersioned( + key: SessionIndexKey, ): Promise<{ entry: SessionIndexEntry; revision: string } | undefined> - upsertIf?(write: { + upsert(entry: SessionIndexEntry): Promise + upsertIf(input: { entry: SessionIndexEntry - expectedRevision: string | null + revision: string | null }): Promise<{ ok: true; revision: string } | { ok: false; reason: 'conflict' }> + list(options?: SessionIndexListOptions): Promise + delete(key: SessionIndexKey): Promise } ``` -Rules for `upsert`, `get`, and `delete`: +The key of an entry is its `tenantId` and its `threadId` together. A key without +`tenantId` is in the default tenant. So one thread id in two tenants is two entries. +In SQL, store the default tenant as `''`, and make `(tenant_id, thread_id)` the +primary key. + +Rules for `get`, `upsert`, and `delete`: -- `upsert` replaces the whole entry for `entry.threadId`. A field that the new entry - does not have is gone. Do not merge. To change one field, read the entry and write - a changed copy. -- `get` returns `undefined` for an unknown thread, not `null`. -- `delete` removes the entry only. The messages, the log, and the other data of the - thread stay. For an unknown thread, `delete` does nothing. +- `get` returns `undefined` for an unknown key, not `null`. +- `upsert` replaces the whole entry for the key of `entry`. A field that the new + entry does not have is gone. Do not merge. To change one field, read the entry and + write a changed copy with `upsertIf`. +- Keep `members` as they are written. The host decides what each role can do. +- `delete` is a hard delete. After it, `get` and `getVersioned` return `undefined`, + and `upsertIf` with revision `null` writes a new entry. The messages, the log, and + the other data of the thread stay. For an unknown key, `delete` does nothing. Rules for `list`: +- Return the entries of one tenant only: the `tenantId` of the options, or the + default tenant without it. Never return an entry of another tenant. - Sort by `updatedAt`, the newest first. Entries with the same `updatedAt` go in `threadId` order. `pinned` gets no special place. +- `memberId` returns only the entries whose `members` have that user id, with any + role. A subagent child has no members, so it never matches. - `parentThreadId: 'x'` returns only the entries whose parent is `x`. `null` returns - only the entries with no parent. -- `principal` returns only the entries of that owner id. With a `tenantId`, the - tenant must match too. An entry without a `principal` never matches. -- `search` returns only the entries whose `title` contains the text, without case. - `harness` and `metadata` match exact strings. All filters must match. -- With `limit`, when more entries match, return `truncated: true` and a `cursor`. The - next call with that `cursor` returns the entries after the last entry of the page. - -With `stores.sessions`, `withPersistence` writes an entry for each subagent child. The -entry has the parent thread, the tool call that started the child, and the owner and -`harness` of the parent entry. + only the entries with no parent. `search` returns only the entries whose `title` + contains the text, without case. `harness` and `metadata` match exact strings. +- All filters must match. With `limit`, when more entries match, return + `truncated: true` and a `cursor`. The next call with that `cursor` returns the + entries after the last entry of the page. + +With `stores.sessions`, `withPersistence` writes an entry for each subagent child, in +the tenant of its `tenantId` option. The entry has the parent thread, the tool call +that started the child, and the `harness` of the parent entry. It has no members: +access to a child goes through its top-level thread. ### Keep the writes of several hosts Two hosts can write one entry at the same time. For example, one host writes the -usage of a turn while another host writes the pinned files of the app. Without -`getVersioned` and `upsertIf`, the second write can remove the change of the first. +usage of a turn while another host writes the pinned files of the app. The host +changes an entry with `getVersioned` and `upsertIf`, so the second write never removes +the change of the first. Both methods are required: without them, `createHarnessHost` +throws. Rules for `getVersioned` and `upsertIf`: - `getVersioned` returns the entry and its revision, or `undefined` for an unknown - thread. The format of the revision is yours. + key. The format of the revision is yours. - A write that changes the entry gives it a new revision: `upsert` and `upsertIf` both. A revision can be a hash of the content (an ETag of S3 or Netlify Blobs). Then the same entry, written again after a `delete`, gets the same revision back. -- `upsertIf` writes only when the current revision is `expectedRevision`, and returns - `{ ok: true, revision }`. Pass `null` to write only when the thread has no entry. +- `upsertIf` writes only when the current revision of the key is `revision`, and + returns `{ ok: true, revision }` with the new revision. Pass `null` to write only + when the key has no entry. - On any other revision, `upsertIf` writes nothing and returns `{ ok: false, reason: 'conflict' }`. In SQL, keep a revision column. Then: -- For a revision, write with `UPDATE ... WHERE thread_id = ? AND revision = ?`. - When no row changed, return a conflict. +- For a revision, write with + `UPDATE ... WHERE tenant_id = ? AND thread_id = ? AND revision = ?`. When no row + changed, return a conflict. - For `null`, write with `INSERT ... ON CONFLICT DO NOTHING`. When no row was added, return a conflict. -`memoryPersistence()` has both methods. Its cursor is `:` of the +`memoryPersistence()` has every method. Its cursor is `:` of the last entry of a page. ### Check the session stores with the conformance suite -`runPersistenceConformance` runs the cases of `inbox`, `workClaims`, -`log`, and `sessions` when your persistence has that store. A persistence without one -needs no `skip` entry for it. The compare-and-set cases run only when `sessions` has -`getVersioned` and `upsertIf`. +`runPersistenceConformance` runs the cases of `log` and `sessions` when your +persistence has that store. A persistence without one +needs no `skip` entry for it. The `sessions` cases also check the tenants, the +`memberId` filter, the hard delete, and compare-and-set. To check one session store alone, skip the stores that the suite reads by default: @@ -1006,6 +1192,48 @@ runPersistenceConformance( ) ``` +Some `log` cases need two machines on one backend: + +- A listener on one instance runs after an append on the other. +- Only one instance gets the log from `acquireOwnership`. +- An offer on one instance is a `'duplicate'` on the other. +- A listener that throws does not stop the listeners of the other instance. + +Pass `twin`: a function that returns a second store set on the same backend. Its `log` +must be a new store instance, not the same object. If your persistence has `log` and +no `twin`, these cases fail. + +```ts +import { runPersistenceConformance } from '@tanstack/ai-persistence/testkit' +import { db } from './db' +import { myLogStore } from './log-store' + +runPersistenceConformance( + 'my log store', + () => ({ stores: { log: myLogStore(db) } }), + { + skip: [ + 'messages', + 'activities', + 'runs', + 'interrupts', + 'metadata', + 'generationRuns', + 'artifacts', + 'blobs', + ], + // A second store on the same database acts as a second machine. + twin: (persistence) => ({ + stores: { ...persistence.stores, log: myLogStore(db) }, + }), + }, +) +``` + +For the memory store, share one `memoryLogState()`. Each +`memoryLogStore({ state })` call on it gives one more store instance on that +state. + ## How the records relate The thread is not a table of its own: it exists as the `thread_id` the other records @@ -1064,8 +1292,7 @@ erDiagram Each store has a `define*Store` helper (`defineMessageStore`, `defineRunStore`, `defineInterruptStore`, `defineMetadataStore`, `defineActivityStore`, `defineGenerationRunStore`, `defineArtifactStore`, -`defineBlobStore`, `defineInboxStore`, `defineLogStore`, -`defineWorkClaimStore`, `defineSessionIndexStore`, `defineLeaseStore`). They compose into `defineAIPersistence`, which tracks +`defineBlobStore`, `defineLogStore`, `defineSessionIndexStore`). They compose into `defineAIPersistence`, which tracks exact presence: the stores you passed autocomplete on `persistence.stores`, and reading one you did not pass is a compile error. @@ -1083,4 +1310,4 @@ To annotate the value instead, use a named shape: - [Build a chat adapter](./build-your-own-chat-adapter): these contracts implemented against SQLite. - [Build a generation adapter](./build-your-own-generation-adapter): the generation half. -- [Build your own adapter](./build-your-own-adapter#verify-with-the-conformance-suite): check an implementation against the conformance suite. +- [Build your own adapter](./build-your-own-adapter#prove-it-with-the-conformance-suite): check an implementation against the conformance suite. diff --git a/docs/sandbox/auth.md b/docs/sandbox/auth.md index 5f0ef5b5ee..2a69352af6 100644 --- a/docs/sandbox/auth.md +++ b/docs/sandbox/auth.md @@ -51,8 +51,21 @@ const stream = chat({ }) ``` -`scrubEnv` removes keys the host process inherited. If the CLI sees -`XAI_API_KEY`, it can prefer that key over your login. +`scrubEnv` removes keys from the env of each command, for example a key that +a workspace secret sets. If the CLI sees `XAI_API_KEY`, it can prefer that key +over your login. The keys of the host process do not reach the command. + +The command gets only a few variables of the host process. If your login or +your network needs more, for example `CLAUDE_CONFIG_DIR`, `CODEX_HOME`, +`HTTPS_PROXY`, or `SSH_AUTH_SOCK`, name them in `passEnv`: + +```ts +import { localProcessSandbox } from '@tanstack/ai-sandbox-local-process' + +const provider = localProcessSandbox({ + passEnv: ['CODEX_HOME', 'HTTPS_PROXY'], +}) +``` ## API key @@ -100,8 +113,9 @@ const stream = chat({ | [Codex](../adapters/codex) | `CODEX_API_KEY` | `codex login` | | [ACP-Compatible](../adapters/acp-compatible) | set `authMethodId` (for Grok, `xai.api_key`) | skip ACP `authenticate` | -OpenCode still reads `OPENAI_API_KEY` from the process env. It has no `authMode` -flag. +OpenCode reads `OPENAI_API_KEY` from its env. It has no `authMode` flag. On +local-process, the key of the host process does not reach it: set the key as a +workspace secret or with `env.set`. ## Client and server @@ -162,4 +176,4 @@ export async function POST(request: Request) { ``` See [Harnesses](./harnesses) for which adapter to pick, and [Providers](./providers) -for `scrubEnv` on local-process. +for `scrubEnv` and `passEnv` on local-process. diff --git a/docs/sandbox/providers.md b/docs/sandbox/providers.md index 8eaa138c18..821d9d27e3 100644 --- a/docs/sandbox/providers.md +++ b/docs/sandbox/providers.md @@ -78,26 +78,29 @@ import { localProcessSandbox } from '@tanstack/ai-sandbox-local-process' const dev = localProcessSandbox() ``` -- **Isolation:** none. The harness runs directly on your host, inheriting your - host environment. Use it for trusted or dev work only. There is no boundary - between the agent and your machine. -- **Auth / env:** inherits the host environment. Set `authMode` on the harness +- **Isolation:** none. The harness runs directly on your host. Use it for + trusted or dev work only. There is no boundary between the agent and your + machine. +- **Auth / env:** a command gets only a few variables of the host process + (`PATH`, `HOME`, `LANG`, `LC_ALL`, `TMPDIR`, `TEMP`, `TMP`, and on Windows + `SystemRoot`, `ComSpec`, `PATHEXT`, `WINDIR`, `USERPROFILE`), plus the + workspace secrets and the env of the call. Set `authMode` on the harness (`'host'` or `'api-key'`). The provider does not pick this. See [Harness Auth](./auth). - **Snapshot / resume:** no snapshots and no durable resume-by-id; each run re-creates and re-bootstraps under the same identity. The snapshot step is skipped silently (see [Capabilities](#capabilities)). -### Host login vs API key (`scrubEnv`) +### Host login vs API key (`scrubEnv`, `passEnv`) The provider is where the agent runs, not how it signs in. The default `authMode` is `'api-key'`. Set `'host'` when the machine already has a CLI login. A local-process run can be your laptop or a GitHub runner. See [Harness Auth](./auth). -`localProcessSandbox` inherits the host environment, including any API keys -exported there. If you set `authMode: 'host'`, pass `scrubEnv` so those keys -do not override the CLI login: +An API key that the host process exports does not reach a command. A key +that a workspace secret or the call sets does. If you set `authMode: 'host'`, +pass `scrubEnv` so those keys do not override the CLI login: ```ts import { localProcessSandbox } from '@tanstack/ai-sandbox-local-process' @@ -107,6 +110,18 @@ const hostLogin = localProcessSandbox({ }) ``` +A command gets only a few variables of the host process: `PATH`, `HOME`, +`LANG`, `LC_ALL`, `TMPDIR`, `TEMP`, and `TMP` (and a few more on Windows). +When a CLI login or the network needs more, name them in `passEnv`: + +```ts +import { localProcessSandbox } from '@tanstack/ai-sandbox-local-process' + +const behindProxy = localProcessSandbox({ + passEnv: ['CLAUDE_CONFIG_DIR', 'HTTPS_PROXY', 'NO_PROXY', 'NODE_EXTRA_CA_CERTS'], +}) +``` + If the same local-process sandbox runs on a CI machine, set `authMode: 'api-key'`. Then inject the key as a workspace secret. Isolated and cloud providers have no host CLI login. Use `authMode: 'api-key'` and diff --git a/docs/sandbox/quick-start.md b/docs/sandbox/quick-start.md index ab43c418b3..0b93610b22 100644 --- a/docs/sandbox/quick-start.md +++ b/docs/sandbox/quick-start.md @@ -184,7 +184,7 @@ const stream = chat({ ``` Set `authMode: 'host'` so the adapter uses `grok login`. `scrubEnv` removes -keys the host process inherited. Those keys would override the login. +keys from the env of each command. Those keys would override the login. A local-process run can also be a CI runner. That machine has no browser login. Set `authMode: 'api-key'`. Then inject `XAI_API_KEY` as a workspace diff --git a/docs/tools/server-tools.md b/docs/tools/server-tools.md index 9106cb0e00..e3d290d0c3 100644 --- a/docs/tools/server-tools.md +++ b/docs/tools/server-tools.md @@ -359,7 +359,7 @@ Malformed JSON arguments and input validation failures (Standard Schema or JSON The `inputSchema` checks the input just before the tool runs: -- The check runs after approval edits and after middleware `onBeforeToolCall`. An approval edit that does not fit the schema stops the resume with an `invalid-edited-args` error. A middleware change that does not fit becomes a tool error. +- The check runs after approval edits and after middleware `onBeforeToolCall`. An approval edit or a middleware change that does not fit the schema becomes a tool error. The resume goes on, the tool does not run, and the model gets the input error as the result of that call. - An async validator is awaited. - The tool gets the schema's output, so a transform such as `z.string().transform(Number)` applies once. diff --git a/examples/harness-cli/src/tui.tsx b/examples/harness-cli/src/tui.tsx index 53b8b89694..3e942440cd 100644 --- a/examples/harness-cli/src/tui.tsx +++ b/examples/harness-cli/src/tui.tsx @@ -104,7 +104,9 @@ function Screen({ view }: { view: SessionView }) { .answer(answerOf(text, question.schema)) .then((receipt) => { if (receipt.status === 'rejected') { - view.notice(`Answer again: ${receipt.reason ?? 'not accepted'}`) + view.notice({ + text: `Answer again: ${receipt.reason ?? 'not accepted'}`, + }) } }) } @@ -168,7 +170,35 @@ function Screen({ view }: { view: SessionView }) { ) } +/** The ids of the child agent parts in `parts`, at any depth. */ +function agentIds(parts: ReadonlyArray): Array { + return parts.flatMap((part) => + part.type === 'agent' ? [part.id, ...agentIds(part.parts)] : [], + ) +} + +/** + * Fill the saved child agents of a reopened thread. Ink prints the old turns + * once, so they load before the first render. A loaded child can have saved + * children of its own, so it loads again until no new id shows up. + */ +async function loadSavedAgents(view: SessionView) { + const loaded = new Set() + for (;;) { + const ids = view.store + .get() + .messages.flatMap((message) => + message.role === 'assistant' ? agentIds(message.parts) : [], + ) + .filter((id) => !loaded.has(id)) + if (ids.length === 0) return + for (const id of ids) loaded.add(id) + await Promise.all(ids.map((id) => view.loadAgent({ id }))) + } +} + /** The `ui` of runCli: show the screen until the user quits. */ export async function runTui(view: SessionView) { + await loadSavedAgents(view) await render().waitUntilExit() } diff --git a/knip.json b/knip.json index bcaf1c98b0..cc250d19a7 100644 --- a/knip.json +++ b/knip.json @@ -5,7 +5,7 @@ "@aws-sdk/client-bedrock", "@typescript/typescript6" ], - "ignoreBinaries": ["taskkill"], + "ignoreBinaries": ["taskkill", "ps"], "ignoreWorkspaces": [ "examples/**", "testing/**", diff --git a/packages/ai-acp/package.json b/packages/ai-acp/package.json index 4893a07c80..47a45e5bb8 100644 --- a/packages/ai-acp/package.json +++ b/packages/ai-acp/package.json @@ -48,7 +48,7 @@ "test:types": "tsc" }, "dependencies": { - "@agentclientprotocol/sdk": "^1.5.0" + "@agentclientprotocol/sdk": "~1.8.0" }, "peerDependencies": { "@tanstack/ai": "workspace:^", diff --git a/packages/ai-acp/src/agent/index.ts b/packages/ai-acp/src/agent/index.ts index cc43fdde84..a23ee5bb71 100644 --- a/packages/ai-acp/src/agent/index.ts +++ b/packages/ai-acp/src/agent/index.ts @@ -1,4 +1,4 @@ -import { EventType } from '@tanstack/ai' +import { EventType, INTERRUPT_PAYLOAD_METADATA_KEY } from '@tanstack/ai' import { ContentBlock, PROTOCOL_VERSION, @@ -7,15 +7,17 @@ import { ndJsonStream, } from '@agentclientprotocol/sdk/experimental/v2' import { + AccessDeniedError, InputRejectedError, - MediaError, - kindOf, - mediaPart, + waitFromInterrupt, } from '@tanstack/ai-harness' +import { MediaError, kindOf, mediaPart } from '@tanstack/ai-harness/media' import type { AgentApp, AgentContext, AgentConnection, + ElicitationSchema, + PermissionOption, SessionUpdate, Stream, } from '@agentclientprotocol/sdk/experimental/v2' @@ -30,11 +32,20 @@ import type { HarnessHost, HarnessSession, Operation, + HarnessUser, + Wait, } from '@tanstack/ai-harness' -export interface AcpAgentOptions { - host: HarnessHost +export interface AcpAgentOptions { + host: HarnessHost harness: AnyHarness + /** + * The user that the agent serves: an editor starts the agent for one + * local user. Each session opens as this user, and `session/list` + * shows the sessions of this user. Required when the host has + * `access`. + */ + user?: U /** Reported in `initialize`. Default `'0.0.0'`. */ version?: string /** @@ -79,7 +90,9 @@ async function storedPart( `The data of ${info.name} is not base64.`, ) } - return mediaPart(await session.putMedia(Buffer.from(data, 'base64'), info)) + return mediaPart( + await session.putMedia({ body: Buffer.from(data, 'base64'), ...info }), + ) } /** One ACP prompt block as a content part, or `undefined` to skip it. */ @@ -216,53 +229,243 @@ function parseArgs(raw: string | undefined): unknown { } } -/** True for an interrupt that asks to approve a tool call. */ -function isApproval(interrupt: Interrupt) { - return ( - interrupt.metadata?.kind === 'approval' || - interrupt.reason === 'approval_required' +/** + * The tool call of an interrupt: its own `toolCallId`, or for an approval of + * `permissions()`, the `toolCallId` in its payload. Core gives such an + * approval no `toolCallId` of its own. + */ +function toolCallIdOf(interrupt: Interrupt) { + if (interrupt.toolCallId) return interrupt.toolCallId + const payload = interrupt.metadata?.[INTERRUPT_PAYLOAD_METADATA_KEY] + return isRecord(payload) && typeof payload.toolCallId === 'string' + ? payload.toolCallId + : undefined +} + +/** The sign-in page of a sign-in interrupt, when it has one. */ +function signInUrl(interrupt: Interrupt) { + const payload = interrupt.metadata?.[INTERRUPT_PAYLOAD_METADATA_KEY] + const request = isRecord(payload) ? payload.request : undefined + return isRecord(request) && typeof request.url === 'string' + ? request.url + : undefined +} + +/** The JSON Schema of a wait, as a record, when it has one. */ +function schemaOf(wait: Wait) { + return isRecord(wait.schema) ? wait.schema : undefined +} + +/** + * True when the answer schema of an approval takes `always` (an approval of + * `permissions()`): the editor can then allow the call from now on. + */ +function takesAlways(schema: unknown): boolean { + if (!isRecord(schema)) return false + if (Array.isArray(schema.enum) && schema.enum.includes('always')) return true + const branches = [schema.anyOf, schema.oneOf].flatMap((list) => + Array.isArray(list) ? list : [], ) + const answer = isRecord(schema.properties) + ? schema.properties.answer + : undefined + return [...branches, answer].some(takesAlways) +} + +/** The permission options of an approval: allow once, maybe always, reject. */ +function approvalOptions(wait: Wait): Array { + return [ + { optionId: 'allow', name: 'Allow', kind: 'allow_once' }, + ...(takesAlways(wait.schema) + ? [ + { + optionId: 'always', + name: 'Always allow', + kind: 'allow_always', + } as const, + ] + : []), + { optionId: 'reject', name: 'Reject', kind: 'reject_once' }, + ] } -const APPROVAL_OPTIONS = [ - { optionId: 'allow', name: 'Allow', kind: 'allow_once' }, - { optionId: 'reject', name: 'Reject', kind: 'reject_once' }, -] as const +/** The answer to an approval for the option the editor selected. */ +function approvalAnswer(optionId: unknown) { + if (optionId === 'always') return { answer: 'always' } + return optionId === 'allow' +} + +/** + * The form of an elicitation for the answer schema of a wait. An object + * schema is the form. Any other answer is the one field `answer`. + */ +function formOf(wait: Wait) { + const schema = schemaOf(wait) + const isForm = schema?.type === 'object' && isRecord(schema.properties) + const requestedSchema = ( + isForm + ? schema + : { + type: 'object', + properties: { answer: schema ?? { type: 'string' } }, + required: ['answer'], + } + ) as ElicitationSchema + return { + requestedSchema, + answerOf: (content: unknown) => + isForm || !isRecord(content) ? content : content.answer, + } +} + +/** The waits of the stopped turn of `session`. */ +function turnWaitsOf(session: HarnessSession) { + return session.snapshot().pendingWaits.filter(({ turn }) => turn) +} + +function isRecord(value: unknown): value is Record { + return typeof value === 'object' && value !== null +} + +/** + * The answer of the editor to one wait: `resolved` with its payload, + * `cancelled` (the editor said no, or cannot answer it), or `stop` (the + * editor cancelled the prompt). + */ +type WaitAnswer = + | { status: 'resolved'; payload: unknown } + | { status: 'cancelled' } + | { status: 'stop' } /** * An ACP v2 agent backed by a harness. Each ACP session is one harness * session, and `session/list` reads the top-level sessions of the harness from * the session index of the host. The prompts of a session run one after - * another, whatever the harness `busy` setting says. Tool approvals become - * `session/request_permission` requests, and cancel cancels the running - * turn. Image, audio, and embedded resource blocks of a prompt go to the media + * another, whatever the harness `busy` setting says. Each wait of a turn + * goes to the editor by its kind: an approval becomes a + * `session/request_permission` request (with `allow_always` for an approval + * of `permissions()`), a question or a `human` wait a form elicitation, and + * a sign-in a URL elicitation, when the editor declares them in + * `initialize`. A wait that the editor cannot answer is cancelled. Cancel + * cancels the running turn. Image, audio, and embedded resource blocks of a prompt go to the media * store of the session. A file the harness refuses (too big, or a type it * does not take) fails `session/prompt` with an invalid params error. * + * Each session opens as `user`. On a host with `access`, the host + * checks each action, and a session that the user may not read is not + * found. Without `user`, a host with access throws at the start. + * + * The agent holds a handle on a session only while the session has work: a + * new session, or prompts that run or wait in its queue. When the queue is + * empty, the agent releases the handle, and the host closes the session when + * it is idle. The next prompt opens it again. `session/close` and the close + * of the connection release the handles too. `session/resume` only checks the + * session index: the session opens at its first prompt. + * + * One app serves one connection: it keeps the session handles and prompt + * queues of that connection, and a second connection to it throws. For more + * connections, make one app for each, for example + * `new AcpServer({ createAgent: () => createAcpAgent(options) })`. + * * ACP v2 is a draft, so this API is experimental too. */ -export function createAcpAgent(options: AcpAgentOptions): AgentApp { - const { host, harness } = options - /** The ids of the sessions that this agent opened. */ - const opened = new Set() - /** The last pump of each session. The pumps of one session run in order. */ +export function createAcpAgent( + options: AcpAgentOptions, +): AgentApp { + const { host, harness, user } = options + if (user === undefined && host.hasAccess) { + throw new Error( + 'createAcpAgent needs a user: the host has access. Pass the local user as the user option.', + ) + } + /** + * The handle of each session that this agent holds, by session id. The + * agent holds one while the session has work, and releases it when the + * prompt queue of the session is empty. `session/close` releases it too, + * and so does the close of the connection. + */ + const handles = new Map>() + /** + * The last prompt of each session. The prompts of one session run in the + * order that they came in. + */ const tails = new Map>() + /** + * The sessions that this agent opened in this connection. They are known + * without the session index, so a host with no `stores.sessions` still + * takes their prompts after the agent released their handles. + */ + const opened = new Set() + + /** The handle of a session: the one that the agent holds, or a new one. */ + const live = (sessionId: string) => { + const held = handles.get(sessionId) + if (held) return held + const opening = host.open(harness, { threadId: sessionId, user }) + handles.set(sessionId, opening) + // A failed open holds nothing, so the next request opens again. + opening.then( + () => opened.add(sessionId), + () => { + if (handles.get(sessionId) === opening) handles.delete(sessionId) + }, + ) + return opening + } + + /** Release the handle of a session. The session closes on idle. */ + const release = async (sessionId: string) => { + const held = handles.get(sessionId) + if (!held) return + handles.delete(sessionId) + await held.then( + (session) => session.close(), + () => {}, + ) + } + + /** + * Release the handle of a session when no prompt of it runs or waits. An + * idle session holds nothing, and the next prompt opens it again. + */ + const releaseIfIdle = (sessionId: string) => { + if (tails.has(sessionId)) return + void release(sessionId) + } - // The host gives back the live session of a thread, or opens it again after - // it closed, so the agent keeps no session objects. - const live = (sessionId: string) => - host.open(harness, { threadId: sessionId }) + /** + * Throw unless this agent or the session index knows the id. The agent + * knows the sessions that it opened in this connection. A subagent + * child is not a session of its own: ACP reaches it through its parent. A + * session that the user may not read gets the same answer as an + * unknown id. It reads the index only: no session opens. + */ + const checkKnown = async (sessionId: string) => { + if (handles.has(sessionId) || opened.has(sessionId)) return + try { + const entry = await host.sessions.get({ threadId: sessionId, user }) + const isOwnSession = + entry?.harness === harness.name && entry.parentThreadId === undefined + if (!isOwnSession) throw RequestError.resourceNotFound(sessionId) + } catch (error) { + if (error instanceof AccessDeniedError) { + throw RequestError.resourceNotFound(sessionId) + } + throw error + } + } /** The session of an id that this agent or the session index knows. */ const known = async (sessionId: string) => { - if ( - !opened.has(sessionId) && - (await host.sessions.get(sessionId))?.harness !== harness.name - ) { - throw RequestError.resourceNotFound(sessionId) + await checkKnown(sessionId) + try { + return await live(sessionId) + } catch (error) { + if (error instanceof AccessDeniedError) { + throw RequestError.resourceNotFound(sessionId) + } + throw error } - opened.add(sessionId) - return live(sessionId) } const notify = ( @@ -272,13 +475,101 @@ export function createAcpAgent(options: AcpAgentOptions): AgentApp { ) => client.notify('session/update', { sessionId, update }).catch(() => {}) /** - * Answer the interrupts of a turn, then continue it. The client gets a - * permission request for each tool approval. The other interrupts (a - * sign-in, an input form, a client tool) are cancelled, because the client - * cannot answer them. A permission answer other than a selected option (the - * client cancelled the prompt) stops the turn, and the interrupts stay open - * until the next prompt. The next operation is wrapped, because an - * operation is thenable and an async function would unwrap it. + * True when the editor can answer `wait` (see `askEditor`). What the editor + * can show comes from the `initialize` of its own connection. + */ + const canAsk = async ( + client: AgentContext, + wait: Wait, + url: string | undefined, + ) => { + const { request } = await client.initialized + const elicitation = request.capabilities?.elicitation + switch (wait.kind) { + case 'approval': + return true + case 'question': + case 'human': + return Boolean(elicitation?.form) + case 'sign-in': + return url !== undefined && Boolean(elicitation?.url) + case 'client-tool': + return false + } + } + + /** + * Ask the editor for the answer to `wait`, by its kind: + * + * - `approval`: a permission request. `allow_always` only when the + * answer schema takes `always`. + * - `question` and `human`: a form elicitation of the wait schema. + * - `sign-in`: a URL elicitation of the sign-in page. + * - `client-tool`, and a kind that the editor cannot show: cancelled. + */ + const askEditor = async ({ + client, + sessionId, + wait, + toolCallId, + url, + }: { + client: AgentContext + sessionId: string + wait: Wait + toolCallId?: string + url?: string + }): Promise => { + if (!(await canAsk(client, wait, url))) return { status: 'cancelled' } + if (wait.kind === 'approval') { + const { outcome } = await client.request('session/request_permission', { + sessionId, + title: wait.message ?? 'Allow this tool call?', + subject: { + type: 'tool_call', + toolCall: { toolCallId: toolCallId ?? wait.waitId }, + }, + options: approvalOptions(wait), + }) + if (outcome.outcome !== 'selected' || !('optionId' in outcome)) { + return { status: 'stop' } + } + return { status: 'resolved', payload: approvalAnswer(outcome.optionId) } + } + const scope = { sessionId, ...(toolCallId ? { toolCallId } : {}) } + if (wait.kind === 'question' || wait.kind === 'human') { + const form = formOf(wait) + const reply = await client.request('elicitation/create', { + ...scope, + mode: 'form', + message: wait.message ?? 'The agent asks for input.', + requestedSchema: form.requestedSchema, + }) + if (reply.action === 'cancel') return { status: 'stop' } + if (reply.action !== 'accept') return { status: 'cancelled' } + const content = 'content' in reply ? reply.content : undefined + return { status: 'resolved', payload: form.answerOf(content) } + } + const reply = await client.request('elicitation/create', { + ...scope, + mode: 'url', + elicitationId: wait.waitId, + url: url ?? '', + message: wait.message ?? 'Sign in to go on.', + }) + if (reply.action === 'cancel') return { status: 'stop' } + return reply.action === 'accept' + ? { status: 'resolved', payload: true } + : { status: 'cancelled' } + } + + /** + * Answer the interrupts of a turn, then continue it. The editor answers + * each wait of its kind (see `askEditor`). A wait that it cannot answer + * (a client tool) is cancelled. A cancelled request (the editor cancelled + * the prompt) stops the turn, and the interrupts stay open until the next + * prompt. The next operation is wrapped, because an operation is thenable + * and an async function would unwrap it. */ const answer = async ( client: AgentContext, @@ -286,33 +577,42 @@ export function createAcpAgent(options: AcpAgentOptions): AgentApp { session: HarnessSession, interrupts: ReadonlyArray, ): Promise<{ next?: Operation; cancelled?: boolean }> => { - if (interrupts.some(isApproval)) { + const items = interrupts.map((interrupt) => ({ + interrupt, + wait: waitFromInterrupt({ interrupt, runId: sessionId }), + url: signInUrl(interrupt), + })) + const askable = await Promise.all( + items.map(({ wait, url }) => canAsk(client, wait, url)), + ) + if (askable.includes(true)) { await notify(client, sessionId, { sessionUpdate: 'state_update', state: 'requires_action', }) } const resume: Array = [] - for (const interrupt of interrupts) { - if (!isApproval(interrupt)) { - resume.push({ interruptId: interrupt.id, status: 'cancelled' }) - continue - } - const toolCallId = interrupt.toolCallId ?? interrupt.id - const { outcome } = await client.request('session/request_permission', { + for (const { interrupt, wait, url } of items) { + const toolCallId = toolCallIdOf(interrupt) + const reply = await askEditor({ + client, sessionId, - title: interrupt.message ?? 'Allow this tool call?', - subject: { type: 'tool_call', toolCall: { toolCallId } }, - options: [...APPROVAL_OPTIONS], - }) - if (outcome.outcome !== 'selected') return { cancelled: true } - resume.push({ - interruptId: interrupt.id, - status: 'resolved', - payload: 'optionId' in outcome && outcome.optionId === 'allow', + wait, + ...(toolCallId ? { toolCallId } : {}), + ...(url ? { url } : {}), }) + if (reply.status === 'stop') return { cancelled: true } + resume.push( + reply.status === 'resolved' + ? { + interruptId: interrupt.id, + status: 'resolved', + payload: reply.payload, + } + : { interruptId: interrupt.id, status: 'cancelled' }, + ) } - const receipt = await session.resolve(resume) + const receipt = await session.resolve({ resume }) // For example `busy`: a later prompt runs already. if (receipt.status === 'rejected') throw new InputRejectedError(receipt) return { @@ -322,6 +622,50 @@ export function createAcpAgent(options: AcpAgentOptions): AgentApp { } } + /** + * Answer each question that the operation `operationId` asks while it + * runs (a `step.ask` of a durable tool, or an MCP server that asks), with + * a form elicitation. A question that the editor cannot or will not + * answer cancels the operation. Stops when `signal` aborts, or when the + * operation stops for interrupts: the pump answers those. + */ + const answerQuestions = async ({ + client, + sessionId, + session, + operationId, + signal, + }: { + client: AgentContext + sessionId: string + session: HarnessSession + operationId: string + signal: AbortSignal + }) => { + while (!signal.aborted) { + const wait = await session + .waitForRun({ runId: operationId, signal }) + .catch(() => undefined) + if (!wait) return + // A question that no turn holds takes an answer. + const isQuestion = session + .snapshot() + .pendingWaits.some( + ({ waitId, kind, turn }) => + !turn && kind === 'question' && waitId === wait.waitId, + ) + if (!isQuestion) return + const reply = await askEditor({ client, sessionId, wait }) + if (signal.aborted) return + if (reply.status !== 'resolved') { + await session.cancel({ operationId }) + return + } + // A refused answer (it does not fit the schema) asks again. + await session.answer({ questionId: wait.waitId, value: reply.payload }) + } + } + /** * Cancel the open interrupts of a session (from a cancelled approval, or * from another client), and wait for the turn that takes the answers. The @@ -329,14 +673,14 @@ export function createAcpAgent(options: AcpAgentOptions): AgentApp { * when the client cancelled that turn: its interrupts stay open. */ const closeInterrupts = async (session: HarnessSession) => { - const open = session.snapshot().pendingInterrupts + const open = turnWaitsOf(session) if (open.length === 0) return true - const receipt = await session.resolve( - open.map((interrupt) => ({ - interruptId: interrupt.id, + const receipt = await session.resolve({ + resume: open.map(({ waitId }) => ({ + interruptId: waitId, status: 'cancelled' as const, })), - ) + }) if (receipt.status === 'rejected') throw new InputRejectedError(receipt) // ponytail: the reply of the model to the cancel stays out of the client. const turn = receipt.operationId @@ -351,7 +695,7 @@ export function createAcpAgent(options: AcpAgentOptions): AgentApp { ) if (turn?.status() === 'cancelled') return false if (turn?.status() === 'failed') throw failure - if (session.snapshot().pendingInterrupts.length > 0) { + if (turnWaitsOf(session).length > 0) { throw new Error( 'The model asked again for input that the client cannot give.', ) @@ -384,9 +728,22 @@ export function createAcpAgent(options: AcpAgentOptions): AgentApp { else stopReason = 'cancelled' const args = new Map() while (operation) { - for await (const chunk of operation.stream()) { - const update = toSessionUpdate(chunk, args) - if (update) await notify(client, sessionId, update) + const asking = new AbortController() + const questions = answerQuestions({ + client, + sessionId, + session, + operationId: operation.id, + signal: asking.signal, + }).catch(() => {}) + try { + for await (const chunk of operation.stream()) { + const update = toSessionUpdate(chunk, args) + if (update) await notify(client, sessionId, update) + } + } finally { + asking.abort() + await questions } const result: unknown = await operation.then( (value) => value, @@ -430,100 +787,152 @@ export function createAcpAgent(options: AcpAgentOptions): AgentApp { ) } - return agent() - .onRequest('initialize', () => ({ - protocolVersion: PROTOCOL_VERSION, - info: { name: harness.name, version: options.version ?? '0.0.0' }, - // Text and resource links are the baseline. Images, audio, and embedded - // resources go to the media store of the session. - capabilities: { - session: { prompt: { image: {}, audio: {}, embeddedContext: {} } }, - }, - })) - .onRequest('session/new', async () => { - const sessionId = `acp-${Date.now().toString(36)}-${Math.random().toString(36).slice(2, 9)}` - opened.add(sessionId) - await live(sessionId) - return { sessionId } - }) - .onRequest('session/list', async ({ params }) => { - // ponytail: the harness keeps no cwd per thread, so every session - // reports the cwd of the agent. Store one per thread when one agent - // serves more than one project. - const cwd = options.cwd ?? process.cwd() - if (params.cwd && params.cwd !== cwd) return { sessions: [] } - const page = await host.sessions.list({ - harness: harness.name, - limit: LIST_PAGE_SIZE, - ...(params.cursor ? { cursor: params.cursor } : {}), + /** + * True after the first connection. The SDK gives a request handler no + * public id of its connection, so the state above is for one connection + * only. The SDK way to serve many is one app for each connection. + */ + let connected = false + + return ( + agent() + .onConnect((connection) => { + // A second connection would share the handles and queues of the + // first, so the SDK closes it with this error. + if (connected) { + throw new Error( + 'An ACP agent app serves one connection. Call createAcpAgent once for each connection.', + ) + } + connected = true + // The handles of the sessions go with the connection. A session + // with a prompt that runs keeps its handle until that prompt ends: + // a closed handle throws. + void connection.closed.then(() => { + for (const sessionId of handles.keys()) releaseIfIdle(sessionId) + }) }) - return { - sessions: page.entries.map((entry) => ({ - sessionId: entry.threadId, - cwd, - title: entry.title ?? null, - updatedAt: new Date(entry.updatedAt).toISOString(), - })), - nextCursor: page.truncated ? page.cursor : null, - } - }) - .onRequest('session/resume', async ({ params }) => { - await known(params.sessionId) - return {} - }) - .onRequest('session/prompt', async ({ params, client }) => { - const { sessionId } = params - const session = await known(sessionId) - const input = await promptInput(session, params.prompt).catch( - (error: unknown) => { - // The editor sent a file the harness refuses, so the prompt is - // invalid: answer with the reason, not an internal error. - if (error instanceof MediaError) { - throw RequestError.invalidParams( - { status: error.status }, - error.message, - ) - } - throw error - }, - ) - const messageId = `user-${crypto.randomUUID()}` - await notify(client, sessionId, { - sessionUpdate: 'user_message', - messageId, - content: params.prompt, + .onRequest('initialize', () => { + return { + protocolVersion: PROTOCOL_VERSION, + info: { name: harness.name, version: options.version ?? '0.0.0' }, + // Text and resource links are the baseline. Images, audio, and embedded + // resources go to the media store of the session. + capabilities: { + session: { prompt: { image: {}, audio: {}, embeddedContext: {} } }, + }, + } }) - // `pump` catches its own errors and always reports idle. The prompts of - // one session go to the harness in order, each after the turn before it - // ended, so a later prompt cannot overtake a permission request. - const tail = (tails.get(sessionId) ?? Promise.resolve()).then(() => - pump(client, sessionId, session, input), - ) - tails.set(sessionId, tail) - void tail.then(() => { - if (tails.get(sessionId) === tail) tails.delete(sessionId) + .onRequest('session/new', async () => { + const sessionId = `acp-${Date.now().toString(36)}-${Math.random().toString(36).slice(2, 9)}` + // The open saves the session in the session index. The first prompt + // opens it again. + await live(sessionId) + releaseIfIdle(sessionId) + return { sessionId } }) - return { messageId } - }) - .onRequest('session/close', async ({ params }) => { - if (!opened.delete(params.sessionId)) return - await (await live(params.sessionId)).close() - }) - .onNotification('session/cancel', async ({ params }) => { - // While the client asks for an approval, no turn runs: the client - // answers that request with `cancelled`, and the turn stops there. - if (!opened.has(params.sessionId)) return - await (await live(params.sessionId)).cancel() - }) + .onRequest('session/list', async ({ params }) => { + // ponytail: the harness keeps no cwd per thread, so every session + // reports the cwd of the agent. Store one per thread when one agent + // serves more than one project. + const cwd = options.cwd ?? process.cwd() + if (params.cwd && params.cwd !== cwd) return { sessions: [] } + const page = await host.sessions.list({ + user, + harness: harness.name, + limit: LIST_PAGE_SIZE, + ...(params.cursor ? { cursor: params.cursor } : {}), + }) + return { + sessions: page.entries.map((entry) => ({ + sessionId: entry.threadId, + cwd, + title: entry.title ?? null, + updatedAt: new Date(entry.updatedAt).toISOString(), + })), + nextCursor: page.truncated ? page.cursor : null, + } + }) + // The session opens at its first prompt: a resume mounts nothing. + .onRequest('session/resume', async ({ params }) => { + await checkKnown(params.sessionId) + return {} + }) + .onRequest('session/prompt', async ({ params, client }) => { + const { sessionId } = params + const messageId = `user-${crypto.randomUUID()}` + /** + * Open the session, store the files of the prompt, and echo the user + * message. A file upload can take long, so this runs at once, also + * while an earlier prompt still runs. + */ + const accepted = (async () => { + const session = await known(sessionId) + const input = await promptInput(session, params.prompt).catch( + (error: unknown) => { + // The editor sent a file the harness refuses, so the prompt is + // invalid: answer with the reason, not an internal error. + if (error instanceof MediaError) { + throw RequestError.invalidParams( + { status: error.status }, + error.message, + ) + } + throw error + }, + ) + await notify(client, sessionId, { + sessionUpdate: 'user_message', + messageId, + content: params.prompt, + }) + return { session, input } + })() + // Take the place in the queue before the first await, so a prompt + // keeps its place while its files upload. `pump` catches its own + // errors and always reports idle. The prompts of one session go to the + // harness in order, each after the turn before it ended, so a later + // prompt cannot overtake a permission request. A prompt that fails + // here runs no turn: the request answers with the error. + const tail = (tails.get(sessionId) ?? Promise.resolve()) + .then(() => accepted) + .then( + ({ session, input }) => pump(client, sessionId, session, input), + () => {}, + ) + tails.set(sessionId, tail) + void tail.then(() => { + if (tails.get(sessionId) !== tail) return + tails.delete(sessionId) + releaseIfIdle(sessionId) + }) + await accepted + return { messageId } + }) + // Only the handle of this agent goes. Other handles keep the session. + .onRequest('session/close', ({ params }) => release(params.sessionId)) + .onNotification('session/cancel', async ({ params }) => { + // While the client asks for an approval, no turn runs: the client + // answers that request with `cancelled`, and the turn stops there. + if (!handles.has(params.sessionId)) return + await (await live(params.sessionId)).cancel() + }) + ) } /** * Serve a harness as an ACP v2 agent over a stream. With no stream, it uses * stdin and stdout as newline-delimited JSON (for editors that start the - * agent as a process). + * agent as a process). The sessions open as `user`, as in + * `createAcpAgent`. + * + * @example + * ```ts + * serveAcp({ host, harness: assistant, user: { id: 'local-user' } }) + * ``` */ -export function serveAcp( - options: AcpAgentOptions & { stream?: Stream }, +export function serveAcp( + options: AcpAgentOptions & { stream?: Stream }, ): AgentConnection { return createAcpAgent(options).connect(options.stream ?? stdioStream()) } diff --git a/packages/ai-acp/tests/agent-handlers.test.ts b/packages/ai-acp/tests/agent-handlers.test.ts index 3c30efb84b..076c1582c8 100644 --- a/packages/ai-acp/tests/agent-handlers.test.ts +++ b/packages/ai-acp/tests/agent-handlers.test.ts @@ -2,9 +2,14 @@ import { afterEach, describe, expect, it, vi } from 'vitest' import { EventType, toolDefinition } from '@tanstack/ai' import { client, ndJsonStream } from '@agentclientprotocol/sdk/experimental/v2' import { memoryPersistence } from '@tanstack/ai-persistence' -import { createHarnessHost, defineHarness } from '@tanstack/ai-harness' +import { + createHarnessHost, + defineHarness, + membersOnly, +} from '@tanstack/ai-harness' import { createAcpAgent, serveAcp, toSessionUpdate } from '../src/agent' import type { AnyTextAdapter, ModelMessage, StreamChunk } from '@tanstack/ai' +import type { HarnessAccess, HostLifecycleOptions } from '@tanstack/ai-harness' import type { ClientApp, RequestPermissionOutcome, @@ -130,7 +135,11 @@ function model(): AnyTextAdapter { } } -function setup() { +/** With `access`, the host checks each action of each user. */ +function setup({ + access, + lifecycle, +}: { access?: HarnessAccess; lifecycle?: HostLifecycleOptions } = {}) { const remove = vi.fn(async () => ({ removed: true })) const harness = defineHarness({ name: 'test/acp-handlers', @@ -154,7 +163,8 @@ function setup() { }), ], }) - const host = createHarnessHost({ persistence: memoryPersistence() }) + const persistence = memoryPersistence() + const host = createHarnessHost({ persistence, access, lifecycle }) const updates: Array<{ sessionId: string; update: SessionUpdate }> = [] const permissions: Array<{ title: string; toolCallId: string }> = [] // The answer of the editor to each permission request. Default: Reject. @@ -216,6 +226,7 @@ function setup() { ) return { harness, + persistence, host, editor, updates, @@ -327,6 +338,135 @@ describe('ACP agent handlers', () => { close = undefined }) + it('holds no handle on an idle session, and opens it again at the next prompt', async () => { + // A session that no handle holds closes at once when it is idle. + const { harness, host, editor, running, states, textOf } = setup({ + lifecycle: { closeIdleSessionsAfterMs: 0 }, + }) + close = () => host.close() + const isOpen = (threadId: string) => host.isOpen(harness, { threadId }) + await editor.connectWith(createAcpAgent({ host, harness }), async (ctx) => { + await ctx.request('initialize', { + protocolVersion: 2, + info: { name: 'editor', version: '1.0.0' }, + }) + const { sessionId } = await ctx.request('session/new', { + cwd: '/tmp', + mcpServers: [], + }) + await vi.waitFor(() => expect(isOpen(sessionId)).toBe(false)) + + await ctx.request('session/prompt', { + sessionId, + prompt: [{ type: 'text', text: 'wait' }], + }) + await vi.waitFor(() => expect(running(sessionId)).toBe(true)) + // A running prompt holds the session. + expect(isOpen(sessionId)).toBe(true) + await ctx.notify('session/cancel', { sessionId }) + await vi.waitFor(() => expect(isOpen(sessionId)).toBe(false)) + + await ctx.request('session/prompt', { + sessionId, + prompt: [{ type: 'text', text: 'hello' }], + }) + await vi.waitFor(() => + expect(states(sessionId)).toContain('idle:end_turn'), + ) + // The session opened again and ran the prompt. + expect(textOf(sessionId)).toBe('echo: hello') + await vi.waitFor(() => expect(isOpen(sessionId)).toBe(false)) + }) + }) + + it('takes the prompts of its new sessions on a host with no session index', async () => { + const { harness, editor, states, textOf } = setup() + // Only a log store: `host.sessions.get` finds no session. + const host = createHarnessHost({ + persistence: { stores: { log: memoryPersistence().stores.log } }, + lifecycle: { closeIdleSessionsAfterMs: 0 }, + }) + close = () => host.close() + await editor.connectWith(createAcpAgent({ host, harness }), async (ctx) => { + await ctx.request('initialize', { + protocolVersion: 2, + info: { name: 'editor', version: '1.0.0' }, + }) + const { sessionId } = await ctx.request('session/new', { + cwd: '/tmp', + mcpServers: [], + }) + await vi.waitFor(() => + expect(host.isOpen(harness, { threadId: sessionId })).toBe(false), + ) + + await ctx.request('session/prompt', { + sessionId, + prompt: [{ type: 'text', text: 'hello' }], + }) + await vi.waitFor(() => + expect(states(sessionId)).toContain('idle:end_turn'), + ) + expect(textOf(sessionId)).toBe('echo: hello') + }) + }) + + it('keeps the order of two prompts when the first one uploads a file', async () => { + const { harness, host, persistence, editor, updates, states, textOf } = + setup() + close = () => host.close() + // The store of the first file answers only when the test lets it. + let upload = () => {} + const uploaded = new Promise((resolve) => { + upload = resolve + }) + const put = persistence.stores.blobs.put.bind(persistence.stores.blobs) + const slowPut = vi + .spyOn(persistence.stores.blobs, 'put') + .mockImplementation(async (...args) => { + await uploaded + return put(...args) + }) + await editor.connectWith(createAcpAgent({ host, harness }), async (ctx) => { + await ctx.request('initialize', { + protocolVersion: 2, + info: { name: 'editor', version: '1.0.0' }, + }) + const { sessionId } = await ctx.request('session/new', { + cwd: '/tmp', + mcpServers: [], + }) + const first = ctx.request('session/prompt', { + sessionId, + prompt: [ + { type: 'text', text: 'first' }, + { type: 'image', data: 'AQIDBA==', mimeType: 'image/png' }, + ], + }) + await vi.waitFor(() => expect(slowPut).toHaveBeenCalled()) + const second = ctx.request('session/prompt', { + sessionId, + prompt: [{ type: 'text', text: 'second' }], + }) + // The second prompt has no file, so the agent takes it at once. + await vi.waitFor(() => + expect( + updates.filter( + (entry) => entry.update.sessionUpdate === 'user_message', + ), + ).toHaveLength(1), + ) + upload() + await Promise.all([first, second]) + await vi.waitFor(() => + expect( + states(sessionId).filter((state) => state === 'idle:end_turn'), + ).toHaveLength(2), + ) + expect(textOf(sessionId)).toBe('echo: firstecho: second') + }) + }) + it('keeps the tool from running when the editor rejects it', async () => { const { harness, host, editor, permissions, remove, idle, textOf } = setup() close = () => host.close() @@ -696,6 +836,31 @@ describe('ACP agent handlers', () => { }) }) + it('refuses a subagent child: the editor reaches it through its parent', async () => { + const { harness, persistence, host, editor } = setup() + close = () => host.close() + await persistence.stores.sessions.upsert({ + threadId: 'subagent:run-1', + parentThreadId: 'lead', + harness: harness.name, + members: [], + createdAt: 1, + updatedAt: 1, + }) + await editor.connectWith(createAcpAgent({ host, harness }), async (ctx) => { + await ctx.request('initialize', { + protocolVersion: 2, + info: { name: 'editor', version: '1.0.0' }, + }) + await expect( + ctx.request('session/resume', { + sessionId: 'subagent:run-1', + cwd: '/tmp', + }), + ).rejects.toMatchObject({ code: -32002 }) + }) + }) + it('resumes a session by id, cancels a running turn, and closes it', async () => { const { harness, host, editor, idle, running, textOf } = setup() close = () => host.close() @@ -733,6 +898,91 @@ describe('ACP agent handlers', () => { await ctx.notify('session/cancel', { sessionId: 'never-opened' }) }) }) + + it('resumes a session without opening it: it opens at its first prompt', async () => { + const { harness, host, editor, idle, textOf } = setup({ + lifecycle: { closeIdleSessionsAfterMs: 0 }, + }) + close = () => host.close() + await (await host.open(harness, { threadId: 'kept-session' })).close() + const isOpen = () => host.isOpen(harness, { threadId: 'kept-session' }) + await editor.connectWith(createAcpAgent({ host, harness }), async (ctx) => { + await ctx.request('initialize', { + protocolVersion: 2, + info: { name: 'editor', version: '1.0.0' }, + }) + + await ctx.request('session/resume', { + sessionId: 'kept-session', + cwd: '/tmp', + }) + expect(isOpen()).toBe(false) + + await ctx.request('session/prompt', { + sessionId: 'kept-session', + prompt: [{ type: 'text', text: 'hello' }], + }) + await vi.waitFor(() => expect(idle('kept-session')).toBeDefined()) + expect(textOf('kept-session')).toBe('echo: hello') + }) + }) + + describe('on a host with access', () => { + const olga = { id: 'olga', tenantId: 'acme' } + const bea = { id: 'bea', tenantId: 'acme' } + // The agent runs for one local user, so no request is identified. + const access: HarnessAccess = { + getUser: () => null, + rules: membersOnly(), + } + + it('throws at the start without a user', () => { + const { harness, host } = setup({ access }) + close = () => host.close() + + expect(() => createAcpAgent({ host, harness })).toThrow( + 'createAcpAgent needs a user: the host has access. Pass the local user as the user option.', + ) + }) + + it('opens and lists only the sessions of its user', async () => { + const { harness, host, editor, idle, textOf } = setup({ access }) + close = () => host.close() + await host.open(harness, { threadId: 'bea-thread', user: bea }) + const agentApp = createAcpAgent({ host, harness, user: olga }) + await editor.connectWith(agentApp, async (ctx) => { + await ctx.request('initialize', { + protocolVersion: 2, + info: { name: 'editor', version: '1.0.0' }, + }) + const { sessionId } = await ctx.request('session/new', { + cwd: '/tmp', + mcpServers: [], + }) + await ctx.request('session/prompt', { + sessionId, + prompt: [{ type: 'text', text: 'hello' }], + }) + await vi.waitFor(() => expect(idle(sessionId)).toBeDefined()) + + expect(textOf(sessionId)).toBe('echo: hello') + expect( + await host.sessions.get({ threadId: sessionId, user: olga }), + ).toMatchObject({ members: [{ userId: 'olga', role: 'owner' }] }) + const listed = await ctx.request('session/list', {}) + expect(listed.sessions.map((session) => session.sessionId)).toEqual([ + sessionId, + ]) + // A session of another user gets the answer of an unknown id. + await expect( + ctx.request('session/resume', { + sessionId: 'bea-thread', + cwd: '/tmp', + }), + ).rejects.toMatchObject({ code: -32002 }) + }) + }) + }) }) describe('serveAcp', () => { diff --git a/packages/ai-acp/tests/agent-media.test.ts b/packages/ai-acp/tests/agent-media.test.ts index fe9df9b4ba..2d11baf822 100644 --- a/packages/ai-acp/tests/agent-media.test.ts +++ b/packages/ai-acp/tests/agent-media.test.ts @@ -2,11 +2,8 @@ import { afterEach, describe, expect, it, vi } from 'vitest' import { EventType } from '@tanstack/ai' import { client } from '@agentclientprotocol/sdk/experimental/v2' import { memoryPersistence } from '@tanstack/ai-persistence' -import { - createHarnessHost, - defineHarness, - mediaOfMessage, -} from '@tanstack/ai-harness' +import { createHarnessHost, defineHarness } from '@tanstack/ai-harness' +import { mediaOfMessage } from '@tanstack/ai-harness/media' import { createAcpAgent } from '../src/agent' import type { AnyTextAdapter, StreamChunk } from '@tanstack/ai' import type { @@ -14,7 +11,8 @@ import type { ContentBlock, InitializeResponse, } from '@agentclientprotocol/sdk/experimental/v2' -import type { MediaOptions, MediaRecord } from '@tanstack/ai-harness' +import type { MediaRecord } from '@tanstack/ai-harness' +import type { MediaOptions } from '@tanstack/ai-harness/media' const now = () => Date.now() @@ -207,7 +205,7 @@ describe('ACP agent media', () => { }, }, ]) - expect(await session.loadMedia(record.id)).toEqual( + expect(await session.loadMedia({ id: record.id })).toEqual( new Uint8Array([1, 2, 3, 4]), ) }, diff --git a/packages/ai-acp/tests/agent.test.ts b/packages/ai-acp/tests/agent.test.ts index e55af2d032..89c8d95cc5 100644 --- a/packages/ai-acp/tests/agent.test.ts +++ b/packages/ai-acp/tests/agent.test.ts @@ -2,13 +2,44 @@ import { describe, expect, it, vi } from 'vitest' import { EventType, toolDefinition } from '@tanstack/ai' import { client } from '@agentclientprotocol/sdk/experimental/v2' import { memoryPersistence } from '@tanstack/ai-persistence' -import { createHarnessHost, defineHarness } from '@tanstack/ai-harness' +import { + createHarnessHost, + defineHarness, + definePlugin, + durableTool, +} from '@tanstack/ai-harness' +import { + PermissionResources, + PermissionRules, + permissions, +} from '@tanstack/ai-harness/plugins' import { createAcpAgent } from '../src/agent' import type { AnyTextAdapter, StreamChunk } from '@tanstack/ai' import type { SessionUpdate } from '@agentclientprotocol/sdk/experimental/v2' const now = () => Date.now() +// Coverage runs in CI are slow, so polled checks wait up to 5 s. +const waitFor = (check: () => T | Promise) => + vi.waitFor(check, { timeout: 5_000 }) + +/** True once `updates` has an idle state. */ +const isIdle = (updates: Array) => + updates.some( + (update) => + update.sessionUpdate === 'state_update' && update.state === 'idle', + ) + +/** The text that the agent sent in `updates`. */ +const agentText = (updates: Array) => + updates + .map((update) => + update.sessionUpdate === 'agent_message_chunk' + ? JSON.stringify(update) + : '', + ) + .join('') + function scripted(turns: Array>) { let call = 0 const adapter: AnyTextAdapter = { @@ -115,13 +146,13 @@ describe('ACP v2 agent', () => { const agentApp = createAcpAgent({ host, harness, version: '1.0.0' }) const updates: Array = [] - const permissions: Array = [] + const titles: Array = [] const editor = client() .onNotification('session/update', ({ params }) => { updates.push(params.update) }) .onRequest('session/request_permission', ({ params }) => { - permissions.push(params.title) + titles.push(params.title) return { outcome: { outcome: 'selected', optionId: 'allow' } } }) @@ -153,7 +184,7 @@ describe('ACP v2 agent', () => { ) }) - expect(permissions).toHaveLength(1) + expect(titles).toHaveLength(1) expect(remove).toHaveBeenCalledTimes(1) const states = updates .filter((update) => update.sessionUpdate === 'state_update') @@ -177,4 +208,228 @@ describe('ACP v2 agent', () => { ).toBe(true) await host.close() }) + + it('offers allow_always for an approval of permissions(), and keeps the answer', async () => { + const ran: Array = [] + const shell = definePlugin({ + name: 'test/shell', + setup: () => ({ + tools: [ + toolDefinition({ + name: 'bash', + description: 'Run a command', + inputSchema: { + type: 'object', + properties: { command: { type: 'string' } }, + required: ['command'], + }, + }).server(async () => { + ran.push('bash') + return 'ok' + }), + ], + contribute: [ + PermissionRules.item({ + tool: 'bash', + decision: 'ask', + kind: 'execute', + }), + PermissionResources.item({ bash: { commands: () => ['ls'] } }), + ], + }), + }) + const harness = defineHarness({ + name: 'test/acp-permissions', + adapter: scripted([ + toolTurn('bash', '{"command":"ls"}'), + textTurn('Listed.'), + ]), + plugins: () => [permissions({ root: '/work' }), shell], + }) + const host = createHarnessHost({ persistence: memoryPersistence() }) + const updates: Array = [] + const offered: Array> = [] + const subjects: Array = [] + const editor = client() + .onNotification('session/update', ({ params }) => { + updates.push(params.update) + }) + .onRequest('session/request_permission', ({ params }) => { + offered.push(params.options.map((option) => option.kind)) + subjects.push(params.subject) + return { outcome: { outcome: 'selected', optionId: 'always' } } + }) + + await editor.connectWith(createAcpAgent({ host, harness }), async (ctx) => { + await ctx.request('initialize', { + protocolVersion: 2, + info: { name: 'test-editor', version: '1.0.0' }, + }) + const { sessionId } = await ctx.request('session/new', { + cwd: '/tmp', + mcpServers: [], + }) + await ctx.request('session/prompt', { + sessionId, + prompt: [{ type: 'text', text: 'list the files' }], + }) + await waitFor(() => expect(isIdle(updates)).toBe(true)) + }) + + expect(offered).toEqual([['allow_once', 'allow_always', 'reject_once']]) + // The request names the tool call that the editor already shows. + expect(subjects).toEqual([ + { type: 'tool_call', toolCall: { toolCallId: 'call_1' } }, + ]) + expect(ran).toEqual(['bash']) + expect(agentText(updates)).toContain('Listed.') + await host.close() + }) + + it('asks a question of a durable tool with a form elicitation', async () => { + const greet = durableTool( + toolDefinition({ + name: 'greet', + description: 'Ask the user for a name', + inputSchema: { type: 'object', properties: {} }, + }), + async (_args, { step }) => { + const answer = await step.ask({ + name: 'name', + message: 'Name?', + schema: { + type: 'object', + properties: { name: { type: 'string' } }, + required: ['name'], + }, + }) + return { greeted: answer } + }, + ) + const harness = defineHarness({ + name: 'test/acp-questions', + adapter: scripted([toolTurn('greet', '{}'), textTurn('Greeted.')]), + tools: [greet], + }) + const host = createHarnessHost({ persistence: memoryPersistence() }) + const updates: Array = [] + const asked: Array = [] + const editor = client() + .onNotification('session/update', ({ params }) => { + updates.push(params.update) + }) + .onRequest('elicitation/create', ({ params }) => { + asked.push(params) + return { action: 'accept', content: { name: 'Ada' } } + }) + + let sessionId = '' + await editor.connectWith(createAcpAgent({ host, harness }), async (ctx) => { + await ctx.request('initialize', { + protocolVersion: 2, + info: { name: 'test-editor', version: '1.0.0' }, + capabilities: { elicitation: { form: {} } }, + }) + sessionId = ( + await ctx.request('session/new', { cwd: '/tmp', mcpServers: [] }) + ).sessionId + await ctx.request('session/prompt', { + sessionId, + prompt: [{ type: 'text', text: 'greet me' }], + }) + await waitFor(() => expect(isIdle(updates)).toBe(true)) + }) + + expect(asked).toMatchObject([ + { + sessionId, + mode: 'form', + message: 'Name?', + requestedSchema: { type: 'object', required: ['name'] }, + }, + ]) + expect(agentText(updates)).toContain('Greeted.') + const session = await host.open(harness, { threadId: sessionId }) + const result = (await session.transcript()).find( + (message) => message.role === 'tool', + ) + expect(result?.content).toContain('Ada') + await host.close() + }) + + it('refuses a second connection to one app, and keeps the first', async () => { + const greet = durableTool( + toolDefinition({ + name: 'greet', + description: 'Ask the user for a name', + inputSchema: { type: 'object', properties: {} }, + }), + async (_args, { step }) => { + const answer = await step.ask({ + name: 'name', + message: 'Name?', + schema: { + type: 'object', + properties: { name: { type: 'string' } }, + required: ['name'], + }, + }) + return { greeted: answer } + }, + ) + const harness = defineHarness({ + name: 'test/acp-connections', + adapter: scripted([toolTurn('greet', '{}'), textTurn('Greeted.')]), + tools: [greet], + }) + const host = createHarnessHost({ persistence: memoryPersistence() }) + const app = createAcpAgent({ host, harness }) + const updates: Array = [] + const asked: Array = [] + const first = client() + .onNotification('session/update', ({ params }) => { + updates.push(params.update) + }) + .onRequest('elicitation/create', ({ params }) => { + asked.push(params) + return { action: 'accept', content: { name: 'Ada' } } + }) + // An editor that shows no elicitation, on its own connection. + const second = client() + + await first.connectWith(app, async (ctx) => { + await ctx.request('initialize', { + protocolVersion: 2, + info: { name: 'editor-a', version: '1.0.0' }, + capabilities: { elicitation: { form: {} } }, + }) + const { sessionId } = await ctx.request('session/new', { + cwd: '/tmp', + mcpServers: [], + }) + // A second editor on the same app would share its sessions, so the + // connection fails at once. + expect(() => + second.connectWith(app, async (other) => { + await other.request('initialize', { + protocolVersion: 2, + info: { name: 'editor-b', version: '1.0.0' }, + }) + }), + ).toThrow( + 'An ACP agent app serves one connection. Call createAcpAgent once for each connection.', + ) + await ctx.request('session/prompt', { + sessionId, + prompt: [{ type: 'text', text: 'greet me' }], + }) + await waitFor(() => expect(isIdle(updates)).toBe(true)) + }) + + // The first editor still gets its question: the second one did not + // change what the first one can show. + expect(asked).toHaveLength(1) + expect(agentText(updates)).toContain('Greeted.') + await host.close() + }) }) diff --git a/packages/ai-code-mode/src/harness.ts b/packages/ai-code-mode/src/harness.ts index 973c711d5f..8e4fc4497c 100644 --- a/packages/ai-code-mode/src/harness.ts +++ b/packages/ai-code-mode/src/harness.ts @@ -7,8 +7,17 @@ import { } from '@tanstack/ai-harness' import { createCodeMode } from './create-code-mode' import type { AnyTool } from '@tanstack/ai' +import type { PermissionMode } from '@tanstack/ai-harness' import type { CodeModeTool, CodeModeToolConfig } from './types' +/** The permission modes of `permissions()`, for the `mode` setting. */ +const MODES: ReadonlyArray = [ + 'default', + 'plan', + 'acceptEdits', + 'bypass', +] + export interface CodeModePluginOptions extends Omit< CodeModeToolConfig, 'tools' @@ -20,6 +29,12 @@ export interface CodeModePluginOptions extends Omit< * the rules of tool plugins decide. A safe tool with * `metadata.codeMode: true`, for example a tool of an `mcp()` server with * `codeMode: true`, moves even when `include` does not pick it. + * + * `include` cannot move a tool with `needsApproval`, or a tool that a rule + * asks for or denies: such a tool stays a normal tool call, with its + * approval and its checks. Code mode warns once for each such tool that `include` + * picks. The rules decide in the permission mode of the session at each + * turn, so in plan mode a tool that edits stays a normal tool call. */ include?: (tool: CodeModeTool) => boolean /** @@ -58,8 +73,10 @@ function isServerTool(tool: AnyTool): tool is CodeModeTool { * tools that are safe to run without a question move into code mode. The * other tools stay normal tool calls. A tool with `metadata.codeMode: true` * moves even when `include` does not pick it, but only when it is safe in - * the same way. Calls inside the isolate also skip the permission checks, - * so a tool that declares `PermissionResources` never moves. With + * the same way. `include` can pick more tools, but never a tool with + * `needsApproval` or a tool that a rule asks for or denies. Calls inside the + * isolate also skip the permission checks, so a tool that declares + * `PermissionResources` never moves. With * `lazy: true`, the model gets only the names of the moved tools and asks * `discover_tools` for the signatures. * @@ -77,6 +94,15 @@ function isServerTool(tool: AnyTool): tool is CodeModeTool { */ export function codeMode(options: CodeModePluginOptions) { const { include, lazy = false, ...config } = options + /** The tools that `include` picked but cannot move, warned once each. */ + const warned = new Set() + const warnKept = (name: string) => { + if (warned.has(name)) return + warned.add(name) + console.warn( + `[@tanstack/ai-code-mode] include picked the tool ${JSON.stringify(name)}, but it needs approval, or a rule asks for it or denies it. A call inside code mode skips those checks, so the tool stays a normal tool call.`, + ) + } return definePlugin({ name: 'tanstack/code-mode', optionalRequires: [PermissionDecisionCapability], @@ -104,10 +130,20 @@ export function codeMode(options: CodeModePluginOptions) { // Ask permissions() when it is mounted: it knows all rules in its // order and its default. Read here, so it can come after code mode. const decide = ctx.getOptional(PermissionDecisionCapability) + const decisionOf = (tool: CodeModeTool, mode: PermissionMode) => + decide?.(tool.name, mode) ?? + decidePermission(rules, tool.name, mode) const safe = (tool: CodeModeTool) => - !tool.needsApproval && - (decide?.(tool.name, 'plan') ?? - decidePermission(rules, tool.name, 'plan')) === 'allow' + !tool.needsApproval && decisionOf(tool, 'plan') === 'allow' + // The permission mode of the session now, read at each turn. In + // plan mode, a tool that edits must stay a tool call, where the + // rules deny it. + const value = ctx.config.get('mode') + const mode = MODES.find((each) => each === value) ?? 'default' + // `include` widens the pick past plan mode, but a call inside the + // isolate must not skip an approval, an `ask` rule, or a deny rule. + const canInclude = (tool: CodeModeTool) => + !tool.needsApproval && decisionOf(tool, mode) === 'allow' const moves = (tool: CodeModeTool) => { // The permission checks run on tool calls of the model. A call // inside the isolate skips them, so such a tool stays a tool call. @@ -118,7 +154,11 @@ export function codeMode(options: CodeModePluginOptions) { // A marked tool still needs the checks of the default pick. const isMarked = tool.metadata?.codeMode === true if (isMarked && safe(tool)) return true - return include ? include(tool) : safe(tool) + if (!include) return safe(tool) + if (!include(tool)) return false + if (canInclude(tool)) return true + warnKept(tool.name) + return false } // Two tools that map to the same identifier: the first one moves. const byIdentifier = new Map() diff --git a/packages/ai-code-mode/tests/harness.test.ts b/packages/ai-code-mode/tests/harness.test.ts index ed11a2a565..bc232026be 100644 --- a/packages/ai-code-mode/tests/harness.test.ts +++ b/packages/ai-code-mode/tests/harness.test.ts @@ -209,7 +209,9 @@ async function firstToolNames(options: { tools: options.tools, plugins: () => options.plugins, }), - { threadId: 't' }, + { + threadId: 't', + }, ) await session.send('hi') await host.close() @@ -231,7 +233,9 @@ describe('codeMode', () => { tools: [tools.lookup, tools.remove, tools.askUser], plugins: () => [writes, notion, codeMode({ driver })], }), - { threadId: 't' }, + { + threadId: 't', + }, ) const turn = await session.send('look it up') expect(turn.text).toBe('done') @@ -263,16 +267,18 @@ describe('codeMode', () => { defineHarness({ name: 'test/code-mode-include', adapter, - tools: [tools.lookup, tools.remove], + tools: [tools.lookup, tools.notionSearch], plugins: () => [ - codeMode({ driver, include: (tool) => tool.name === 'remove' }), + codeMode({ driver, include: (tool) => tool.name === 'lookup' }), ], }), - { threadId: 'picked' }, + { + threadId: 'picked', + }, ) await picked.send('hi') expect(calls[0].tools.map((tool: { name: string }) => tool.name)).toEqual([ - 'lookup', + 'notion_search', 'execute_typescript', ]) @@ -283,7 +289,9 @@ describe('codeMode', () => { tools: [tools.remove], plugins: () => [codeMode({ driver })], }), - { threadId: 'none' }, + { + threadId: 'none', + }, ) await none.send('hi') expect(calls[1].tools.map((tool: { name: string }) => tool.name)).toEqual([ @@ -309,7 +317,9 @@ describe('codeMode', () => { tools: [tools.lookup, tools.remove], plugins: () => [codeMode({ driver, lazy: true })], }), - { threadId: 't' }, + { + threadId: 't', + }, ) expect((await session.send('what can you call?')).text).toBe('found it') @@ -362,7 +372,9 @@ describe('codeMode', () => { ], plugins: () => [codeMode({ driver })], }), - { threadId: 't' }, + { + threadId: 't', + }, ) await session.send('search') // web_search clashes with web-search after the rename, so it stays direct. @@ -375,6 +387,65 @@ describe('codeMode', () => { await host.close() }) + it('keeps a tool that needs approval, or that a rule asks for or denies, when include picks it, and warns once', async () => { + const warn = vi.spyOn(console, 'warn').mockImplementation(() => {}) + const denyDocs = definePlugin({ + name: 'test/deny-docs', + setup: () => ({ + contribute: [ + PermissionRules.item({ tool: 'docs_deny', decision: 'deny' }), + PermissionRules.item({ tool: 'docs_ask', decision: 'ask' }), + ], + }), + }) + const { adapter, calls } = scripted([textTurn('one'), textTurn('two')]) + const host = createHarnessHost({ persistence: memoryPersistence() }) + const session = await host.open( + defineHarness({ + name: 'test/code-mode-include-guard', + adapter, + tools: [ + tools.lookup, + tools.remove, + toolDefinition({ + name: 'docs_deny', + description: 'A tool that a rule denies', + inputSchema: z.object({}), + }).server(async () => 'denied'), + toolDefinition({ + name: 'docs_ask', + description: 'A tool that a rule asks for', + inputSchema: z.object({}), + }).server(async () => 'asked'), + ], + plugins: () => [ + denyDocs, + codeMode({ driver: fakeDriver().driver, include: () => true }), + ], + }), + { threadId: 't' }, + ) + await session.send('first') + await session.send('second') + await host.close() + + for (const call of calls) { + expect(call.tools.map((tool: { name: string }) => tool.name)).toEqual([ + 'remove', + 'docs_deny', + 'docs_ask', + 'execute_typescript', + ]) + } + // One warning for each kept tool, not one for each turn. + expect(warn.mock.calls.map(([message]) => message)).toEqual([ + '[@tanstack/ai-code-mode] include picked the tool "remove", but it needs approval, or a rule asks for it or denies it. A call inside code mode skips those checks, so the tool stays a normal tool call.', + '[@tanstack/ai-code-mode] include picked the tool "docs_deny", but it needs approval, or a rule asks for it or denies it. A call inside code mode skips those checks, so the tool stays a normal tool call.', + '[@tanstack/ai-code-mode] include picked the tool "docs_ask", but it needs approval, or a rule asks for it or denies it. A call inside code mode skips those checks, so the tool stays a normal tool call.', + ]) + warn.mockRestore() + }) + it('moves a marked tool that include does not pick, unless a rule or an approval stops it', async () => { const rules = definePlugin({ name: 'test/marked-rules', @@ -406,6 +477,40 @@ describe('codeMode', () => { ]) }) + it('keeps a tool that include picks and that edits as a tool call in plan mode', async () => { + const { adapter, calls } = scripted([textTurn('ok'), textTurn('ok')]) + const host = createHarnessHost({ persistence: memoryPersistence() }) + const session = await host.open( + defineHarness({ + name: 'test/code-mode-include-plan', + adapter, + tools: [tools.lookup], + plugins: () => [ + permissions({ + rules: [{ tool: 'lookup', decision: 'allow', kind: 'edit' }], + }), + codeMode({ + driver: fakeDriver().driver, + include: (tool) => tool.name === 'lookup', + }), + ], + }), + { threadId: 't' }, + ) + const namesOf = (call: number) => + calls[call].tools.map((tool: { name: string }) => tool.name) + await session.send('hi') + // In the default mode, the rule allows it, so it moves. + expect(namesOf(0)).toEqual(['execute_typescript']) + + await session.runCommand({ name: 'mode', input: 'plan' }) + await session.send('plan it') + // Plan mode denies an edit, so a call inside the isolate must not run + // it. No tool moves, so code mode adds no tool. + expect(namesOf(1)).toEqual(['lookup']) + await host.close() + }) + it('keeps a tool that a rule of permissions() asks for as a tool call', async () => { const names = await firstToolNames({ tools: [ @@ -503,9 +608,12 @@ describe('codeMode', () => { codeMode({ driver }), ], }), - { threadId: 't' }, + { + threadId: 't', + }, ) - if (mode === 'plan') await session.command('mode', 'plan') + if (mode === 'plan') + await session.runCommand({ name: 'mode', input: 'plan' }) expect((await session.send('look it up')).text).toBe('done') // The program ran: a question would wait, and plan mode would deny it. expect(seen).toHaveLength(1) @@ -530,20 +638,34 @@ describe('codeMode', () => { codeMode({ driver }), ], }), - { threadId: 't' }, + { + threadId: 't', + }, ) - const turn = session.send('look it up') - await vi.waitFor(() => - expect(session.snapshot().pendingQuestions).toHaveLength(1), + // The approval is a turn wait at the boundary: the turn stops, and a + // `resolve` answers it. + void session.send('look it up') + await vi.waitFor( + () => expect(session.snapshot().pendingWaits).toHaveLength(1), + { timeout: 5_000 }, ) - const [question] = session.snapshot().pendingQuestions - if (!question) throw new Error('No question.') - expect(question.message).toContain('execute_typescript') - await session.answer({ - questionId: question.questionId, - value: { answer: 'reject' }, + const [wait] = session.snapshot().pendingWaits + if (!wait) throw new Error('No approval.') + expect(wait.kind).toBe('approval') + expect(wait.message).toContain('execute_typescript') + const inputId = `answer:${wait.waitId}` + await session.resolve({ + resume: [ + { interruptId: wait.waitId, status: 'resolved', payload: 'reject' }, + ], + inputId, + }) + await session.settled({ inputId }) + const messages = await session.transcript() + expect(messages.at(-1)).toMatchObject({ + role: 'assistant', + content: 'done', }) - expect((await turn).text).toBe('done') expect(seen).toEqual([]) await host.close() }) @@ -568,7 +690,9 @@ describe('codeMode', () => { }), ], }), - { threadId: 't' }, + { + threadId: 't', + }, ) await session.send('first') expect(JSON.stringify(calls[0].systemPrompts)).toContain('external_lookup') diff --git a/packages/ai-harness-cli/package.json b/packages/ai-harness-cli/package.json index 95a749a9b1..76c4920ab6 100644 --- a/packages/ai-harness-cli/package.json +++ b/packages/ai-harness-cli/package.json @@ -83,6 +83,7 @@ "devDependencies": { "@tanstack/ai": "workspace:*", "@tanstack/ai-harness": "workspace:*", + "@tanstack/ai-mcp": "workspace:*", "@tanstack/ai-persistence": "workspace:*", "@vitest/coverage-v8": "4.1.10", "zod": "^4.2.0" diff --git a/packages/ai-harness-cli/src/args.ts b/packages/ai-harness-cli/src/args.ts index c5b89c797f..0fc59a389c 100644 --- a/packages/ai-harness-cli/src/args.ts +++ b/packages/ai-harness-cli/src/args.ts @@ -11,8 +11,8 @@ export interface CliArgs { port: number hostname: string token?: string + mediaKey?: string thread: string - dashboard?: string mediaDir?: string } @@ -32,8 +32,8 @@ Options: --port Port for --serve (default 8787) --host Host name for --serve (default 127.0.0.1) --token Bearer token for --serve (default: HARNESS_TOKEN, or a new random token) + --media-key Key that signs media URLs of --serve, 32+ bytes (default: HARNESS_MEDIA_KEY, or a new random key at each start) --thread Conversation id (default: main) - --dashboard Connect to a dashboard (pairs on first use; set HARNESS_DASHBOARD_TOKEN to skip) --media-dir Save the media a turn makes here (default: ./-media) -h, --help Show this help @@ -54,8 +54,8 @@ export function parseCliArgs(argv: ReadonlyArray): CliArgs { port: { type: 'string' }, host: { type: 'string' }, token: { type: 'string' }, + 'media-key': { type: 'string' }, thread: { type: 'string' }, - dashboard: { type: 'string' }, 'media-dir': { type: 'string' }, }, strict: true, @@ -79,8 +79,10 @@ export function parseCliArgs(argv: ReadonlyArray): CliArgs { port, hostname: values.host ?? '127.0.0.1', ...(values.token !== undefined ? { token: values.token } : {}), + ...(values['media-key'] !== undefined + ? { mediaKey: values['media-key'] } + : {}), thread: values.thread ?? 'main', - ...(values.dashboard !== undefined ? { dashboard: values.dashboard } : {}), ...(values['media-dir'] !== undefined ? { mediaDir: values['media-dir'] } : {}), diff --git a/packages/ai-harness-cli/src/attach.ts b/packages/ai-harness-cli/src/attach.ts index 880689a407..c576fcd855 100644 --- a/packages/ai-harness-cli/src/attach.ts +++ b/packages/ai-harness-cli/src/attach.ts @@ -1,6 +1,6 @@ import { mkdir, readFile, stat, writeFile } from 'node:fs/promises' import { basename, join, resolve } from 'node:path' -import { mediaPart, mimeTypeOf } from '@tanstack/ai-harness' +import { mediaPart, mimeTypeOf } from '@tanstack/ai-harness/media' import type { ContentPart } from '@tanstack/ai' import type { HarnessSession, @@ -73,7 +73,8 @@ export async function attach( if (file === undefined) continue rest += text.slice(end, token.index) end = token.index + token[0].length - const record = await session.putMedia(file.bytes, { + const record = await session.putMedia({ + body: file.bytes, mimeType: file.mimeType, name: file.name, }) @@ -109,7 +110,7 @@ export async function saveMedia( `Cannot save ${JSON.stringify(name)}: it is not a plain file name.`, ) } - const bytes = await session.loadMedia(record.id) + const bytes = await session.loadMedia({ id: record.id }) await mkdir(dir, { recursive: true }) const dot = name.lastIndexOf('.') const stem = dot > 0 ? name.slice(0, dot) : name diff --git a/packages/ai-harness-cli/src/commands.ts b/packages/ai-harness-cli/src/commands.ts index 6d34b7d0d0..ba05eb222e 100644 --- a/packages/ai-harness-cli/src/commands.ts +++ b/packages/ai-harness-cli/src/commands.ts @@ -1,5 +1,14 @@ import { attach } from './attach' -import type { HarnessSession, UserInput } from '@tanstack/ai-harness' +import type { + HarnessSession, + SessionSnapshot, + UserInput, +} from '@tanstack/ai-harness' + +/** The first question wait of `snapshot` that no turn holds: `answer` takes it. */ +export function firstQuestion({ pendingWaits }: SessionSnapshot) { + return pendingWaits.find(({ turn, kind }) => !turn && kind === 'question') +} export const HELP_TEXT = [ 'Type a message and press Enter. While the agent works, a new message steers it.', @@ -8,7 +17,7 @@ export const HELP_TEXT = [ ' /agent [json] run an agent in the background', ' /config [key value] show or change a setting', ' /connect sign in to a connector (and /disconnect )', - ' /cancel cancel the running turn (or press Esc)', + ' /cancel cancel the running turn', ' /status show the session status', ' /exit quit', ].join('\n') @@ -55,23 +64,26 @@ async function runCommand( outcome.ok ? show(outcome.value) : `/${name} failed: ${outcome.error instanceof Error ? outcome.error.message : String(outcome.error)}` + const operation = session.runCommand({ name, input }) const settled = Promise.resolve( - session.command(name, input).then( + operation.then( (value) => ({ ok: true as const, value }), (error: unknown) => ({ ok: false as const, error }), ), ) // A command that asks a question waits for the next line, so return now // and print its result when it is done. - let done = false - void settled.then(() => { - done = true - }) - while (!done) { - if (session.snapshot().pendingQuestions.length > 0) { - return { type: 'notice', text: '', later: settled.then(describe) } - } - await new Promise((resolve) => setTimeout(resolve, 10)) + const stop = new AbortController() + const asked = session + .waitForRun({ runId: operation.id, signal: stop.signal }) + .then( + () => true, + () => false, + ) + const isAsking = await Promise.race([settled.then(() => false), asked]) + stop.abort() + if (isAsking) { + return { type: 'notice', text: '', later: settled.then(describe) } } return { type: 'notice', text: describe(await settled) } } @@ -94,11 +106,11 @@ export async function handleLine( ): Promise { const text = line.trim() if (text === '') return { type: 'notice', text: '' } - const [question] = session.snapshot().pendingQuestions + const question = firstQuestion(session.snapshot()) if (question) { // A secret (a key) is the text as typed: `123456` stays a string. const receipt = await session.answer({ - questionId: question.questionId, + questionId: question.waitId, value: question.secret ? text : parseAnswer(text, question.schema), }) return { @@ -120,9 +132,14 @@ export async function handleLine( text: error instanceof Error ? error.message : String(error), } } - if (session.snapshot().status === 'running') - await session.send(input, { mode: 'steer' }).receipt - else void session.send(input) + // While a turn runs, the message joins it. Else it starts a turn. The + // session decides this when it stores the message, so there is no race. + const operation = session.send(input, { busy: 'steer' }) + // The view shows a failed turn from its events. + operation.then(undefined, () => {}) + const receipt = await operation.receipt + if (receipt.status === 'rejected') + return { type: 'notice', text: `Not sent: ${receipt.reason}` } return { type: 'sent' } } const [name = '', ...rest] = text.slice(1).split(' ') @@ -143,7 +160,7 @@ export async function handleLine( case 'config': { const [key = '', ...valueParts] = argText.split(' ') if (!key) { - const entries = Object.entries(session.config()) + const entries = Object.entries(session.pluginSettings()) return { type: 'notice', text: @@ -157,7 +174,7 @@ export async function handleLine( .join('\n'), } } - const receipt = await session.setConfig({ + const receipt = await session.setPluginSetting({ key, value: parseArg(valueParts.join(' ').trim()), }) @@ -223,7 +240,7 @@ export async function handleLine( text: 'The agent input must be JSON, for example {"vendor":"acme"}.', } } - const operation = handle.start(input, { wake: true }) + const operation = handle.start(input, { startTurnWhenDone: true }) operation.then( () => {}, () => {}, diff --git a/packages/ai-harness-cli/src/index.ts b/packages/ai-harness-cli/src/index.ts index 190ba420bd..405cb2fafe 100644 --- a/packages/ai-harness-cli/src/index.ts +++ b/packages/ai-harness-cli/src/index.ts @@ -6,12 +6,13 @@ import { runLines } from './lines' import { loadOptional } from './optional' import { EXIT, runPrint } from './print' import { createToken, serve } from './serve' +import { cancelTurn, stopOnSignals } from './signals' import type { AnyHarness, HarnessHost, HarnessPersistence, HarnessSession, - Principal, + HarnessUser, UserInput, } from '@tanstack/ai-harness' import type { CredentialStore } from '@tanstack/ai/byok' @@ -23,10 +24,10 @@ import type { } from '@tanstack/ai-harness/view' import type { AcpAgentModule, - DashboardConnectModule, McpHarnessModule, McpStdioModule, } from './optional' +import type { SignalSource } from './signals' /** The error for a flag whose optional package is missing or too old. */ function needs(flag: string, name: string): Error { @@ -35,7 +36,7 @@ function needs(flag: string, name: string): Error { ) } -export interface RunCliOptions { +export interface RunCliOptions { /** Where sessions keep state. Default: in memory. Do not set it with `host`. */ persistence?: HarnessPersistence /** @@ -44,16 +45,19 @@ export interface RunCliOptions { */ credentials?: CredentialStore /** - * Your own host, so the CLI keeps its hooks: leases, recovery, and stores. + * Your own host, so the CLI keeps its hooks: ownership, recovery, and stores. * You own it: the CLI does not close it. Do not set it with `persistence` - * or `credentials`. + * or `credentials`. With host `access`, give `user` too. `--serve` + * then reads the user of each request with `access.getUser`, and `--mcp` + * refuses to start, as stdio has no login. */ - host?: HarnessHost + host?: HarnessHost /** - * Who uses the CLI. Sessions open as this principal, so credentials and - * provider keys are theirs. `--serve` authorizes each request as them. + * Who uses the CLI. Sessions open as this user in every mode, so + * credentials and provider keys are theirs. On a host without access, + * `--serve` runs each request with the token as them. */ - principal?: Principal + user?: U /** Default: `process.argv.slice(2)`. */ argv?: ReadonlyArray stdin?: NodeJS.ReadStream @@ -63,10 +67,43 @@ export interface RunCliOptions { /** * Your own screen for an interactive terminal, with any UI library. It gets * a ready session view and resolves when the user quits. Piped input and - * the other modes (`--print`, `--acp`, `--mcp`, `--serve`, `--dashboard`) do not use it. + * the other modes (`--print`, `--acp`, `--mcp`, `--serve`) do not use it. * `view.send(text)` sends each `@path` file in `text`, as line mode does. + * A saved child agent of a reopened thread shows only its card until the + * screen calls `view.loadAgent({ id })`. */ ui?: (view: SessionView) => Promise | void + /** + * Where the CLI hears SIGINT and SIGTERM. Default: `process`. A test can + * give an `EventEmitter` and emit the signals. + * + * In every mode, a signal cancels the running turn. A second signal, or a + * signal while no turn runs, closes what the CLI opened, and `runCli` + * resolves to 130 (SIGINT) or 143 (SIGTERM). A host that the CLI made + * closes with `recoverable: true`, so a durable host runs the cut work + * again. Your own `host` stays open: the CLI releases only its handle of + * the session, and does not wait for a turn that ignores the cancel. + */ + signals?: SignalSource +} + +/** + * Show `ui` on a view of `session`. Resolves when the ui ends, or when a + * signal stops the CLI (`stopped` resolves). + */ +async function showUi( + session: HarnessSession, + ui: NonNullable, + stopped: Promise, +) { + const view = createSessionView(attachingSource(session)) + try { + await view.ready + await Promise.race([ui(view), stopped]) + } finally { + view.dispose() + } + return EXIT.ok } /** @@ -93,23 +130,24 @@ function attachingSource(session: HarnessSession) { operation.then(undefined, () => {}) return operation.receipt }), - resolve: (resume) => session.resolve(resume), - cancel: (operationId) => session.cancel(operationId), + resolve: (options) => session.resolve(options), + cancel: (options) => session.cancel(options), answer: (answer) => session.answer(answer), - command: (name, input) => session.command(name, input), - setConfig: (change) => session.setConfig(change), + runCommand: (options) => session.runCommand(options), + setPluginSetting: (change) => session.setPluginSetting(change), events: (options) => session.events(options), snapshot: () => session.snapshot(), transcript: () => session.transcript(), describe: () => session.describe(), - mediaUrl: (id) => session.mediaUrl(id), - loadMedia: (id) => session.loadMedia(id), + mediaUrl: (options) => session.mediaUrl(options), + loadMedia: (options) => session.loadMedia(options), } return source } /** - * Run a harness from the terminal. Resolves to the process exit code. + * Run a harness from the terminal. Resolves to the process exit code: 130 + * or 143 when SIGINT or SIGTERM stops it (see `signals`). * * @example * ```ts @@ -117,9 +155,9 @@ function attachingSource(session: HarnessSession) { * process.exitCode = await runCli(studio) * ``` */ -export async function runCli( +export async function runCli( harness: AnyHarness, - options: RunCliOptions = {}, + options: RunCliOptions = {}, ): Promise { const stdout = options.stdout ?? process.stdout const stderr = options.stderr ?? process.stderr @@ -146,14 +184,25 @@ export async function runCli( ) return EXIT.failed } - const host = + const host: HarnessHost = options.host ?? createHarnessHost({ ...(options.persistence ? { persistence: options.persistence } : {}), ...(options.credentials ? { credentials: options.credentials } : {}), }) - const { principal } = options + const { user } = options const approvals = args.yes ? 'auto' : 'ask' + // The session modes cancel the turn of their session. The other modes + // have no turn of their own, so a signal stops them at once. + let cancel = () => false + const stop = stopOnSignals(options.signals ?? process, () => cancel()) + /** The handle of the session modes. The CLI releases it at the end. */ + let session: HarnessSession | undefined + /** + * The run of --print waits for a person. A durable host keeps the wait, + * so a later run (or another machine) answers it. + */ + let isWaiting = false try { if (args.acp) { const acp = await loadOptional( @@ -161,12 +210,26 @@ export async function runCli( 'serveAcp', ) if (!acp) throw needs('--acp', '@tanstack/ai-acp') - const connection = acp.serveAcp({ host, harness }) - await connection.closed - return EXIT.ok + const connection = acp.serveAcp({ host, harness, user }) + const code = await Promise.race([ + connection.closed.then(() => EXIT.ok), + stop.stopped, + ]) + connection.close() + // ponytail: serveAcp reads process.stdin and does not stop at close, + // and a stdin that flows keeps the process alive. Pause it here until + // the ACP stdio stream stops reading at close. + if (stop.signal.aborted) process.stdin.pause() + return code } if (args.mcp) { + // A host with access reads the user of each HTTP request. + if (host.hasAccess) { + throw new Error( + '--mcp serves over stdio, which has no login, so it cannot use a host with access. Give runCli a host without access, and the local user as the user option.', + ) + } const [mcp, stdio] = await Promise.all([ loadOptional( '@tanstack/ai-mcp/harness', @@ -181,109 +244,98 @@ export async function runCli( const server = await mcp.createHarnessMcpServer({ host, harness, + // One local user: every call runs as them. + openAccess: user ? { user } : true, threadId: args.thread, approvals, // A local client can attach the files of the working folder by path. filePaths: [process.cwd()], }) // stdout carries only MCP messages. The server stops when stdin ends. - const ended = new Promise((resolve) => { - process.stdin.once('end', resolve) + const ended = new Promise((resolve) => { + stdin.once('end', () => resolve(EXIT.ok)) }) const served = stdio.serveMCPStdio(server) - await ended + const code = await Promise.race([ended, stop.stopped]) await served.close() - return EXIT.ok - } - - if (args.dashboard) { - const dashboard = await loadOptional( - '@tanstack/ai-dashboard/connect', - 'connectDashboard', - ) - if (!dashboard) throw needs('--dashboard', '@tanstack/ai-dashboard') - const savedToken = env.HARNESS_DASHBOARD_TOKEN - const connection = await dashboard.connectDashboard({ - host, - harness, - url: args.dashboard, - threads: [args.thread], - ...(savedToken ? { token: savedToken } : {}), - onPairingCode: (code) => - stderr.write( - `Pair this host in the dashboard with the code ${code}. Waiting for approval...\n`, - ), - onToken: (token) => - stderr.write( - `Paired. Set HARNESS_DASHBOARD_TOKEN=${token} to skip pairing next time.\n`, - ), - }) - stderr.write(`Connected to ${args.dashboard}. Press Ctrl+C to stop.\n`) - await new Promise((resolve) => { - process.once('SIGINT', resolve) - process.once('SIGTERM', resolve) - }) - connection.close() - return EXIT.ok + return code } if (args.serve) { const token = args.token ?? env.HARNESS_TOKEN ?? createToken() + const mediaKey = args.mediaKey ?? env.HARNESS_MEDIA_KEY const server = await serve({ host, harness, port: args.port, hostname: args.hostname, token, + ...(mediaKey !== undefined ? { mediaKey } : {}), threadId: args.thread, approvals, - ...(principal ? { principal } : {}), + ...(user ? { user } : {}), }) stderr.write(`Serving ${harness.name} at ${server.url}\n`) if (!args.token && !env.HARNESS_TOKEN) stderr.write(`Token: ${token}\n`) - await new Promise((resolve) => { - process.once('SIGINT', resolve) - process.once('SIGTERM', resolve) - }) + const code = await stop.stopped await server.close() - return EXIT.ok + return code } - const session = await host.open(harness, { + session = await host.open(harness, { threadId: args.thread, - ...(principal ? { principal } : {}), + ...(user ? { user } : {}), }) + cancel = cancelTurn(session) // Relative to the working folder, as a relative --media-dir is. const mediaDir = args.mediaDir ?? defaultMediaDir(harness.name) - if (args.print !== undefined) { - return await runPrint(session, args.print, { - output: args.output, - stdout, - stderr, - mediaDir, - }) + const running = + args.print !== undefined + ? runPrint(session, args.print, { + output: args.output, + stdout, + stderr, + mediaDir, + }) + : stdin.isTTY && options.ui + ? showUi(session, options.ui, stop.stopped) + : runLines(session, stdin, stdout, { + interactive: Boolean(stdin.isTTY), + mediaDir, + signal: stop.signal, + onCtrlC: () => stop.receive('SIGINT'), + }).then(() => EXIT.ok) + const code = await Promise.race([running, stop.stopped]) + isWaiting = code === EXIT.needsAction + if (!stop.signal.aborted) { + // A signal that cancelled the turn of --print gives its exit code. + return code === EXIT.cancelled ? (stop.exitCode() ?? code) : code } - if (stdin.isTTY && options.ui) { - const view = createSessionView(attachingSource(session)) - try { - await view.ready - await options.ui(view) - } finally { - view.dispose() - } - } else { - await runLines(session, stdin, stdout, { - openSignIns: Boolean(stdin.isTTY), - mediaDir, - }) + // A host that the CLI made closes now, so the mode ends with its + // session, and its running work stops as on a crash: a durable host runs + // it again. On your own host, the CLI releases its handle and does not + // wait: a turn that ignores the cancel runs on in your host. Work that + // the close cuts can fail, and the signal gives the exit code. + if (options.host) { + await session.close() + running.catch(() => {}) + return await stop.stopped } - return EXIT.ok + await host.close({ recoverable: true }) + await running.catch(() => {}) + return await stop.stopped } catch (error) { stderr.write(`${error instanceof Error ? error.message : String(error)}\n`) return EXIT.failed } finally { - // A host the caller gave stays open: the caller owns it. - if (!options.host) await host.close() + stop.dispose() + await session?.close() + // A host the caller gave stays open: the caller owns it. After a + // signal, its running work stops as on a crash, so a durable host runs + // it again. + if (!options.host) { + await host.close({ recoverable: stop.signal.aborted || isWaiting }) + } } } diff --git a/packages/ai-harness-cli/src/lines.ts b/packages/ai-harness-cli/src/lines.ts index e23916cbee..c3553b0742 100644 --- a/packages/ai-harness-cli/src/lines.ts +++ b/packages/ai-harness-cli/src/lines.ts @@ -1,9 +1,16 @@ import { createInterface } from 'node:readline' import { createSessionView } from '@tanstack/ai-harness/view' import { saveMediaLine } from './attach' -import { handleLine } from './commands' +import { firstQuestion, handleLine } from './commands' import { createPrinter } from './printer' -import { approvalQuestion, openUrl, resolveAll, waitIdle } from './session-view' +import { + answerOf, + openUrl, + turnWaits, + waitIdle, + waitQuestion, +} from './session-view' +import type { RunAgentResumeItem } from '@tanstack/ai' import type { HarnessSession } from '@tanstack/ai-harness' interface Output { @@ -31,16 +38,16 @@ function typedText(raw: string) { /** * Hide what the user types in a terminal while a secret question (a key) - * waits. Raw mode stops the echo. It also stops Ctrl+C, so this sends - * SIGINT itself. Does nothing for a pipe. + * waits. Raw mode stops the echo. It also makes Ctrl+C a typed character + * and not SIGINT, so this calls `onCtrlC`. Does nothing for a pipe. */ -function secretTyping(input: NodeJS.ReadableStream) { +function secretTyping(input: NodeJS.ReadableStream, onCtrlC: () => void) { const terminal = isTerminal(input) ? input : undefined let hidden = false const onKeys = (chunk: unknown) => { if (!String(chunk).includes('\u0003')) return show() - process.kill(process.pid, 'SIGINT') + onCtrlC() } const show = () => { if (!terminal || !hidden) return @@ -62,25 +69,37 @@ function secretTyping(input: NodeJS.ReadableStream) { } /** - * The line mode, for piped input: one message or command per line. Each - * line waits for the turn it started. When a turn stops for approval, the - * next line answers it (`y` approves, anything else rejects). In a - * terminal, the answer to a secret question (a key) is not shown as the - * user types. Line mode never prints the answer. + * The line mode: one message or command per line. When a turn stops for + * its waits, each next line answers one of them, by its kind and schema + * (see `answerOf`: `y` approves an approval). After the last one, the turn + * goes on. A question of a command or a durable tool takes the next line. + * In a terminal, the answer to a secret question (a key) is not shown as + * the user types. Line mode never prints the answer. * - * `openSignIns`: open sign-in links in the browser (for an interactive terminal). + * `interactive`: the user types the lines (a terminal). Each line acts at + * once: `/cancel` cancels the running turn, and a message steers it. Sign-in + * links open in the browser. Without it (piped input), each line waits for + * the turn that the line before it started, so a script runs in order. * `mediaDir`: the folder for the media a turn makes. Each new file is saved * there, with one `[image saved: ]` line. + * `signal`: stop reading lines when it aborts. + * `onCtrlC`: runs on Ctrl+C while typing is hidden, as the terminal then + * sends no SIGINT. */ export async function runLines( session: HarnessSession, input: NodeJS.ReadableStream, stdout: Output, - options: { openSignIns?: boolean; mediaDir: string }, + options: { + interactive?: boolean + mediaDir: string + signal: AbortSignal + onCtrlC: () => void + }, ) { const view = createSessionView(session) const saves: Array> = [] - const typing = secretTyping(input) + const typing = secretTyping(input, options.onCtrlC) const printer = createPrinter((text) => stdout.write(text), { onMedia: (part) => { const saved = saveMediaLine(session, part, options.mediaDir) @@ -92,26 +111,57 @@ export async function runLines( // Line mode prints only what happens from now on. printer.mark(view.store.get()) const printing = view.store.subscribe((state) => printer.print(state)) - const stopOpening = options.openSignIns + const stopOpening = options.interactive ? view.on('signIn', (signIn) => { if (signIn.url) openUrl(signIn.url) }) : () => {} const line = printer.line - const lines = createInterface({ input, crlfDelay: Infinity }) + const lines = createInterface({ + input, + crlfDelay: Infinity, + signal: options.signal, + }) const pendingLater: Array> = [] + /** The answers to the waits of the stopped turn so far, by wait id. */ + const decided = new Map() + /** + * Wait for the turn to end. Then ask for its next wait that has no + * answer, and hide the typing for a secret question. In a terminal, this + * runs while the next lines are read. Only one runs at a time, so each + * wait is asked once. + */ + const afterTurn = async () => { + await waitIdle(session) + const after = session.snapshot() + const ask = + after.status === 'requires_action' + ? turnWaits(after).find(({ waitId }) => !decided.has(waitId)) + : undefined + if (ask) line(waitQuestion(ask)) + if (firstQuestion(after)?.secret) typing.hide() + } + let waiting: Promise | undefined try { for await (const raw of lines) { + // Lines that came before the stop do not run. + if (options.signal.aborted) break const typed = typing.isHidden() ? typedText(raw) : raw typing.show() const snapshot = session.snapshot() - if (snapshot.status === 'requires_action') { - await resolveAll( - session, - snapshot.pendingInterrupts, - /^y(es)?$/i.test(typed.trim()), - ) + const waits = turnWaits(snapshot) + const next = waits.find(({ waitId }) => !decided.has(waitId)) + if (snapshot.status === 'requires_action' && next) { + decided.set(next.waitId, answerOf(next, typed)) + // One resolve answers every wait of the turn. + if (waits.every(({ waitId }) => decided.has(waitId))) { + const resume = waits.flatMap( + ({ waitId }) => decided.get(waitId) ?? [], + ) + decided.clear() + await session.resolve({ resume }) + } } else { const result = await handleLine(session, typed) if (result.type === 'exit') break @@ -120,17 +170,21 @@ export async function runLines( pendingLater.push(result.later.then((text) => line(text))) } } - await waitIdle(session) - const after = session.snapshot() - if (after.status === 'requires_action') { - line(approvalQuestion(after.pendingInterrupts)) + if (!options.interactive) { + await afterTurn() + continue } - if (after.pendingQuestions[0]?.secret) typing.hide() + // A secret question that the line asked hides the next line at once. + if (firstQuestion(session.snapshot())?.secret) typing.hide() + waiting ??= afterTurn().finally(() => { + waiting = undefined + }) } } finally { typing.show() } lines.close() + await waiting await Promise.allSettled(pendingLater) await waitIdle(session) stopOpening() diff --git a/packages/ai-harness-cli/src/optional.ts b/packages/ai-harness-cli/src/optional.ts index 2fc807be6a..b4e107f85e 100644 --- a/packages/ai-harness-cli/src/optional.ts +++ b/packages/ai-harness-cli/src/optional.ts @@ -1,24 +1,54 @@ -import type { AnyHarness, HarnessHost } from '@tanstack/ai-harness' +import type { AnyHarness, HarnessHost, HarnessUser } from '@tanstack/ai-harness' +import type { OpenAccess } from '@tanstack/ai-harness/server' -// The packages behind `--acp`, `--mcp`, `/mcp`, and `--dashboard` are -// optional. The specifier is a variable, so a build or a type check never -// needs them, and a version without harness support is the same as none. +// The packages behind `--acp`, `--mcp`, and `/mcp` are optional. The +// specifier is a variable, so a build or a type check never needs them, and +// a version without harness support is the same as none. /** * Import `specifier` and check that it has the functions in `names`. - * Resolves to `undefined` when the package is not installed, or when it has - * no such functions. + * Resolves to `undefined` when the package is not installed, when its + * version does not export `specifier`, or when it has no such functions. + * Any other error (a syntax error, or a failing import inside the module) + * throws, with `specifier` in the message. */ export async function loadOptional( specifier: string, ...names: Array -): Promise { - const loaded: unknown = await import(/* @vite-ignore */ specifier).catch( - () => undefined, - ) +) { + // Resolve first: a resolve error is about `specifier` itself, so its code + // alone says "not installed". A missing package that the module imports + // fails the import below, and throws. + try { + import.meta.resolve(specifier) + } catch (error) { + if (isMissing(error)) return undefined + throw loadError(specifier, error) + } + let loaded: unknown + try { + loaded = await import(/* @vite-ignore */ specifier) + } catch (error) { + throw loadError(specifier, error) + } return hasFunctions(loaded, names) ? loaded : undefined } +/** True when a resolve error says that the package or its export is not there. */ +function isMissing(error: unknown) { + if (!(error instanceof Error) || !('code' in error)) return false + return ( + error.code === 'ERR_MODULE_NOT_FOUND' || + error.code === 'MODULE_NOT_FOUND' || + error.code === 'ERR_PACKAGE_PATH_NOT_EXPORTED' + ) +} + +function loadError(specifier: string, error: unknown) { + const reason = error instanceof Error ? error.message : String(error) + return new Error(`Could not load ${specifier}: ${reason}`, { cause: error }) +} + function hasFunctions( value: unknown, names: ReadonlyArray, @@ -37,9 +67,11 @@ export interface McpFetchServer { /** `@tanstack/ai-mcp/harness` */ export interface McpHarnessModule { - createHarnessMcpServer: (options: { - host: HarnessHost + createHarnessMcpServer: (options: { + host: HarnessHost harness: AnyHarness + /** For a host without access: who the calls run as. */ + openAccess?: OpenAccess threadId?: string approvals?: 'ask' | 'auto' filePaths?: Array @@ -53,20 +85,13 @@ export interface McpStdioModule { /** `@tanstack/ai-acp/agent` */ export interface AcpAgentModule { - serveAcp: (options: { host: HarnessHost; harness: AnyHarness }) => { + serveAcp: (options: { + host: HarnessHost + harness: AnyHarness + /** The local user. Required on a host with access. */ + user?: U + }) => { closed: Promise + close: () => void } } - -/** `@tanstack/ai-dashboard/connect` */ -export interface DashboardConnectModule { - connectDashboard: (options: { - host: HarnessHost - harness: AnyHarness - url: string - threads: Array - token?: string - onPairingCode?: (code: string) => void - onToken?: (token: string) => void - }) => Promise<{ close: () => void }> -} diff --git a/packages/ai-harness-cli/src/print.ts b/packages/ai-harness-cli/src/print.ts index 9265a931b7..899da08df9 100644 --- a/packages/ai-harness-cli/src/print.ts +++ b/packages/ai-harness-cli/src/print.ts @@ -1,15 +1,32 @@ import { EventType } from '@tanstack/ai' -import { HARNESS_EVENTS, isMediaRecord } from '@tanstack/ai-harness' +import { HARNESS_EVENTS } from '@tanstack/ai-harness' +import { isMediaRecord } from '@tanstack/ai-harness/media' import { attach, saveMediaLine } from './attach' -import type { HarnessSession } from '@tanstack/ai-harness' +import type { HarnessSession, Wait } from '@tanstack/ai-harness' +/** + * The exit codes of the CLI. `needsAction` (3): the turn waits for a person + * (an approval, an answer, a sign-in). + */ export const EXIT = { ok: 0, failed: 1, - needsAction: 2, + needsAction: 3, cancelled: 130, } as const +/** What `--print` says on stderr when the turn waits for `wait`. */ +function needsActionText(wait: Wait | undefined) { + const what = + wait?.kind === 'approval' + ? 'an approval' + : wait?.kind === 'sign-in' + ? 'a sign-in' + : 'an answer' + const message = wait?.message ? `: ${wait.message}` : '' + return `The turn waits for ${what}${message}. Run without --print, in line mode, to answer it.\n` +} + interface Output { write: (text: string) => unknown } @@ -22,6 +39,10 @@ interface Output { * Each media file the turn makes is saved into `mediaDir`. `ndjson` adds the * saved `path` to the value of its `harness.media` event. `text` prints the * saved line to stderr, so stdout keeps only the answer. + * + * When the turn waits for a person (an approval, a question, a sign-in), + * it stops there and resolves with `EXIT.needsAction`: nobody can answer + * in this mode. */ export async function runPrint( session: HarnessSession, @@ -36,43 +57,70 @@ export async function runPrint( const { stdout, stderr } = options const input = await attach(session, prompt, { cwd: process.cwd() }) const operation = session.send(input) - for await (const chunk of operation.stream()) { - const media = - chunk.type === EventType.CUSTOM && - chunk.name === HARNESS_EVENTS.media && - isMediaRecord(chunk.value) - ? chunk.value - : undefined - if (media !== undefined) { - const saved = await saveMediaLine(session, media, options.mediaDir) - if (options.output === 'text' || saved.path === undefined) { - stderr.write(`${saved.text}\n`) - } - if (options.output === 'ndjson') { - // A file that was not saved has no `path`: JSON drops `undefined`. - const value = { ...media, path: saved.path } - stdout.write(`${JSON.stringify({ ...chunk, value })}\n`) + /** Print the events of the turn. Resolves with its failure, if any. */ + const stream = async (): Promise => { + for await (const chunk of operation.stream()) { + const media = + chunk.type === EventType.CUSTOM && + chunk.name === HARNESS_EVENTS.media && + isMediaRecord(chunk.value) + ? chunk.value + : undefined + if (media !== undefined) { + const saved = await saveMediaLine(session, media, options.mediaDir) + if (options.output === 'text' || saved.path === undefined) { + stderr.write(`${saved.text}\n`) + } + if (options.output === 'ndjson') { + // A file that was not saved has no `path`: JSON drops `undefined`. + const value = { ...media, path: saved.path } + stdout.write(`${JSON.stringify({ ...chunk, value })}\n`) + } + } else if (options.output === 'ndjson') { + stdout.write(`${JSON.stringify(chunk)}\n`) + } else if ( + chunk.type === EventType.TEXT_MESSAGE_CONTENT && + !('subagentRunId' in chunk && chunk.subagentRunId) + ) { + stdout.write(chunk.delta) } - } else if (options.output === 'ndjson') { - stdout.write(`${JSON.stringify(chunk)}\n`) - } else if ( - chunk.type === EventType.TEXT_MESSAGE_CONTENT && - !('subagentRunId' in chunk && chunk.subagentRunId) - ) { - stdout.write(chunk.delta) } + return operation.then( + () => undefined, + (error: unknown) => error, + ) } - const failure: unknown = await operation.then( - () => undefined, - (error: unknown) => error, - ) + const stop = new AbortController() + // A wait of the turn ends the run of --print: nobody can answer it here. + const opened = session + .waitForRun({ runId: operation.id, signal: stop.signal }) + .then( + (wait) => ({ type: 'wait' as const, wait }), + () => new Promise(() => {}), + ) + const ended = stream().then((failure) => ({ type: 'end' as const, failure })) + const first = await Promise.race([ended, opened]) + stop.abort() + const isTurnStop = + first.type === 'wait' && + session + .snapshot() + .pendingWaits.some( + ({ waitId, turn }) => turn && waitId === first.wait.waitId, + ) + if (first.type === 'wait' && !isTurnStop) { + if (options.output === 'text') stdout.write('\n') + stderr.write(needsActionText(first.wait)) + return EXIT.needsAction + } + // A turn that stopped for its waits ends its stream first. + const { failure } = await ended if (options.output === 'text') stdout.write('\n') const status = operation.status() if (status === 'completed') return EXIT.ok if (status === 'interrupted') { - stderr.write( - 'The turn stopped for an approval. Run without --print, in line mode, to answer it.\n', - ) + const [wait] = session.snapshot().pendingWaits + stderr.write(needsActionText(wait)) return EXIT.needsAction } if (status === 'cancelled') return EXIT.cancelled diff --git a/packages/ai-harness-cli/src/printer.ts b/packages/ai-harness-cli/src/printer.ts index f39bb7aaf0..7c008e35a3 100644 --- a/packages/ai-harness-cli/src/printer.ts +++ b/packages/ai-harness-cli/src/printer.ts @@ -23,6 +23,11 @@ export function createPrinter( ) { const done = new Set() const written = new Map() + /** + * The messages and parts that printed. The view keeps a message or a part + * that did not change, so a print walks only the new ones. + */ + const printed = new WeakSet() // True while streamed text has no line break at its end yet. let midLine = false let lastStream = '' @@ -87,19 +92,25 @@ export function createPrinter( } const print = (state: SessionViewState) => { for (const message of state.messages) { + if (printed.has(message)) continue + printed.add(message) if (message.role === 'notice') { if (message.kind === 'error' || message.kind === 'info') once(message.id, message.text) continue } if (message.role !== 'assistant') continue + const fresh: Array = [] message.parts.forEach((part, index) => { + if (printed.has(part)) return + printed.add(part) + fresh.push(part) const key = `${message.id}:${index}` if (part.type === 'text') stream(key, part.text) if (part.type === 'tool-call') once(key, `tool ${part.name}`) if (part.type === 'agent') agent(key, part) }) - media(message.parts) + media(fresh) } for (const question of state.questions) { const isHidden = question.secret === true && options.hidesSecrets === true diff --git a/packages/ai-harness-cli/src/serve.ts b/packages/ai-harness-cli/src/serve.ts index ed8c574847..64fe9e604d 100644 --- a/packages/ai-harness-cli/src/serve.ts +++ b/packages/ai-harness-cli/src/serve.ts @@ -2,43 +2,84 @@ import { createServer } from 'node:http' import { Readable } from 'node:stream' import { pipeline } from 'node:stream/promises' import { randomBytes, timingSafeEqual } from 'node:crypto' -import { createHarnessHandler } from '@tanstack/ai-harness' +import { createHarnessHandler } from '@tanstack/ai-harness/server' import { loadOptional } from './optional' import type { IncomingMessage, ServerResponse } from 'node:http' import type { ReadableStream as NodeReadableStream } from 'node:stream/web' -import type { AnyHarness, HarnessHost, Principal } from '@tanstack/ai-harness' +import type { AnyHarness, HarnessHost, HarnessUser } from '@tanstack/ai-harness' import type { McpHarnessModule } from './optional' -export interface ServeOptions { - host: HarnessHost +export interface ServeOptions { + /** + * The host. With `access`, `access.getUser` reads the user of each + * request that has the token. + */ + host: HarnessHost harness: AnyHarness port: number hostname: string - /** Every request must send `Authorization: Bearer `. */ + /** + * Every request must send `Authorization: Bearer `. Only a signed + * media URL needs no token. On a host with access, the handler still reads + * its user with `getUser`. + */ token: string + /** + * The key that signs media URLs, at least 32 bytes (UTF-8). Default: a new + * random key at each start, so a signed URL stops working after a restart. + * It is never made from `token`: a signed URL is given out without the + * token, and must not let anyone guess it. + */ + mediaKey?: string /** The conversation of an MCP tool call that names none. Default `'main'`. */ threadId?: string /** How `/mcp` handles tool calls that need approval. Default `'ask'`. */ approvals?: 'ask' | 'auto' - /** Who a request with the token is. Default `{ id: 'cli' }`. */ - principal?: Principal + /** + * On a host without access: who a request with the token is. Without it, + * each request runs with no user (one user). + */ + user?: U } -/** The MCP server for `/mcp`, or `undefined` without `@tanstack/ai-mcp`. */ -async function mcpServer(options: ServeOptions) { +/** + * The MCP server for `/mcp`, or `undefined` without `@tanstack/ai-mcp`. It + * runs each call as the same user as the session protocol routes. + */ +async function mcpServer( + options: ServeOptions, + withAccess: boolean, +) { const mcp = await loadOptional( '@tanstack/ai-mcp/harness', 'createHarnessMcpServer', ) if (mcp === undefined) return undefined - return mcp.createHarnessMcpServer({ - host: options.host, - harness: options.harness, + const { host, harness, user } = options + const shared = { + host, + harness, threadId: options.threadId, approvals: options.approvals, + } + // On a host with access, `getUser` reads the user of each request. + if (withAccess) return mcp.createHarnessMcpServer(shared) + return mcp.createHarnessMcpServer({ + ...shared, + openAccess: user ? { user } : true, }) } +/** A signed media URL needs no token: the handler checks its signature. */ +function isSignedMedia(request: Request) { + const url = new URL(request.url) + return ( + request.method === 'GET' && + url.pathname.split('/').at(-1) === 'media' && + url.searchParams.has('sig') + ) +} + /** A random token for `--serve` when none is given. */ export function createToken(): string { return randomBytes(24).toString('base64url') @@ -89,31 +130,40 @@ async function send(res: ServerResponse, response: Response): Promise { /** * Serve the harness session protocol over HTTP (capabilities, run, events, * control, snapshot). With `@tanstack/ai-mcp` installed, `/mcp` serves the - * harness as an MCP server. Every request needs the bearer token. + * harness as an MCP server. Every request needs the bearer token, except a + * signed media URL. Then on a host with access, `getUser` reads the user of + * the request, and the host checks each action. Without access, each + * request runs as `user`. */ -export async function serve( - options: ServeOptions, +export async function serve( + options: ServeOptions, ): Promise<{ url: string; close: () => Promise }> { + const { host, harness, user } = options const isAllowed = (request: Request) => sameToken(request.headers.get('authorization'), options.token) - const principal = options.principal ?? { id: 'cli' } - const handler = createHarnessHandler({ - host: options.host, - harness: options.harness, - authorize: (request) => (isAllowed(request) ? principal : null), - // One principal holds the token, so it can use every thread. - canAccess: () => true, - // ponytail: the token signs media URLs, so they work as long as it does. - mediaSecret: options.token, - }) - const mcp = await mcpServer(options) + const withAccess = host.hasAccess + // Without `mediaKey`, the handler signs with a new random key, so signed + // media URLs stop working after a restart. + const media = + options.mediaKey !== undefined + ? { media: { secret: options.mediaKey } } + : {} + const handler = withAccess + ? createHarnessHandler({ host, harness, ...media }) + : createHarnessHandler({ + host, + harness, + // One user holds the token, so it can use every thread. + openAccess: user ? { user } : true, + ...media, + }) + const mcp = await mcpServer(options, withAccess) const route = (request: Request) => { - const isMcp = mcp !== undefined && new URL(request.url).pathname === '/mcp' - if (!isMcp) return handler(request) - if (!isAllowed(request)) { + if (!isAllowed(request) && !isSignedMedia(request)) { return Response.json({ error: 'unauthorized' }, { status: 401 }) } - return mcp.fetch(request) + const isMcp = mcp !== undefined && new URL(request.url).pathname === '/mcp' + return isMcp ? mcp.fetch(request) : handler(request) } const server = createServer((req, res) => { const aborter = new AbortController() diff --git a/packages/ai-harness-cli/src/session-view.ts b/packages/ai-harness-cli/src/session-view.ts index 29dea42704..924eb29696 100644 --- a/packages/ai-harness-cli/src/session-view.ts +++ b/packages/ai-harness-cli/src/session-view.ts @@ -1,19 +1,72 @@ -import type { Interrupt, RunAgentResumeItem } from '@tanstack/ai' -import type { HarnessSession } from '@tanstack/ai-harness' +import { parseAnswer } from './commands' +import type { RunAgentResumeItem } from '@tanstack/ai' +import type { + HarnessSession, + SessionSnapshot, + Wait, +} from '@tanstack/ai-harness' import type { SignIn } from '@tanstack/ai-harness/view' -/** Answer every open interrupt of the last turn with one decision. */ -export function resolveAll( - session: HarnessSession, - interrupts: ReadonlyArray, - approved: boolean, -) { - const resume: Array = interrupts.map((interrupt) => ({ - interruptId: interrupt.id, +/** The waits of the stopped turn, in order. One resolve answers them all. */ +export function turnWaits(snapshot: SessionSnapshot) { + return snapshot.pendingWaits.filter(({ turn }) => turn) +} + +const isYes = (text: string) => /^y(es)?$/i.test(text.trim()) + +/** + * The answer of the reply `text` to `wait`, by its kind and schema: + * + * - `approval`: `y` approves and `n` rejects. Any other reply is read with + * the schema of the wait (for example `always` for an approval of + * `permissions()`), and an approval with no schema rejects it. + * - `question` and `human`: the reply, read with the schema. + * - `sign-in`: `y` goes on after the sign-in. Any other reply skips it. + * - `client-tool`: a terminal cannot run it, so any reply skips it. + */ +export function answerOf(wait: Wait, text: string): RunAgentResumeItem { + const resolved = (payload: unknown): RunAgentResumeItem => ({ + interruptId: wait.waitId, status: 'resolved', - payload: approved, - })) - return session.resolve(resume) + payload, + }) + const skipped: RunAgentResumeItem = { + interruptId: wait.waitId, + status: 'cancelled', + } + switch (wait.kind) { + case 'approval': { + const reply = text.trim() + if (isYes(reply)) return resolved(true) + if (/^no?$/i.test(reply) || wait.schema === undefined) { + return resolved(false) + } + return resolved(parseAnswer(reply, wait.schema)) + } + case 'question': + case 'human': + return resolved(parseAnswer(text, wait.schema)) + case 'sign-in': + return isYes(text) ? resolved(true) : skipped + case 'client-tool': + return skipped + } +} + +/** The line that asks the user for the answer to `wait`. */ +export function waitQuestion(wait: Wait) { + const message = wait.message ?? wait.waitId + switch (wait.kind) { + case 'approval': + return `Approve ${message}? [y/n]` + case 'question': + case 'human': + return `? ${message}` + case 'sign-in': + return `${message} Sign in, then answer y to go on, or n to skip. [y/n]` + case 'client-tool': + return `${message}: this tool runs in a client, not here. Send any line to skip it.` + } } /** Open an http(s) URL in the default browser. No shell is involved. */ @@ -35,22 +88,14 @@ export function openUrl(url: string) { }) } -/** A short question for the open interrupts. */ -export function approvalQuestion(interrupts: ReadonlyArray) { - const names = interrupts.map( - (interrupt) => interrupt.message ?? interrupt.toolCallId ?? interrupt.id, - ) - return `Approve ${names.join(', ')}? [y/n]` -} - /** - * Resolves when no chat turn runs or waits in the queue, or when the - * session waits for an answer to a question. + * Resolves when no chat turn runs or waits in the queue, or when a wait is + * open (see `session.waitForRun`): the user answers it on the next line. */ export async function waitIdle(session: HarnessSession) { while (true) { const snapshot = session.snapshot() - if (snapshot.pendingQuestions.length > 0) return + if (snapshot.pendingWaits.length > 0) return const chatActive = snapshot.activeOperations.some( (operation) => operation.kind === 'chat', ) diff --git a/packages/ai-harness-cli/src/signals.ts b/packages/ai-harness-cli/src/signals.ts new file mode 100644 index 0000000000..25695d0507 --- /dev/null +++ b/packages/ai-harness-cli/src/signals.ts @@ -0,0 +1,83 @@ +import { EXIT } from './print' +import type { HarnessSession } from '@tanstack/ai-harness' + +/** The signals that stop the CLI. */ +export type StopSignal = 'SIGINT' | 'SIGTERM' + +/** + * Where the CLI hears SIGINT and SIGTERM. `process` is one. An + * `EventEmitter` is one too, so a test can emit the signals. + */ +export interface SignalSource { + on: (signal: StopSignal, handler: () => void) => unknown + off: (signal: StopSignal, handler: () => void) => unknown +} + +/** The exit code of a process that a signal stopped: 128 plus its number. */ +const EXIT_CODES = { SIGINT: EXIT.cancelled, SIGTERM: 143 } + +/** + * Listen for SIGINT and SIGTERM on `source` while a CLI mode runs. + * + * A signal first calls `cancel`, which stops the running turn and returns + * `true`. When `cancel` returns `false` (no turn runs, or a signal cancelled + * this turn already), the signal stops the mode: the listeners go, `signal` + * aborts, and `stopped` resolves with the exit code, 130 for SIGINT or 143 + * for SIGTERM. A signal after that gets the default action of `process`, so + * a close that does not end can still be killed. + */ +export function stopOnSignals(source: SignalSource, cancel: () => boolean) { + const controller = new AbortController() + let resolveStopped: (code: number) => void = () => {} + const stopped = new Promise((resolve) => { + resolveStopped = resolve + }) + let last: StopSignal | undefined + const receive = (signal: StopSignal) => { + if (controller.signal.aborted) return + last = signal + if (cancel()) return + dispose() + controller.abort() + resolveStopped(EXIT_CODES[signal]) + } + const onInterrupt = () => receive('SIGINT') + const onTerminate = () => receive('SIGTERM') + function dispose() { + source.off('SIGINT', onInterrupt) + source.off('SIGTERM', onTerminate) + } + source.on('SIGINT', onInterrupt) + source.on('SIGTERM', onTerminate) + return { + /** Aborts when a signal stops the mode. */ + signal: controller.signal, + /** Resolves with the exit code when a signal stops the mode. */ + stopped, + /** The exit code of the last signal. `undefined` before a signal. */ + exitCode: () => (last === undefined ? undefined : EXIT_CODES[last]), + /** Handle `signal` as if `source` sent it. */ + receive, + /** Stop listening. */ + dispose, + } +} + +/** + * The `cancel` of `stopOnSignals` for a session: cancel its running turn. + * Returns `false` when no turn runs, or when it cancelled this turn + * already, so a second signal stops a turn that does not end. + */ +export function cancelTurn(session: HarnessSession) { + let cancelled: string | undefined + return () => { + const turn = session + .snapshot() + .activeOperations.find((operation) => operation.kind === 'chat') + if (turn === undefined || turn.id === cancelled) return false + cancelled = turn.id + // When the cancel fails, the turn runs on, and the next signal closes. + void session.cancel({ operationId: turn.id }).catch(() => {}) + return true + } +} diff --git a/packages/ai-harness-cli/tests/attach.test.ts b/packages/ai-harness-cli/tests/attach.test.ts index 1f0aaea15b..6d4fd9c72a 100644 --- a/packages/ai-harness-cli/tests/attach.test.ts +++ b/packages/ai-harness-cli/tests/attach.test.ts @@ -10,11 +10,8 @@ import { it, vi, } from 'vitest' -import { - createHarnessHost, - defineHarness, - mediaIdOf, -} from '@tanstack/ai-harness' +import { createHarnessHost, defineHarness } from '@tanstack/ai-harness' +import { mediaIdOf } from '@tanstack/ai-harness/media' import { memoryPersistence } from '@tanstack/ai-persistence' import { attach, defaultMediaDir, saveMedia } from '../src/attach' import type { AnyTextAdapter } from '@tanstack/ai' @@ -78,7 +75,9 @@ beforeAll(async () => { host = createHarnessHost({ persistence: memoryPersistence() }) session = await host.open( defineHarness({ name: 'test/attach', adapter: idleModel() }), - { threadId: 't' }, + { + threadId: 't', + }, ) }) @@ -97,8 +96,8 @@ async function storedFiles(input: UserInput) { const ids = parts.flatMap((part) => mediaIdOf(part) ?? []) return Promise.all( ids.map(async (id) => ({ - name: (await session.getMedia(id))?.name, - text: new TextDecoder().decode(await session.loadMedia(id)), + name: (await session.getMedia({ id }))?.name, + text: new TextDecoder().decode(await session.loadMedia({ id })), })), ) } @@ -161,7 +160,8 @@ describe('attach', () => { describe('saveMedia', () => { it('makes the folder and adds a number instead of overwriting a file', async () => { - const record = await session.putMedia(new TextEncoder().encode('hello'), { + const record = await session.putMedia({ + body: new TextEncoder().encode('hello'), mimeType: 'image/png', name: 'cat.png', }) @@ -184,7 +184,8 @@ describe('saveMedia', () => { it.each(['../escaped.png', 'sub/escaped.png', 'sub\\escaped.png', '..'])( 'refuses the name %s', async (name) => { - const record = await session.putMedia(new TextEncoder().encode('x'), { + const record = await session.putMedia({ + body: new TextEncoder().encode('x'), mimeType: 'image/png', name, }) diff --git a/packages/ai-harness-cli/tests/child-view.test.ts b/packages/ai-harness-cli/tests/child-view.test.ts index f06771b9fa..996148a962 100644 --- a/packages/ai-harness-cli/tests/child-view.test.ts +++ b/packages/ai-harness-cli/tests/child-view.test.ts @@ -117,7 +117,9 @@ describe('line mode with a child agent', () => { adapter: lead, subagents: { agents: [helper] }, }), - { threadId: 't' }, + { + threadId: 't', + }, ) const input = new PassThrough() let out = '' @@ -126,7 +128,11 @@ describe('line mode with a child agent', () => { session, input, { write: (text: string) => (out += text) }, - { mediaDir: join(tmpdir(), 'child-view-media') }, + { + mediaDir: join(tmpdir(), 'child-view-media'), + signal: new AbortController().signal, + onCtrlC: () => {}, + }, ) input.end('go\n') await running diff --git a/packages/ai-harness-cli/tests/cli-media.test.ts b/packages/ai-harness-cli/tests/cli-media.test.ts index 5cce9b5142..f7f97d032f 100644 --- a/packages/ai-harness-cli/tests/cli-media.test.ts +++ b/packages/ai-harness-cli/tests/cli-media.test.ts @@ -1,118 +1,13 @@ import { mkdtemp, readFile, readdir, rm, writeFile } from 'node:fs/promises' import { tmpdir } from 'node:os' import { join } from 'node:path' -import { Readable } from 'node:stream' import { afterAll, beforeAll, describe, expect, it, vi } from 'vitest' -import { EventType, defineAgent } from '@tanstack/ai' +import { defineAgent } from '@tanstack/ai' import { defineHarness } from '@tanstack/ai-harness' import { memoryPersistence } from '@tanstack/ai-persistence' import { EXIT, runCli } from '../src' -import type { AnyTextAdapter, ImageAdapter, StreamChunk } from '@tanstack/ai' - -// ponytail: `textTurn`, `toolTurn`, `scripted`, `capture`, and `stdinFrom` -// repeat the ones in cli.test.ts. Move them to a helpers file when a third -// test file needs them. -const now = () => Date.now() -const textTurn = (text: string): Array => [ - { type: EventType.RUN_STARTED, runId: 'r', threadId: 't', timestamp: now() }, - { - type: EventType.TEXT_MESSAGE_START, - messageId: 'm', - role: 'assistant', - timestamp: now(), - }, - { - type: EventType.TEXT_MESSAGE_CONTENT, - messageId: 'm', - delta: text, - timestamp: now(), - }, - { type: EventType.TEXT_MESSAGE_END, messageId: 'm', timestamp: now() }, - { - type: EventType.RUN_FINISHED, - runId: 'r', - threadId: 't', - timestamp: now(), - metadata: { tanstack: { finishReason: 'stop' } }, - }, -] -const toolTurn = (name: string, args: string): Array => [ - { type: EventType.RUN_STARTED, runId: 'r', threadId: 't', timestamp: now() }, - { - type: EventType.TOOL_CALL_START, - toolCallId: 'call_1', - toolCallName: name, - timestamp: now(), - }, - { - type: EventType.TOOL_CALL_ARGS, - toolCallId: 'call_1', - delta: args, - timestamp: now(), - }, - { type: EventType.TOOL_CALL_END, toolCallId: 'call_1', timestamp: now() }, - { - type: EventType.RUN_FINISHED, - runId: 'r', - threadId: 't', - timestamp: now(), - metadata: { tanstack: { finishReason: 'tool_calls' } }, - }, -] - -function scripted(turns: Array>) { - let call = 0 - const seen: Array> = [] - const adapter: AnyTextAdapter = { - kind: 'text', - name: 'mock', - model: 'test-model', - // `~types` holds types only. Its values are never read, so they are casts. - '~types': { - providerOptions: {} as Record, - inputModalities: ['text'] as readonly ['text'], - messageMetadataByModality: { - text: undefined as unknown, - image: undefined as unknown, - audio: undefined as unknown, - video: undefined as unknown, - document: undefined as unknown, - }, - toolCapabilities: [] as ReadonlyArray, - toolCallMetadata: undefined as unknown, - systemPromptMetadata: undefined as never, - }, - chatStream: (options) => { - seen.push(options.messages) - const chunks = turns[call] ?? textTurn('') - call += 1 - return (async function* () { - yield* chunks - })() - }, - structuredOutput: async () => ({ data: {}, rawText: '{}' }), - } - return { adapter, seen } -} - -function capture() { - let text = '' - return { - write: (chunk: string) => (text += chunk), - get text() { - return text - }, - } -} - -/** A stdin that reads `lines`, as a terminal or as a pipe. */ -function stdinFrom(lines: Array, { isTTY }: { isTTY: boolean }) { - // NodeJS.ReadStream is a TTY socket. A test cannot make one without a real - // terminal, and runCli only reads the lines and `isTTY`, so this casts. - return Object.assign(Readable.from(lines), { - isTTY, - }) as unknown as NodeJS.ReadStream -} +import { capture, scripted, stdinFrom, textTurn, toolTurn } from './helpers' +import type { ImageAdapter } from '@tanstack/ai' /** An image model that paints the bytes "hello". */ function imageAdapter(): ImageAdapter { diff --git a/packages/ai-harness-cli/tests/cli.test.ts b/packages/ai-harness-cli/tests/cli.test.ts index 1a3172997a..4ea6e030e7 100644 --- a/packages/ai-harness-cli/tests/cli.test.ts +++ b/packages/ai-harness-cli/tests/cli.test.ts @@ -1,7 +1,7 @@ +import { PassThrough } from 'node:stream' import { memoryCredentialStore } from '@tanstack/ai/byok/server' -import { Readable } from 'node:stream' import { describe, expect, it, vi } from 'vitest' -import { EventType, defineAgent, toolDefinition } from '@tanstack/ai' +import { defineAgent, toolDefinition } from '@tanstack/ai' import { z } from 'zod' import { configOption, @@ -9,104 +9,36 @@ import { defineCommand, defineHarness, definePlugin, + durableTool, + membersOnly, } from '@tanstack/ai-harness' +import { question as questionTool } from '@tanstack/ai-harness/plugins' +import { createMCPClient } from '@tanstack/ai-mcp' import { memoryPersistence } from '@tanstack/ai-persistence' import { EXIT, parseCliArgs, runCli } from '../src' import { serve } from '../src/serve' -import type { AnyTextAdapter, StreamChunk } from '@tanstack/ai' - -const now = () => Date.now() -const textTurn = (text: string): Array => [ - { type: EventType.RUN_STARTED, runId: 'r', threadId: 't', timestamp: now() }, - { - type: EventType.TEXT_MESSAGE_START, - messageId: 'm', - role: 'assistant', - timestamp: now(), - }, - { - type: EventType.TEXT_MESSAGE_CONTENT, - messageId: 'm', - delta: text, - timestamp: now(), - }, - { type: EventType.TEXT_MESSAGE_END, messageId: 'm', timestamp: now() }, - { - type: EventType.RUN_FINISHED, - runId: 'r', - threadId: 't', - timestamp: now(), - metadata: { tanstack: { finishReason: 'stop' } }, - }, -] -const toolTurn = (name: string, args: string): Array => [ - { type: EventType.RUN_STARTED, runId: 'r', threadId: 't', timestamp: now() }, - { - type: EventType.TOOL_CALL_START, - toolCallId: 'call_1', - toolCallName: name, - timestamp: now(), - }, - { - type: EventType.TOOL_CALL_ARGS, - toolCallId: 'call_1', - delta: args, - timestamp: now(), - }, - { type: EventType.TOOL_CALL_END, toolCallId: 'call_1', timestamp: now() }, - { - type: EventType.RUN_FINISHED, - runId: 'r', - threadId: 't', - timestamp: now(), - metadata: { tanstack: { finishReason: 'tool_calls' } }, - }, -] - -function scripted(turns: Array>) { - let call = 0 - const seen: Array> = [] - const adapter: AnyTextAdapter = { - kind: 'text', - name: 'mock', - model: 'test-model', - // `~types` holds types only. Its values are never read, so they are casts. - '~types': { - providerOptions: {} as Record, - inputModalities: ['text'] as readonly ['text'], - messageMetadataByModality: { - text: undefined as unknown, - image: undefined as unknown, - audio: undefined as unknown, - video: undefined as unknown, - document: undefined as unknown, - }, - toolCapabilities: [] as ReadonlyArray, - toolCallMetadata: undefined as unknown, - systemPromptMetadata: undefined as never, - }, - chatStream: (options) => { - seen.push(options.messages) - const chunks = turns[call] ?? textTurn('') - call += 1 - return (async function* () { - yield* chunks - })() - }, - structuredOutput: async () => ({ data: {}, rawText: '{}' }), - } - return { adapter, seen } -} +import { + asStdin, + capture, + gate, + heldTurn, + scripted, + stdinFrom, + textTurn, + toolTurn, +} from './helpers' +import type { AnyHarness } from '@tanstack/ai-harness' /** `/connect acme` asks for a key as a secret question, then saves it. */ const acmeKeys = definePlugin({ name: 'test/acme-keys', - setup: (ctx) => ({ + setup: () => ({ commands: { 'connect:acme': defineCommand({ description: 'Save an Acme API key', - run: async (_input, { credentials }) => { - const key = await ctx.session.ask({ + run: async (_input, { credentials, step }) => { + const key = await step.ask({ + name: 'key', message: 'Paste your Acme API key', schema: z.string(), secret: true, @@ -122,23 +54,20 @@ const acmeKeys = definePlugin({ }), }) -function capture() { - let text = '' - return { - write: (chunk: string) => (text += chunk), - get text() { - return text - }, - } -} +const olga = { id: 'olga' } +const cid = { id: 'cid' } +const users = new Map([olga, cid].map((user) => [user.id, user])) -/** A stdin that reads `lines`, as a terminal or as a pipe. */ -function stdinFrom(lines: Array, { isTTY }: { isTTY: boolean }) { - // NodeJS.ReadStream is a TTY socket. A test cannot make one without a real - // terminal, and runCli only reads the lines and `isTTY`, so this casts. - return Object.assign(Readable.from(lines), { - isTTY, - }) as unknown as NodeJS.ReadStream +/** A membersOnly() host whose users come from the `x-user` header. */ +function accessHost() { + return createHarnessHost({ + persistence: memoryPersistence(), + access: { + getUser: ({ request }) => + users.get(request.headers.get('x-user') ?? '') ?? null, + rules: membersOnly(), + }, + }) } describe('parseCliArgs', () => { @@ -192,7 +121,7 @@ describe('print mode', () => { expect(events.at(-1).name).toBe('harness.operation.finished') }) - it('exits 2 when the turn waits for an approval', async () => { + it('exits 3 when the turn waits for an approval', async () => { const { adapter } = scripted([toolTurn('remove', '{}')]) const stderr = capture() const code = await runCli( @@ -215,8 +144,40 @@ describe('print mode', () => { }, ) expect(code).toBe(EXIT.needsAction) + expect(EXIT.needsAction).toBe(3) expect(stderr.text).toContain('approval') }) + + it('exits 3 when the turn waits for the answer to a question', async () => { + const { adapter } = scripted([toolTurn('greet', '{}')]) + const greet = durableTool( + toolDefinition({ + name: 'greet', + description: 'Ask the user for a name', + inputSchema: z.object({}), + }), + async (_args, { step }) => { + const name = await step.ask({ + name: 'name', + message: 'Name?', + schema: z.string(), + }) + return `Hello, ${name}.` + }, + ) + const stderr = capture() + const code = await runCli( + defineHarness({ name: 'test/print-question', adapter, tools: [greet] }), + { + argv: ['-p', 'greet me'], + stdout: capture(), + stderr, + persistence: memoryPersistence(), + }, + ) + expect(code).toBe(EXIT.needsAction) + expect(stderr.text).toContain('an answer: Name?') + }) }) describe('line mode', () => { @@ -266,11 +227,151 @@ describe('line mode', () => { }) }) +describe('line mode in a terminal', () => { + /** Line mode on a terminal that the test types into. */ + function terminal(harness: AnyHarness) { + const input = new PassThrough() + const stdout = capture() + const exit = runCli(harness, { + argv: [], + stdin: asStdin(input, { isTTY: true }), + stdout, + stderr: capture(), + persistence: memoryPersistence(), + }) + return { + stdout, + type: (text: string) => input.write(`${text}\n`), + shows: (text: string) => + vi.waitFor(() => expect(stdout.text).toContain(text)), + quit: () => { + input.end('/exit\n') + return exit + }, + } + } + + it('cancels the running turn at once with /cancel', async () => { + // Never opened: only the cancel can end this turn. + const release = gate() + const { adapter } = scripted([heldTurn('Working on it.', release.opened)]) + const cli = terminal(defineHarness({ name: 'test/cancel', adapter })) + cli.type('write a long story') + await cli.shows('Working on it.') + cli.type('/cancel') + await cli.shows('Cancelled.') + expect(await cli.quit()).toBe(EXIT.ok) + }) + + it('sends a line typed while a turn runs as a steer that the turn sees', async () => { + const release = gate() + const { adapter, seen } = scripted([ + heldTurn('Working on it.', release.opened), + textTurn('Checked the logs too.'), + ]) + const joined: Array = [] + const cli = terminal( + defineHarness({ + name: 'test/steer', + adapter, + turn: { + canJoin: (candidate) => { + joined.push(candidate.message) + return true + }, + }, + }), + ) + cli.type('fix the bug') + await cli.shows('Working on it.') + cli.type('also check the logs') + // The steer waits for the next model call of the running turn. + cli.type('/status') + await cli.shows('Queued turns: 1.') + release.open() + await cli.shows('Checked the logs too.') + expect(await cli.quit()).toBe(EXIT.ok) + expect(joined).toEqual(['also check the logs']) + expect(seen[1]?.at(-1)).toMatchObject({ + role: 'user', + content: 'also check the logs', + }) + }) + + it('asks for an approval once, and the next line answers it', async () => { + const remove = vi.fn(async () => 'removed') + const { adapter } = scripted([ + toolTurn('remove', '{}'), + textTurn('Done removing.'), + ]) + const cli = terminal( + defineHarness({ + name: 'test/approve', + adapter, + tools: [ + toolDefinition({ + name: 'remove', + description: 'Remove', + needsApproval: true, + }).server(remove), + ], + }), + ) + cli.type('remove the file') + await cli.shows('[y/n]') + cli.type('y') + await cli.shows('Done removing.') + expect(await cli.quit()).toBe(EXIT.ok) + expect(remove).toHaveBeenCalledTimes(1) + expect(cli.stdout.text.split('[y/n]')).toHaveLength(2) + }) +}) + +describe('waits in line mode', () => { + it('answers a question of the question tool by its schema', async () => { + const { adapter, seen } = scripted([ + toolTurn( + 'question', + JSON.stringify({ + questions: [ + { + question: 'Which database?', + options: [{ label: 'Postgres' }, { label: 'SQLite' }], + }, + ], + }), + ), + textTurn('Picked.'), + ]) + const stdout = capture() + const code = await runCli( + defineHarness({ + name: 'test/lines-question', + adapter, + plugins: () => [questionTool()], + }), + { + argv: [], + stdin: stdinFrom(['ask me\n', 'SQLite\n', '/exit\n'], { + isTTY: false, + }), + stdout, + stderr: capture(), + persistence: memoryPersistence(), + }, + ) + expect(code).toBe(EXIT.ok) + expect(stdout.text).toContain('? Which database?') + expect(stdout.text).toContain('Picked.') + expect(JSON.stringify(seen[1])).toContain('SQLite') + }) +}) + describe('plugin commands in line mode', () => { it('runs plugin commands, answers questions, and changes settings', async () => { const tools = definePlugin({ name: 'test/tools', - setup: (ctx) => ({ + setup: () => ({ config: { tone: configOption.select({ options: ['plain', 'warm'], @@ -285,8 +386,9 @@ describe('plugin commands in line mode', () => { }), confirm: defineCommand({ description: 'Ask first', - run: async () => { - const sure = await ctx.session.ask({ + run: async (_input, { step }) => { + const sure = await step.ask({ + name: 'sure', message: 'Really?', schema: { type: 'boolean' }, }) @@ -388,8 +490,8 @@ describe('secret answers in line mode', () => { }) }) -describe('a host and a principal', () => { - it('uses the given host, leaves it open, and saves keys for the principal', async () => { +describe('a host and a user', () => { + it('uses the given host, leaves it open, and saves keys for the user', async () => { const persistence = memoryPersistence() const credentials = memoryCredentialStore() const host = createHarnessHost({ persistence, credentials }) @@ -408,7 +510,7 @@ describe('a host and a principal', () => { stdout: capture(), stderr: capture(), host, - principal: { id: 'ada' }, + user: { id: 'ada' }, }, ) expect(code).toBe(EXIT.ok) @@ -423,17 +525,17 @@ describe('a host and a principal', () => { await host.close() }) - it('opens the session as the principal', async () => { + it('opens the session as the user', async () => { const { adapter } = scripted([textTurn('ok')]) const seen: Array = [] const code = await runCli( defineHarness({ - name: 'test/principal', + name: 'test/user', adapter, plugins: () => [ definePlugin({ name: 'test/whoami', - setup: (ctx) => void seen.push(ctx.session.principal?.id), + setup: (ctx) => void seen.push(ctx.session.user?.id), }), ], }), @@ -442,7 +544,7 @@ describe('a host and a principal', () => { stdout: capture(), stderr: capture(), persistence: memoryPersistence(), - principal: { id: 'ada' }, + user: { id: 'ada' }, }, ) expect(code).toBe(EXIT.ok) @@ -467,14 +569,14 @@ describe('a host and a principal', () => { await host.close() }) - it('serves as the principal', async () => { + it('serves as the user', async () => { const persistence = memoryPersistence() const credentials = memoryCredentialStore() const host = createHarnessHost({ persistence, credentials }) const server = await serve({ host, harness: defineHarness({ - name: 'test/serve-principal', + name: 'test/serve-user', adapter: scripted([]).adapter, plugins: () => [acmeKeys], expose: { commands: ['connect:acme'] }, @@ -482,7 +584,7 @@ describe('a host and a principal', () => { port: 0, hostname: '127.0.0.1', token: 'secret', - principal: { id: 'ada' }, + user: { id: 'ada' }, }) try { const control = (input: unknown) => @@ -499,13 +601,13 @@ describe('a host and a principal', () => { const snapshot = await fetch(`${server.url}/snapshot?threadId=t`, { headers: { authorization: 'Bearer secret' }, }) - const [pending] = (await snapshot.json()).pendingQuestions + const [pending] = (await snapshot.json()).pendingWaits expect(pending).toBeDefined() return pending }) await control({ op: 'answer', - questionId: question.questionId, + questionId: question.waitId, value: 'sk-ada', }) await vi.waitFor(async () => @@ -521,6 +623,82 @@ describe('a host and a principal', () => { await host.close() } }) + + it('serves /mcp as the user', async () => { + const host = createHarnessHost({ persistence: memoryPersistence() }) + const server = await serve({ + host, + harness: defineHarness({ + name: 'test/serve-mcp', + adapter: scripted([textTurn('ok')]).adapter, + }), + port: 0, + hostname: '127.0.0.1', + token: 'secret', + user: { id: 'ada' }, + }) + const client = await createMCPClient({ + transport: { + type: 'http', + url: `${server.url}/mcp`, + headers: { authorization: 'Bearer secret' }, + }, + }) + try { + await client.callTool('chat', { threadId: 't-mcp', message: 'hi' }) + + expect(await host.sessions.get({ threadId: 't-mcp' })).toMatchObject({ + members: [{ userId: 'ada', role: 'owner' }], + }) + } finally { + await client.close() + await server.close() + await host.close() + } + }) + + it('passes the user to --acp, so a host with access starts', async () => { + const host = accessHost() + // The agent reads process.stdin. Ending it ends the agent. + const ends: Array<() => void> = [] + const on = vi + .spyOn(process.stdin, 'on') + .mockImplementation((event: string | symbol, listener: () => void) => { + if (event === 'end') ends.push(listener) + return process.stdin + }) + const stderr = capture() + try { + const running = runCli( + defineHarness({ name: 'test/acp', adapter: scripted([]).adapter }), + { argv: ['--acp'], stdout: capture(), stderr, host, user: olga }, + ) + await vi.waitFor(() => expect(ends).toHaveLength(1)) + for (const end of ends) end() + + expect(await running).toBe(EXIT.ok) + expect(stderr.text).toBe('') + } finally { + on.mockRestore() + await host.close() + } + }) + + it('refuses --mcp on a host with access, as stdio has no login', async () => { + const host = accessHost() + const stderr = capture() + + const code = await runCli( + defineHarness({ name: 'test/mcp', adapter: scripted([]).adapter }), + { argv: ['--mcp'], stdout: capture(), stderr, host, user: olga }, + ) + + expect(code).toBe(EXIT.failed) + expect(stderr.text).toBe( + '--mcp serves over stdio, which has no login, so it cannot use a host with access. Give runCli a host without access, and the local user as the user option.\n', + ) + await host.close() + }) }) describe('serve mode', () => { @@ -563,6 +741,77 @@ describe('serve mode', () => { await host.close() } }) + + it('signs media URLs with --media-key, never with the token', async () => { + expect(parseCliArgs(['--serve', '--media-key', 'k'])).toMatchObject({ + mediaKey: 'k', + }) + const host = createHarnessHost({ persistence: memoryPersistence() }) + const start = (mediaKey?: string) => + serve({ + host, + harness: defineHarness({ + name: 'test/serve-key', + adapter: scripted([]).adapter, + }), + port: 0, + hostname: '127.0.0.1', + token: 'secret', + ...(mediaKey !== undefined ? { mediaKey } : {}), + }) + + // The key is checked: a short one is refused at the start. + await expect(start('short')).rejects.toThrow('Media signing key 0 is 5') + // A short token is fine: it does not sign anything. + const server = await start() + await server.close() + await host.close() + }) + + it('reads the user of each request with getUser on a host with access, behind the token', async () => { + const host = accessHost() + const server = await serve({ + host, + harness: defineHarness({ + name: 'test/serve-access', + adapter: scripted([textTurn('ok')]).adapter, + }), + port: 0, + hostname: '127.0.0.1', + token: 'secret', + }) + const call = (path: string, headers: Record) => + fetch(`${server.url}/${path}`, { headers }) + const prompt = (headers: Record) => + fetch(`${server.url}/control`, { + method: 'POST', + headers: { 'content-type': 'application/json', ...headers }, + body: JSON.stringify({ + threadId: 't1', + input: { op: 'prompt', message: 'hi' }, + }), + }) + const token = { authorization: 'Bearer secret' } + try { + expect((await prompt({ ...token, 'x-user': 'olga' })).status).toBe(200) + + const noToken = await call('transcript?threadId=t1', { 'x-user': 'olga' }) + expect(noToken.status).toBe(401) + const noUser = await call('transcript?threadId=t1', token) + expect(noUser.status).toBe(401) + const stranger = await call('transcript?threadId=t1', { + ...token, + 'x-user': 'cid', + }) + expect(stranger.status).toBe(403) + expect( + await host.sessions.get({ threadId: 't1', user: olga }), + ).toMatchObject({ members: [{ userId: 'olga', role: 'owner' }] }) + } finally { + await server.close() + await host.close() + } + }) }) describe('custom ui', () => { diff --git a/packages/ai-harness-cli/tests/commands.test.ts b/packages/ai-harness-cli/tests/commands.test.ts index e1d9c650cf..351b6466c2 100644 --- a/packages/ai-harness-cli/tests/commands.test.ts +++ b/packages/ai-harness-cli/tests/commands.test.ts @@ -83,7 +83,7 @@ function model(): AnyTextAdapter { const tools = definePlugin({ name: 'test/tools', - setup: (ctx) => ({ + setup: () => ({ config: { level: configOption.select({ options: ['low', 'high'], default: 'low' }), }, @@ -108,8 +108,9 @@ const tools = definePlugin({ }), confirm: defineCommand({ description: 'Asks a yes or no question', - run: async () => { - const answer = await ctx.session.ask({ + run: async (_input, { step }) => { + const answer = await step.ask({ + name: 'sure', message: 'Sure?', schema: z.boolean(), }) @@ -145,7 +146,9 @@ async function open(plugins = true) { adapter: model(), ...(plugins ? { agents: [counter], plugins: () => [tools] } : {}), }), - { threadId: 't' }, + { + threadId: 't', + }, ) const say = async (line: string) => { const result = await handleLine(session, line) @@ -213,7 +216,7 @@ describe('handleLine', () => { expect(asked).toMatchObject({ type: 'notice', text: '' }) const later = asked.type === 'notice' ? asked.later : undefined expect(later).toBeDefined() - expect(session.snapshot().pendingQuestions).toHaveLength(1) + expect(session.snapshot().pendingWaits).toHaveLength(1) expect(await handleLine(session, 'maybe')).toMatchObject({ text: expect.stringMatching(/^Answer again: /), }) diff --git a/packages/ai-harness-cli/tests/helpers.ts b/packages/ai-harness-cli/tests/helpers.ts new file mode 100644 index 0000000000..5938d78092 --- /dev/null +++ b/packages/ai-harness-cli/tests/helpers.ts @@ -0,0 +1,157 @@ +import { Readable } from 'node:stream' +import { EventType } from '@tanstack/ai' +import type { AnyTextAdapter, StreamChunk } from '@tanstack/ai' + +const now = () => Date.now() + +/** The chunks of a model call that answers `text`. */ +export const textTurn = (text: string): Array => [ + { type: EventType.RUN_STARTED, runId: 'r', threadId: 't', timestamp: now() }, + { + type: EventType.TEXT_MESSAGE_START, + messageId: 'm', + role: 'assistant', + timestamp: now(), + }, + { + type: EventType.TEXT_MESSAGE_CONTENT, + messageId: 'm', + delta: text, + timestamp: now(), + }, + { type: EventType.TEXT_MESSAGE_END, messageId: 'm', timestamp: now() }, + { + type: EventType.RUN_FINISHED, + runId: 'r', + threadId: 't', + timestamp: now(), + metadata: { tanstack: { finishReason: 'stop' } }, + }, +] + +/** The chunks of a model call that calls the tool `name` with `args`. */ +export const toolTurn = (name: string, args: string): Array => [ + { type: EventType.RUN_STARTED, runId: 'r', threadId: 't', timestamp: now() }, + { + type: EventType.TOOL_CALL_START, + toolCallId: 'call_1', + toolCallName: name, + timestamp: now(), + }, + { + type: EventType.TOOL_CALL_ARGS, + toolCallId: 'call_1', + delta: args, + timestamp: now(), + }, + { type: EventType.TOOL_CALL_END, toolCallId: 'call_1', timestamp: now() }, + { + type: EventType.RUN_FINISHED, + runId: 'r', + threadId: 't', + timestamp: now(), + metadata: { tanstack: { finishReason: 'tool_calls' } }, + }, +] + +/** A model call that the test drives. It gets the abort signal of the call. */ +type LiveTurn = (signal: AbortSignal | undefined) => AsyncIterable + +/** A promise that the test resolves with `open`. */ +export function gate() { + let open = () => {} + const opened = new Promise((resolve) => { + open = resolve + }) + return { open, opened } +} + +/** Resolves when `signal` aborts. Never resolves without a signal. */ +function whenAborted(signal: AbortSignal | undefined) { + return new Promise((resolve) => { + if (signal?.aborted) resolve() + signal?.addEventListener('abort', () => resolve(), { once: true }) + }) +} + +/** + * A model call that streams `text`, then holds until `release` resolves or + * the call is aborted (a cancel). Then it ends. + */ +export function heldTurn(text: string, release: Promise): LiveTurn { + const chunks = textTurn(text) + return async function* (signal) { + yield* chunks.slice(0, 3) + await Promise.race([release, whenAborted(signal)]) + yield* chunks.slice(3) + } +} + +/** + * A model that answers each call with the next of `turns`, and keeps the + * messages of each call in `seen`. + */ +export function scripted(turns: Array | LiveTurn>) { + let call = 0 + const seen: Array> = [] + const adapter: AnyTextAdapter = { + kind: 'text', + name: 'mock', + model: 'test-model', + // `~types` holds types only. Its values are never read, so they are casts. + '~types': { + providerOptions: {} as Record, + inputModalities: ['text'] as readonly ['text'], + messageMetadataByModality: { + text: undefined as unknown, + image: undefined as unknown, + audio: undefined as unknown, + video: undefined as unknown, + document: undefined as unknown, + }, + toolCapabilities: [] as ReadonlyArray, + toolCallMetadata: undefined as unknown, + systemPromptMetadata: undefined as never, + }, + chatStream: (options) => { + seen.push(options.messages) + const turn = turns[call] ?? textTurn('') + call += 1 + if (typeof turn === 'function') { + return turn( + options.abortController?.signal ?? + options.request?.signal ?? + undefined, + ) + } + return (async function* () { + yield* turn + })() + }, + structuredOutput: async () => ({ data: {}, rawText: '{}' }), + } + return { adapter, seen } +} + +/** An stdout or stderr that keeps what is written. */ +export function capture() { + let text = '' + return { + write: (chunk: string) => (text += chunk), + get text() { + return text + }, + } +} + +/** `stream` as the stdin of runCli, as a terminal or as a pipe. */ +export function asStdin(stream: Readable, { isTTY }: { isTTY: boolean }) { + // NodeJS.ReadStream is a TTY socket. A test cannot make one without a real + // terminal, and runCli only reads the lines and `isTTY`, so this casts. + return Object.assign(stream, { isTTY }) as unknown as NodeJS.ReadStream +} + +/** A stdin that reads `lines`, as a terminal or as a pipe. */ +export function stdinFrom(lines: Array, options: { isTTY: boolean }) { + return asStdin(Readable.from(lines), options) +} diff --git a/packages/ai-harness-cli/tests/optional.test.ts b/packages/ai-harness-cli/tests/optional.test.ts index a448cfeffe..022ea3f5f5 100644 --- a/packages/ai-harness-cli/tests/optional.test.ts +++ b/packages/ai-harness-cli/tests/optional.test.ts @@ -1,10 +1,28 @@ -import { describe, expect, it } from 'vitest' +import { mkdtempSync, rmSync, writeFileSync } from 'node:fs' +import { tmpdir } from 'node:os' +import { join } from 'node:path' +import { pathToFileURL } from 'node:url' +import { afterAll, describe, expect, it, vi } from 'vitest' import { fakeText } from '@tanstack/ai/testing' import { defineHarness } from '@tanstack/ai-harness' import { memoryPersistence } from '@tanstack/ai-persistence' import { EXIT, runCli } from '../src' import { loadOptional } from '../src/optional' +// An installed version of @tanstack/ai-acp without harness support: it has +// no serveAcp function. +vi.mock('@tanstack/ai-acp/agent', () => ({ serveAcp: undefined })) + +const folder = mkdtempSync(join(tmpdir(), 'cli-optional-')) +afterAll(() => rmSync(folder, { recursive: true, force: true })) + +/** The URL of a temp module with `source`. */ +function tempModule(name: string, source: string) { + const path = join(folder, name) + writeFileSync(path, source) + return pathToFileURL(path).href +} + describe('optional packages', () => { it('loads a module that has the functions', async () => { const path = await loadOptional<{ join: () => string }>('node:path', 'join') @@ -18,27 +36,49 @@ describe('optional packages', () => { 'run', ), ).toBeUndefined() + expect( + await loadOptional<{ run: () => void }>( + '@tanstack/no-such-package/harness', + 'run', + ), + ).toBeUndefined() // An installed package with no harness support. expect( await loadOptional<{ serveAcp: () => void }>('node:path', 'serveAcp'), ).toBeUndefined() }) - it('fails a flag whose package is missing, with a clear message', async () => { - let text = '' - const harness = defineHarness({ - name: 'test/optional', - adapter: fakeText(), - }) - const code = await runCli(harness, { - argv: ['--dashboard', 'http://127.0.0.1:1'], - stdout: { write: () => {} }, - stderr: { write: (chunk) => (text += chunk) }, - persistence: memoryPersistence(), - }) + it('throws, with the specifier, for a module that fails on import', async () => { + const broken = tempModule('broken.mjs', 'throw new Error("boom")\n') + await expect( + loadOptional<{ run: () => void }>(broken, 'run'), + ).rejects.toThrow(`Could not load ${broken}: boom`) + }) + + it('throws for a missing package that the module imports', async () => { + const importer = tempModule( + 'importer.mjs', + 'import "@tanstack/no-such-dependency"\n', + ) + await expect( + loadOptional<{ run: () => void }>(importer, 'run'), + ).rejects.toThrow(`Could not load ${importer}`) + }) + + it('fails a flag whose package has no harness support, with a clear message', async () => { + let stderr = '' + const code = await runCli( + defineHarness({ name: 'test/optional', adapter: fakeText() }), + { + argv: ['--acp'], + stdout: { write: () => {} }, + stderr: { write: (chunk) => (stderr += chunk) }, + persistence: memoryPersistence(), + }, + ) expect(code).toBe(EXIT.failed) - expect(text).toContain( - '--dashboard needs @tanstack/ai-dashboard with harness support.', + expect(stderr).toContain( + '--acp needs @tanstack/ai-acp with harness support.', ) }) }) diff --git a/packages/ai-harness-cli/tests/printer.test.ts b/packages/ai-harness-cli/tests/printer.test.ts index 87c4767190..dc8bb14d4a 100644 --- a/packages/ai-harness-cli/tests/printer.test.ts +++ b/packages/ai-harness-cli/tests/printer.test.ts @@ -12,6 +12,7 @@ const base: SessionViewState = { interrupts: [], questions: [], signIns: [], + waits: [], agents: [], queuedTurns: 0, waitingInputs: [], @@ -242,4 +243,39 @@ describe('line printer', () => { expect(out).toBe('') expect(onMedia).not.toHaveBeenCalled() }) + + it('walks only new messages and parts, and writes only the new text', () => { + const writes: Array = [] + const printer = createPrinter((text) => writes.push(text)) + let reads = 0 + const oldParts: Array = [{ type: 'text', text: 'Old.' }] + const old = { + id: 'turn-0', + role: 'assistant' as const, + get parts() { + reads += 1 + return oldParts + }, + } + const tool: ViewPart = { + type: 'tool-call', + id: 'c', + name: 'read', + argsText: '{}', + args: {}, + status: 'done', + } + const turn = (text: string) => ({ + id: 'turn-1', + role: 'assistant' as const, + parts: [tool, { type: 'text' as const, text }], + }) + + printer.print({ ...base, messages: [old] }) + printer.print({ ...base, messages: [old, turn('Hi')] }) + printer.print({ ...base, messages: [old, turn('Hi there')] }) + + expect(writes).toEqual(['Old.', '\n', '[tool read]\n', 'Hi', ' there']) + expect(reads).toBe(1) + }) }) diff --git a/packages/ai-harness-cli/tests/signals.test.ts b/packages/ai-harness-cli/tests/signals.test.ts new file mode 100644 index 0000000000..cd2f10fa01 --- /dev/null +++ b/packages/ai-harness-cli/tests/signals.test.ts @@ -0,0 +1,316 @@ +import { EventEmitter } from 'node:events' +import { PassThrough } from 'node:stream' +import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest' +import { fakeText } from '@tanstack/ai/testing' +import { + createHarnessHost, + defineCommand, + defineHarness, + definePlugin, +} from '@tanstack/ai-harness' +import { memoryLogStore, memoryPersistence } from '@tanstack/ai-persistence' +import { z } from 'zod' +import { EXIT, runCli } from '../src' +import { asStdin, capture, stdinFrom } from './helpers' +import type { FakeResponse } from '@tanstack/ai/testing' +import type { AnyHarness } from '@tanstack/ai-harness' +import type { RunCliOptions } from '../src' + +/** + * `vi.waitFor` with more time: CI runs these tests slower, for example under + * coverage, and a poll costs nothing when the check passes early. + */ +const waitFor = (check: () => T | Promise) => + vi.waitFor(check, { timeout: 5_000 }) + +/** Resolves when `signal` aborts. */ +function untilAborted(signal: AbortSignal | undefined) { + return new Promise((resolve) => { + if (signal?.aborted) return resolve() + signal?.addEventListener('abort', () => resolve(), { once: true }) + }) +} + +/** + * A model whose first call waits until it is aborted. With `stuck`, it + * waits for `release` instead, as a model that does not stop at once. Its + * later calls answer with `answers`. `calls` has the signal of each call. + */ +function waitingModel( + answers: Array = [], + { stuck = false } = {}, +) { + let release = () => {} + const released = new Promise((resolve) => { + release = resolve + }) + const calls: Array = [] + const model = fakeText() + model.setResponses([ + async ({ request }) => { + const signal = request.request?.signal ?? request.abortController?.signal + calls.push(signal) + await (stuck ? released : untilAborted(signal)) + // The text stops at the abort, so the call does not finish. + return { text: 'cut' } + }, + ...answers, + ]) + return { model, calls, release } +} + +/** Start runCli with signals that the test emits. */ +function startCli(harness: AnyHarness, options: RunCliOptions) { + const signals = new EventEmitter() + const stdout = capture() + const stderr = capture() + const running = runCli(harness, { stdout, stderr, signals, ...options }) + return { running, signals, stdout, stderr } +} + +/** `/connect acme` asks for a key as a secret question. */ +const acmeKeys = definePlugin({ + name: 'test/acme-keys', + setup: () => ({ + commands: { + 'connect:acme': defineCommand({ + description: 'Save an Acme API key', + run: (_input, { step }) => + step.ask({ + name: 'key', + message: 'Paste your Acme API key', + schema: z.string(), + secret: true, + }), + }), + }, + }), +}) + +describe('signals in line mode', () => { + it('cancels the running turn, reads the next line, and closes on a signal while nothing runs', async () => { + const { model, calls } = waitingModel([{ text: 'Second answer.' }]) + const input = new PassThrough() + const cli = startCli( + defineHarness({ name: 'test/signal-lines', adapter: model }), + { argv: [], stdin: asStdin(input, { isTTY: false }) }, + ) + input.write('hello\n') + await waitFor(() => expect(calls).toHaveLength(1)) + + cli.signals.emit('SIGINT') + await waitFor(() => expect(calls[0]?.aborted).toBe(true)) + input.write('again\n') + await waitFor(() => expect(cli.stdout.text).toContain('Second answer.')) + cli.signals.emit('SIGINT') + + expect(await cli.running).toBe(EXIT.cancelled) + }) + + it('closes on Ctrl+C while a secret question hides typing', async () => { + const rawModes: Array = [] + const input = new PassThrough() + const stdin = Object.assign(asStdin(input, { isTTY: true }), { + setRawMode: (mode: boolean) => rawModes.push(mode), + }) + const cli = startCli( + defineHarness({ + name: 'test/signal-secret', + adapter: fakeText(), + plugins: () => [acmeKeys], + }), + { argv: [], stdin }, + ) + input.write('/connect acme\n') + await waitFor(() => expect(rawModes).toEqual([true])) + + // Raw mode makes Ctrl+C a typed character. + input.write('\u0003') + + expect(await cli.running).toBe(EXIT.cancelled) + expect(rawModes).toEqual([true, false]) + }) +}) + +describe('signals in print mode', () => { + it('cancels the turn on SIGTERM and exits 143', async () => { + const { model, calls } = waitingModel() + const cli = startCli( + defineHarness({ name: 'test/signal-print', adapter: model }), + { argv: ['-p', 'hi'] }, + ) + await waitFor(() => expect(calls).toHaveLength(1)) + + cli.signals.emit('SIGTERM') + + expect(await cli.running).toBe(143) + expect(calls[0]?.aborted).toBe(true) + }) + + it('releases only its handle of your host on a second signal while the cancelled turn runs on', async () => { + const { model, calls, release } = waitingModel([], { stuck: true }) + const harness = defineHarness({ name: 'test/signal-stuck', adapter: model }) + // A session that no handle holds closes at once when it is idle. + const host = createHarnessHost({ + persistence: memoryPersistence(), + lifecycle: { closeIdleSessionsAfterMs: 0 }, + }) + const close = vi.spyOn(host, 'close') + const cli = startCli(harness, { argv: ['-p', 'hi'], host }) + await waitFor(() => expect(calls).toHaveLength(1)) + cli.signals.emit('SIGINT') + await waitFor(() => expect(calls[0]?.aborted).toBe(true)) + + cli.signals.emit('SIGINT') + + // runCli does not wait for the turn that ignores the cancel. + expect(await cli.running).toBe(EXIT.cancelled) + expect(close).not.toHaveBeenCalled() + release() + await waitFor(() => + expect(host.isOpen(harness, { threadId: 'main' })).toBe(false), + ) + await host.close() + }) +}) + +describe('signals in a custom ui', () => { + it('stops a ui that is still open, and releases its handle of your host', async () => { + const harness = defineHarness({ + name: 'test/signal-ui', + adapter: fakeText(), + }) + // A session that no handle holds closes at once when it is idle. + const host = createHarnessHost({ + persistence: memoryPersistence(), + lifecycle: { closeIdleSessionsAfterMs: 0 }, + }) + let shown = false + const cli = startCli(harness, { + argv: [], + stdin: stdinFrom([], { isTTY: true }), + host, + // A ui that the user never quits. + ui: () => { + shown = true + return new Promise(() => {}) + }, + }) + await waitFor(() => expect(shown).toBe(true)) + + cli.signals.emit('SIGTERM') + + expect(await cli.running).toBe(143) + expect(host.isOpen(harness, { threadId: 'main' })).toBe(false) + await host.close() + }) +}) + +describe('signals in --serve', () => { + it('stops on SIGTERM with 143, and the next host runs the cut turn again', async () => { + const persistence = { stores: { log: memoryLogStore() } } + const { model, calls } = waitingModel() + const cli = startCli( + defineHarness({ name: 'test/signal-serve', adapter: model }), + { argv: ['--serve', '--port', '0', '--token', 'secret'], persistence }, + ) + const url = await waitFor(() => { + const [, served] = /at (\S+)\n/.exec(cli.stderr.text) ?? [] + expect(served).toBeDefined() + return served + }) + const receipt = await fetch(`${url}/control`, { + method: 'POST', + headers: { + authorization: 'Bearer secret', + 'content-type': 'application/json', + }, + body: JSON.stringify({ + threadId: 't', + input: { op: 'prompt', message: 'go' }, + }), + }) + const { inputId } = await receipt.json() + await waitFor(() => expect(calls).toHaveLength(1)) + + cli.signals.emit('SIGTERM') + + expect(await cli.running).toBe(143) + await expect(fetch(`${url}/capabilities`)).rejects.toThrow() + const recovered = fakeText() + recovered.setResponses([{ text: 'recovered' }]) + const next = createHarnessHost({ persistence }) + const session = await next.open( + defineHarness({ name: 'test/signal-serve', adapter: recovered }), + { threadId: 't' }, + ) + expect(await session.settled({ inputId })).toMatchObject({ + outcome: 'completed', + }) + await next.close() + }) +}) + +describe('signals in the stdio modes', () => { + // The MCP and ACP servers read process.stdin. The test keeps them off it, + // and keeps the events they listen for. + let listened: Array = [] + beforeEach(() => { + listened = [] + vi.spyOn(process.stdin, 'on').mockImplementation( + (event: string | symbol) => { + listened.push(event) + return process.stdin + }, + ) + }) + afterEach(() => { + vi.restoreAllMocks() + }) + + /** Start `argv` with `options`, and wait until it reads stdio. */ + async function startServing( + argv: Array, + options: RunCliOptions = {}, + ) { + const harness = defineHarness({ + name: 'test/signal-stdio', + adapter: fakeText(), + }) + const cli = startCli(harness, { argv, ...options }) + await waitFor(() => expect(listened).toContain('end')) + return cli + } + + it('ends --mcp when the stdin of runCli ends', async () => { + const stdin = stdinFrom([], { isTTY: false }) + const cli = await startServing(['--mcp'], { stdin }) + + // In a real run, the MCP server reads this stdin, so it ends. + stdin.resume() + + expect(await cli.running).toBe(EXIT.ok) + }) + + it('stops --mcp on SIGINT with 130', async () => { + const cli = await startServing(['--mcp'], { + stdin: stdinFrom([], { isTTY: false }), + }) + + cli.signals.emit('SIGINT') + + expect(await cli.running).toBe(EXIT.cancelled) + }) + + it('stops --acp on SIGINT with 130, and leaves your host open', async () => { + const host = createHarnessHost({ persistence: memoryPersistence() }) + const close = vi.spyOn(host, 'close') + const cli = await startServing(['--acp'], { host }) + + cli.signals.emit('SIGINT') + + expect(await cli.running).toBe(EXIT.cancelled) + expect(close).not.toHaveBeenCalled() + await host.close() + }) +}) diff --git a/packages/ai-harness/package.json b/packages/ai-harness/package.json index 51920da03c..9e66d8adb3 100644 --- a/packages/ai-harness/package.json +++ b/packages/ai-harness/package.json @@ -30,6 +30,18 @@ "types": "./dist/esm/index.d.ts", "import": "./dist/esm/index.js" }, + "./server": { + "types": "./dist/esm/server.d.ts", + "import": "./dist/esm/server.js" + }, + "./oauth": { + "types": "./dist/esm/oauth.d.ts", + "import": "./dist/esm/oauth.js" + }, + "./media": { + "types": "./dist/esm/media-entry.d.ts", + "import": "./dist/esm/media-entry.js" + }, "./client": { "types": "./dist/esm/client.d.ts", "import": "./dist/esm/client.js" @@ -50,6 +62,10 @@ "types": "./dist/esm/first-party/index.d.ts", "import": "./dist/esm/first-party/index.js" }, + "./plugins/node": { + "types": "./dist/esm/first-party/node.d.ts", + "import": "./dist/esm/first-party/node.js" + }, "./plugins/coding": { "types": "./dist/esm/first-party/coding/index.d.ts", "import": "./dist/esm/first-party/coding/index.js" @@ -67,9 +83,11 @@ "test:types": "tsc" }, "dependencies": { + "@tanstack/ai-utils": "workspace:^", "@tanstack/store": "^0.11.1" }, "devDependencies": { + "@opentelemetry/api": "^1.9.0", "@tanstack/ai": "workspace:*", "@tanstack/ai-persistence": "workspace:*", "@vitest/coverage-v8": "4.1.10", @@ -78,12 +96,16 @@ "zod": "^4.2.0" }, "peerDependencies": { + "@opentelemetry/api": ">=1.9.0", "@tanstack/ai": "workspace:^", "@tanstack/ai-persistence": "workspace:^", "@vscode/ripgrep": "^1.15.0", "turndown": "^7.2.0" }, "peerDependenciesMeta": { + "@opentelemetry/api": { + "optional": true + }, "@vscode/ripgrep": { "optional": true }, diff --git a/packages/ai-harness/src/access.ts b/packages/ai-harness/src/access.ts new file mode 100644 index 0000000000..dbec65fa05 --- /dev/null +++ b/packages/ai-harness/src/access.ts @@ -0,0 +1,290 @@ +import type { SessionMember } from '@tanstack/ai-persistence' +import type { HarnessUser } from './types' +import type { WaitKind } from './waits' + +/** + * What a user tries to do with a session. + * + * - `read`: stream, snapshot, transcript, media, and child runs. + * - `list`: see the session in a list of sessions. + * - `send`: prompt, steer, follow up, start agents, and run commands. + * - `answer`: answer a question or an approval of a run. + * - `operator`: answer a `human` wait of any run, and reply as an operator + * who takes over from the agent. A `human` wait takes only `operator`, + * so the user who waits for a person cannot answer it. + * - `cancel`: stop a run. + * - `configure`: change the settings of the thread. + * - `share`: add or remove members. + * - `delete`: delete the session. + * - `fork`: copy the session into a new session. + * - `transfer`: move the conversation to a new thread, maybe of another + * harness, with `host.transfer`. + */ +export type AccessAction = + | 'read' + | 'send' + | 'answer' + | 'operator' + | 'cancel' + | 'configure' + | 'share' + | 'delete' + | 'fork' + | 'transfer' + | 'list' + +/** The session that an action targets, as the index row has it. */ +export interface AccessSession { + threadId: string + tenantId?: string + /** The thread that started this one. Set on a subagent child. */ + parentThreadId?: string + /** + * The users of the session. The owner is the member with the role + * `owner`. Without one, the session has no owner. + */ + members: ReadonlyArray +} + +/** The run that an `answer` or `cancel` targets: who started it. */ +export interface AccessRun { + runId: string + /** The id of the user who started the run. */ + userId?: string +} + +/** The wait that an `answer` or `operator` action answers. */ +export interface AccessWait { + waitId: string + kind: WaitKind +} + +/** What `isAllowed` gets for one action. */ +export interface AuthorizeInput { + user: U + action: AccessAction + session: AccessSession + /** The run that an `answer` or `cancel` targets, when the host knows it. */ + run?: AccessRun + /** The wait that an `answer` or `operator` action answers, when the host knows it. */ + wait?: AccessWait +} + +/** Decide if a user may do one action. Return `false` to refuse. */ +export type Authorize = ( + input: AuthorizeInput, +) => boolean | Promise + +/** + * What the host does with the members list of a session: `share` and + * `unshare` change it, and a fork copies it. + */ +export interface AccessMembers { + /** + * Who is a member of the new session after a fork without a `members` + * option. + * + * - `'self'` (default): only the user who forked it, as the owner. + * - `'all'`: the members of the source session too. + */ + fork?: 'self' | 'all' +} + +/** + * The rules of a host: `rules` in {@link HarnessAccess}. + * {@link membersOnly} returns one, or write your own with only `isAllowed`. + */ +export interface AccessPolicy { + /** + * What a user may do. Called for every action. The harness checks + * the tenant before it calls this. + */ + isAllowed: Authorize + /** + * A filter that `host.sessions.list` passes to the sessions store, so a + * page holds only sessions that the user can list. `isAllowed` still + * checks each entry. Without it, the store gives every session of the + * tenant, and `isAllowed` drops the others, so a page can come back short. + */ + filterSessionList?: (user: U) => { memberId?: string } + /** + * Turn on `host.sessions.share` and `unshare`, which change the members + * list of a session, and set the members of a fork. Without it, `share` + * and `unshare` throw: change your own access tables and call + * `host.revoke`. + */ + members?: AccessMembers +} + +/** + * Who may use the sessions of a host, and what they may do. The type of the + * user comes from the return value of `getUser`. + * + * @example + * access: { + * getUser: ({ request }) => findUser(request), + * rules: membersOnly(), + * } + */ +export interface HarnessAccess { + /** + * Who is calling: your own login check, once per request. Return null to + * refuse with 401. + */ + getUser: (input: { request: Request }) => U | null | Promise + /** + * What each user may do: `membersOnly()`, or your own + * `{ isAllowed, filterSessionList?, members? }`. + */ + rules: AccessPolicy + /** + * A guest for a signed media URL with no login, for example an `` + * that sends no cookies. The handler calls it only on the signed media + * route, after the signature checks out, and only when `getUser` finds no + * user. `tenantId` is the tenant of the URL. `isAllowed` then decides + * `read` for the guest. Return null, or leave it out, to answer 401. A + * guest never reaches another route. + * + * @example + * getGuestUser: ({ tenantId }) => ({ id: 'guest', tenantId, isGuest: true }) + */ + getGuestUser?: (input: { + tenantId: string | undefined + }) => U | null | Promise + /** + * Let keyed (BYOK) adapters and `ctx.keys` fall back to the server's env + * vars. Default `false` when access is on. + */ + allowServerEnvKeys?: boolean +} + +/** + * Thrown when the host refuses an action: `isAllowed` returned `false`, or + * `host.revoke` took the access of the handle's user. The HTTP handler + * answers it with 403. Catch it with `instanceof AccessDeniedError`. + */ +export class AccessDeniedError extends Error { + readonly code = 'forbidden' + readonly action: AccessAction + readonly threadId: string + + constructor({ + action, + threadId, + }: { + action: AccessAction + threadId: string + }) { + super(`Access denied: ${action} on thread ${JSON.stringify(threadId)}.`) + this.name = 'AccessDeniedError' + this.action = action + this.threadId = threadId + } +} + +/** Options for {@link membersOnly}. */ +export interface MembersOnlyOptions { + fork?: { + /** + * Who is a member of the new session after a fork. + * + * - `'self'` (default): only the user who forked it, as the owner. + * - `'all'`: the members of the source session too. + */ + members?: 'self' | 'all' + } + operator?: { + /** + * The roles that have the `operator` action: they answer a `human` wait + * and reply as an operator. Default `['operator']`: only members with + * the role `operator`. The owner and plain members do not have it, so + * the user who waits for a person cannot act as that person. + */ + roles?: ReadonlyArray + } +} + +type Role = SessionMember['role'] + +/** The role of a user in a session. */ +function roleOf(session: AccessSession, userId: string) { + return session.members.find((member) => member.userId === userId)?.role +} + +/** The user id of the owner of a session, or `undefined` for no owner. */ +export function ownerOf(session: Pick) { + return session.members.find((member) => member.role === 'owner')?.userId +} + +function allows( + role: Role, + { user, action, run }: AuthorizeInput, + operators: ReadonlyArray, +) { + switch (action) { + case 'read': + case 'list': + return true + case 'send': + case 'fork': + return role === 'owner' || role === 'member' + case 'operator': + return operators.includes(role) + case 'answer': + case 'cancel': { + const startedTheRun = run?.userId === user.id + return role === 'owner' || (role === 'member' && startedTheRun) + } + case 'configure': + case 'share': + case 'delete': + case 'transfer': + return role === 'owner' + } +} + +/** + * Ready-made access rules: only the members of a session may use it, each + * with the limits of the member's role. Use it as the `rules` of `access`. + * + * - Owner: every action but `operator`. + * - Member: read, list, send, and fork. Answer and cancel only the runs + * that the member started. + * - Operator: read, list, and `operator` (answer a `human` wait, and reply + * as an operator). + * - Viewer: read and list. + * - Not a member: no action. + * + * `operator.roles` changes who has `operator`. + * + * The owner is the member with the role `owner`. `isAllowed` does not check + * the tenant: the host does that before it calls it. For a child thread, the + * host passes the root session, so it has no special rules for children. + * + * It also lists only the sessions of a member (`filterSessionList`), and turns on + * `host.sessions.share` and `unshare` (`members`). + * + * @param options - `fork.members` sets who is a member of a forked session. + * The default is `'self'`. `operator.roles` sets the roles with the + * `operator` action. The default is `['operator']`. + * + * @example + * access: { + * getUser: ({ request }) => findUser(request), + * rules: membersOnly({ fork: { members: 'self' } }), + * } + */ +export function membersOnly( + options: MembersOnlyOptions = {}, +): Required { + const operators = options.operator?.roles ?? ['operator'] + return { + isAllowed: (input) => { + const role = roleOf(input.session, input.user.id) + if (role === undefined) return false + return allows(role, input, operators) + }, + filterSessionList: (user) => ({ memberId: user.id }), + members: { fork: options.fork?.members ?? 'self' }, + } +} diff --git a/packages/ai-harness/src/agents.ts b/packages/ai-harness/src/agents.ts index 681d717c2d..ca739d11bd 100644 --- a/packages/ai-harness/src/agents.ts +++ b/packages/ai-harness/src/agents.ts @@ -3,10 +3,21 @@ import type { DefinedAgent, InferSchemaType, SchemaInput, + StreamChunk, + SubagentRunContext, } from '@tanstack/ai' +import { askWithoutSession, stepRunWithoutSession } from './durable-tool' +import { waitUntil } from './utils' import type { ToolStep } from './durable-tool' -import type { AgentStartOptions } from './session/index' -import type { AgentRun } from './types' +import type { HandoffOptions } from './handoff' +import type { + AgentRunOptions, + AgentStartOptions, + RunArgs, +} from './session/types' +import type { AgentRun, RunContext } from './types' +import type { PluginState } from './plugins' +import type { AnyHarness } from './define' /** Any agent made with `defineAgent`, with its types kept. */ export type AnyAgent = DefinedAgent @@ -106,6 +117,11 @@ export class AgentRegistry implements AgentRegistryView { return this.agents.get(name)?.agent } + /** Who added the agent `name`: `'the harness'` or a plugin name. */ + ownerOf(name: string): string | undefined { + return this.agents.get(name)?.owner + } + find(query: { produces: AgentProduces }): AnyAgent | undefined { return this.list().find((agent) => agent.produces === query.produces) } @@ -118,58 +134,230 @@ export class AgentRegistry implements AgentRegistryView { } } -/** `ctx.harness.agents`: start agents in the background. */ -export interface AgentStarter { +/** + * Start one agent: a definition, typed from its schemas, or a name the + * session knows, untyped. + */ +export interface AgentCall { + ( + agent: TAgent, + ...args: RunArgs + ): AgentRun> + (agent: string, input?: unknown, options?: TOptions): AgentRun +} + +/** How a group of agents runs. */ +export interface AgentGroupOptions { + /** How many children run at the same time. Default: all of them. */ + concurrency?: number + /** + * `'cancel-siblings'` (default): the first failure cancels the other + * children, and `group` rejects with that error. `'collect'`: every child + * runs to its end, and `group` resolves to one entry per child. + */ + onFailure?: 'cancel-siblings' | 'collect' + /** + * The durable name of the group in the run that starts it. Default: the + * order of the `group` calls in that run. + */ + key?: string +} + +/** One child of a `group` with `onFailure: 'collect'`. */ +export type AgentSettled = + | { ok: true; value: TResult } + | { ok: false; error: unknown } + +/** The children of one `group` call. */ +export interface AgentGroup { + /** Start a child of the group. Its run is like the run of `start`. */ + start: AgentCall +} + +/** + * Start agents and run them: `start` in the background, `run` and `group` to + * get the results. `ctx.harness.agents` in agent code and `ctx.agents` in a + * plugin have this shape. + */ +export interface AgentActions { /** * Start `agent` (a definition, or a name the session knows) in the - * background, and return its run at once. The child counts against the - * subagent tree budget of the run that starts it. Over the budget, `start` - * throws and nothing starts. + * background, and return its run at once. In agent code, the child counts + * against the subagent tree budget of the run that starts it. Over the + * budget, `start` throws and nothing starts. */ - start: ( - agent: AnyAgent | string, - input?: unknown, - options?: AgentStartOptions, - ) => AgentRun + start: AgentCall + /** + * `start` without a wake turn. Await the run to get the result. The run + * can still be cancelled and get messages. + */ + run: AgentCall + /** + * Run children together. `body` starts them with `g.start` and returns + * their runs. `group` resolves to their results, in the same order. Every + * child that `body` started settles before `group` returns. + */ + group: { + >>( + options: AgentGroupOptions & { onFailure: 'collect' }, + body: (g: AgentGroup) => TRuns | Promise, + ): Promise<{ + -readonly [K in keyof TRuns]: AgentSettled> + }> + >>( + options: AgentGroupOptions & { onFailure?: 'cancel-siblings' }, + body: (g: AgentGroup) => TRuns | Promise, + ): Promise<{ -readonly [K in keyof TRuns]: Awaited }> + } } /** What a harness session adds to an agent run that it starts from code. */ export interface HarnessAgentContext { /** - * The steps of the run. On a durable host, in a run started with - * `resume: true`, a finished step returns its stored value when the run - * runs again, and `fn` does not run again. Else `fn` runs each time. + * The steps of the run. In a run started with `runAgainAfterCrash: true`, a + * finished step returns its stored value when the run + * runs again, and `fn` does not run again. Else `fn` runs each time, + * and `sleep` waits its full time. */ step: ToolStep - /** Start agents in the background, as children of this run. */ - agents: AgentStarter + /** Start and run agents, as children of this run. */ + agents: AgentActions + /** + * Hand the turn to another root agent, or to `'main'` (the main model). + * The target answers next, in this turn, and the later turns go to it. + * `{ human: true }` hands it to a person, when `routing.allowHandoffToHuman` is true: + * the turn ends `interrupted`, and the session pauses until an operator + * answers the `human` wait. + * It works in a root agent that answers a turn of a harness with + * `routing`. Anywhere else it throws. It throws for a target that is not a + * root agent too. + * + * @example + * ```ts + * await ctx.harness.handoff({ to: 'billing', note: 'Refund for order 42' }) + * ``` + */ + handoff: (options: HandoffOptions) => Promise + /** + * State that the agents of one session share, by `key`. The session log + * keeps it, so it survives a restart, and a fork of the thread has it. + * Before the first update, `get` gives `initial`. The value is JSON. + * + * @example + * ```ts + * const seen = ctx.harness.state({ key: 'seen', initial: [] as Array }) + * await seen.update((list) => [...list, 'docs.md']) + * ``` + */ + state: (options: { key: string; initial: T }) => PluginState } +const noSessionState = () => { + throw new Error( + 'ctx.harness.state needs a run that a harness session started.', + ) +} + +const noSessionAgents = () => { + throw new Error( + 'ctx.harness.agents needs a run that a harness session started.', + ) +} + +const noTurnHandoff = () => + Promise.reject( + new Error( + 'ctx.harness.handoff needs a root agent that answers a turn of a harness with routing.', + ), + ) + /** - * `ctx.harness` outside a run that a session started from code (a tool-mode - * child of a chat turn): `step.do` runs `fn` each time, `agents.start` throws. + * The parts of `ctx.harness` that a tool-mode child of a chat turn gets in + * place of a durable session context: `step.do` + * runs `fn` each time, `step.sleep` waits its full time, `step.ask` + * throws, and each call of `agents`, `state`, and `handoff` throws. */ export const NO_SESSION_CONTEXT: HarnessAgentContext = { - step: { do: async (_name, fn) => fn() }, + // `fn` runs each time, so it needs no JSON copy. + step: { + do: async (name, fn) => fn(stepRunWithoutSession(name)), + sleep: ({ ms }) => waitUntil(Date.now() + ms), + ask: askWithoutSession, + }, agents: { - start: () => { - throw new Error( - 'ctx.harness.agents needs a run that a harness session started.', - ) - }, + start: noSessionAgents, + run: noSessionAgents, + group: noSessionAgents, }, + handoff: noTurnHandoff, + state: noSessionState, +} + +/** + * Run `harness` as a child session of the run that owns a `ctx.harness`, and + * stream its events. See `harnessAgent`. + */ +export type HarnessChildRun = ( + harness: AnyHarness, + ctx: SubagentRunContext, +) => AsyncIterable + +/** + * The child hook of each `ctx.harness` that a session made. Not a field of + * the context, so agent code cannot call it. `NO_SESSION_CONTEXT` has none. + */ +const childRuns = new WeakMap() + +/** The run context of the run of each `ctx.harness` that a session made. */ +const runContexts = new WeakMap() + +/** + * @internal Give `context` the child hook `run`, and the run context of the + * run that it belongs to. Returns `context`. + */ +export function withHarnessChild( + context: HarnessAgentContext, + run: HarnessChildRun, + runContext: RunContext, +): HarnessAgentContext { + childRuns.set(context, run) + runContexts.set(context, runContext) + return context +} + +/** + * @internal The run context of the run of `context`: for an agent that runs + * as a tool of a turn, the run of that turn. `undefined` outside a run that + * a harness session started. + */ +export function runContextOf(context: HarnessAgentContext | undefined) { + return context ? runContexts.get(context) : undefined +} + +/** + * @internal The child hook of `context`, or `undefined` outside a run that + * a harness session started. + */ +export function harnessChildOf(context: HarnessAgentContext | undefined) { + return context ? childRuns.get(context) : undefined } declare module '@tanstack/ai' { /** - * `ctx.harness` in the `run` of an agent: `step` and `agents`. A harness - * session gives them to each agent run it starts from code (a session - * agent, `ctx.harness.agents.start`, a background child), and to the - * children those runs start through `ctx.chat({ subagents })`. A run - * that a plain `chat()` starts outside a harness has no `harness` at - * run time: such an agent reads `ctx.harness` only inside a harness. + * `ctx.harness` in the `run` of an agent: `step`, `agents`, `state`, and + * `handoff`. A harness gives it to every agent run in a session, and to + * the children those runs start through `ctx.chat({ subagents })`. A run + * that a plain `chat()` starts has no `ctx.harness`, so check it before + * you use it. `handoff` works only in a root agent that answers a turn of + * a harness with `routing`. + * + * @example + * ```ts + * if (!ctx.harness) throw new Error('This agent must run in a harness.') + * const total = await ctx.harness.step.do('sum', () => 1 + 2) + * ``` */ interface SubagentContextExtension { - harness: HarnessAgentContext + harness?: HarnessAgentContext } } diff --git a/packages/ai-harness/src/auth.ts b/packages/ai-harness/src/auth.ts index b1c5fb1362..d8a226b7bd 100644 --- a/packages/ai-harness/src/auth.ts +++ b/packages/ai-harness/src/auth.ts @@ -1,17 +1,30 @@ import { isKeyedAdapter } from '@tanstack/ai' -import type { ProviderKeys, Scope } from '@tanstack/ai' +import { scrubSecrets } from '@tanstack/ai/byok' import { envKeyEntry, readCredential, + readCredentialEntry, tenantScopeOf, } from '@tanstack/ai/byok/server' +import type { ChatMiddleware, ProviderKeys, Scope } from '@tanstack/ai' import type { ByokProvider, Credential, + CredentialScope, CredentialStore, ProviderId, } from '@tanstack/ai/byok' +// The harness API had its own copy. The core one is the only body now. +export { scrubSecrets } + +/** + * @internal The id prefix of a secret answer in the credential store. A host + * that does not own the log saves the answer there, and the owner reads it + * once. Its credential type is `secret_answer`, which `list` hides. + */ +export const SECRET_ANSWER_PREFIX = 'harness:answer:' + /** * Thrown when a tool or command needs a credential the user has not saved. * The session publishes `harness.auth_required`, so a host can show the @@ -33,24 +46,46 @@ export class AuthRequiredError extends Error { } /** - * Credentials of the running turn's sender (outside a turn, of the session's - * principal), as plugins see them. + * The credentials of one user: the user of a run (`ctx.context.run`), or of + * a command. Reads and saves go to that user only. */ export interface CredentialsAccess { get: (id: string) => Promise /** * Like `get`, but throws {@link AuthRequiredError} when the credential is - * missing. With `wait: true`, a tool of a chat turn waits for the sign-in - * instead: the turn stops with an interrupt with the reason - * `'auth_required'`. When the credential is saved (for example by - * `connect:`), the turn goes on and the tool runs again. Cancel the - * interrupt to fail the tool. `wait` is for a tool of the running chat - * turn. Do not pass it elsewhere (a command, a background agent, a hook): - * the credentials of a command never wait, and outside a chat turn `wait` - * does nothing. + * missing. With `wait: true`, a tool of a chat turn or of a background + * agent run waits for the sign-in instead: the run stops with an + * interrupt with the reason `'auth_required'`, a `sign-in` wait. When the + * credential is saved (for example by `connect:`), the run goes on + * and the tool runs again. Cancel the interrupt to fail the tool. The + * wait expires after 15 minutes, and then the tool fails too. Do not pass + * `wait` elsewhere (a command, a hook): the credentials of a command never + * wait, and outside a run `wait` does nothing. */ require: (id: string, options?: { wait?: boolean }) => Promise set: (entry: { id: string; credential: Credential }) => Promise + /** + * Swap the saved credential `id` for a new one, for example an OAuth + * refresh. `update` gets the saved credential and returns the new one, or + * the same object to keep it. Resolves with the credential to use, or + * `null` when none is saved. + * + * - The new credential goes back to the scope where it was found. A + * tenant credential stays the tenant's, and is not copied to the user. + * - One host runs one `update` at a time per saved credential. A call + * that comes while one runs waits for it and gets its result. + * - When the store has `getVersioned` and `setIf`, the save is a + * compare-and-set. If another host saved first, the call uses that + * credential, and does not run `update` again. + * - `update` gets the signal of the host, not of a caller: an abort of + * `options.signal` rejects this call only, and the refresh goes on for + * the calls that wait for it. + */ + refresh: ( + id: string, + update: (current: Credential, signal: AbortSignal) => Promise, + options?: { signal?: AbortSignal }, + ) => Promise delete: (id: string) => Promise list: () => Promise< Array<{ id: string; type: Credential['type']; expiresAt?: number }> @@ -61,14 +96,58 @@ export interface CredentialsAccess { * What a session adds to `credentialsFor`: `canWait` says if a `require` * with `wait` can stop the turn now, and `onSet` runs after each save. * `secrets` gets each secret value the access reads or saves, so the session - * can scrub them from the errors it sends to clients. + * can scrub them from what it stores, publishes, and sends to a model. */ export interface CredentialHooks { canWait?: (id: string) => boolean onSet?: (id: string) => Promise - secrets?: Set + secrets?: Redactor + /** + * The refreshes that run now, shared by every access of one host. See + * {@link RefreshFlights}. Without it, only the calls of this access share + * a refresh. + */ + refreshes?: RefreshFlights +} + +/** + * @internal The refreshes of one host. A host keeps one for all its + * sessions. + */ +export interface RefreshFlights { + /** The refresh that runs now for each saved credential, by {@link refreshKey}. */ + running: Map> + /** The signal of each refresh. The host aborts it when it closes. */ + signal: AbortSignal +} + +/** A {@link RefreshFlights} whose signal aborts with `controller`. */ +export function refreshFlights( + controller = new AbortController(), +): RefreshFlights { + return { running: new Map(), signal: controller.signal } } +/** + * `promise`, or a rejection with the reason of `signal` when it aborts + * first. The work of `promise` goes on. + */ +function untilAborted(promise: Promise, signal?: AbortSignal) { + if (!signal) return promise + if (signal.aborted) return Promise.reject(signal.reason) + return new Promise((resolve, reject) => { + const onAbort = () => reject(signal.reason) + signal.addEventListener('abort', onAbort, { once: true }) + promise + .then(resolve, reject) + .finally(() => signal.removeEventListener('abort', onAbort)) + }) +} + +/** The key of one saved credential: its tenant, its user, and its id. */ +const refreshKey = (scope: CredentialScope, id: string) => + JSON.stringify([scope.tenantId ?? null, scope.userId ?? null, id]) + /** * The error that chat() reads as "stop the turn for input" (the shape of * `MCPInputRequiredError`), with the reason `'auth_required'`. @@ -86,7 +165,7 @@ function signInRequired(error: AuthRequiredError) { } function secretValues(credential: Credential) { - if (credential.type === 'api_key') return [credential.value] + if (credential.type !== 'oauth') return [credential.value] const { accessToken, refreshToken, client } = credential return [accessToken, refreshToken, client?.clientSecret].filter( (value) => value !== undefined, @@ -94,19 +173,18 @@ function secretValues(credential: Credential) { } /** - * Scope credentials to one principal and thread. A function `scope` is read - * at each call, so the scope can follow the sender of the running turn. + * Scope credentials to one user and thread. * * A read finds the user's own credential first, then the tenant's (saved - * without a `userId`). A save or a delete changes the user's own only. + * without a `userId`). A save or a delete changes the user's own only. A + * `refresh` writes back to the scope where it found the credential. */ export function credentialsFor( store: CredentialStore, - scope: Scope | (() => Scope), + scope: Scope, onMissing: (error: AuthRequiredError) => void, hooks?: CredentialHooks, ): CredentialsAccess { - const scopeNow = typeof scope === 'function' ? scope : () => scope const secrets = hooks?.secrets const keep = (credential: Credential | null) => { for (const value of credential ? secretValues(credential) : []) { @@ -114,11 +192,49 @@ export function credentialsFor( } return credential } - // ponytail: a refreshed tenant OAuth token is saved for the user, so each - // user keeps a copy. A provider that rotates refresh tokens would need a - // save for the whole tenant to share one token. - const get = async (id: string) => - keep(await readCredential(store, scopeNow(), id)) + const get = async (id: string) => keep(await readCredential(store, scope, id)) + const flights = hooks?.refreshes ?? refreshFlights() + type Entry = NonNullable>> + /** True when the saved credential is not the one in `seen` any more. */ + const isChanged = (seen: Entry, now: Entry) => + seen.revision !== undefined && now.revision !== undefined + ? seen.revision !== now.revision + : JSON.stringify(seen.credential) !== JSON.stringify(now.credential) + const runRefresh = async ( + id: string, + seen: Entry, + update: (current: Credential, signal: AbortSignal) => Promise, + ): Promise => { + let next: Credential + try { + next = await update(seen.credential, flights.signal) + } catch (error) { + // Another host can refresh first and use up the refresh token, so + // this refresh fails. Then the token that host saved wins. + const now = await readCredentialEntry(store, scope, id) + if (now && isChanged(seen, now)) return now.credential + throw error + } + if (next === seen.credential) return next + if (store.setIf && seen.revision !== undefined) { + const saved = await store.setIf({ + scope: seen.scope, + id, + credential: next, + expectedRevision: seen.revision, + }) + if (saved.ok) return next + // Another host saved first. Use its credential, and do not refresh + // again: with refresh token rotation, a second refresh can revoke + // both tokens. + return (await readCredentialEntry(store, scope, id))?.credential ?? null + } + // A store without `getVersioned` and `setIf` has no compare-and-set. + // Then the refresh is single-flight on this host only: two hosts can + // refresh at the same time, and the last save wins. + await store.set({ scope: seen.scope, id, credential: next }) + return next + } const access: CredentialsAccess = { get, require: async (id, options) => { @@ -131,16 +247,36 @@ export function credentialsFor( }, set: async ({ id, credential }) => { keep(credential) - await store.set({ scope: scopeNow(), id, credential }) + await store.set({ scope, id, credential }) await hooks?.onSet?.(id) }, - delete: (id) => store.delete({ scope: scopeNow(), id }), + refresh: async (id, update, options) => { + options?.signal?.throwIfAborted() + const seen = await readCredentialEntry(store, scope, id) + if (!seen) return null + // Keyed by the scope where the credential is, so the users of one + // tenant credential share one refresh too. + const key = refreshKey(seen.scope, id) + let flight = flights.running.get(key) + if (!flight) { + // The calls that join use the first call's `update`. It runs with + // the host signal, so an abort of one caller stops no other caller. + flight = runRefresh(id, seen, update).finally(() => { + flights.running.delete(key) + }) + flights.running.set(key, flight) + } + return keep(await untilAborted(flight, options?.signal)) + }, + delete: (id) => store.delete({ scope, id }), list: async () => { - const user = scopeNow() - const tenant = tenantScopeOf(user) - const own = await store.list(user) + // A secret answer that the harness keeps is not the user's own. + const saved = async (at: CredentialScope) => + (await store.list(at)).filter((item) => item.type !== 'secret_answer') + const tenant = tenantScopeOf(scope) + const own = await saved(scope) if (!tenant) return own - const shared = await store.list(tenant) + const shared = await saved(tenant) return [ ...own, ...shared.filter((item) => !own.some((o) => o.id === item.id)), @@ -151,22 +287,24 @@ export function credentialsFor( } /** - * The provider keys of the session's principal. A key is the `api_key` + * The provider keys of the user of `credentials`. A key is the `api_key` * credential saved under the provider id (`/connect ` saves it), else - * the first `provider.env` name that is set. A missing key calls `onMissing` - * and throws {@link AuthRequiredError}, so the user sees `/connect `. - * A key read from env goes in `secrets`, like a saved credential. + * the first `provider.env` name that is set. With `envFallback: false`, env + * vars are not read. A missing key calls `onMissing` and throws + * {@link AuthRequiredError}, so the user sees `/connect `. A key read + * from env goes in `secrets`, like a saved credential. */ export function providerKeysFor( credentials: CredentialsAccess, onMissing: (error: AuthRequiredError) => void, - secrets?: Set, + { secrets, envFallback }: { secrets?: Redactor; envFallback: boolean }, ) { const providerIdOf = (provider: ByokProvider | ProviderId) => typeof provider === 'string' ? provider : provider.id const get = async (provider: ByokProvider | ProviderId) => { const saved = await credentials.get(providerIdOf(provider)) if (saved?.type === 'api_key') return saved.value + if (!envFallback) return null const fromEnv = envKeyEntry(provider)?.value if (fromEnv) secrets?.add(fromEnv) return fromEnv ?? null @@ -190,25 +328,134 @@ export function providerKeysFor( } /** - * Replace each of `secrets` in `text` with `[redacted]`: the raw value, its - * JSON-escaped form, and its URL-encoded form. A secret shorter than 6 - * characters is left alone, so a tiny value does not redact common words. + * @internal The secret values that one session handed out, and the scrub of + * them. The session sends each log record, feed event, and model call + * through it. See {@link createRedactor}. + */ +export type Redactor = ReturnType + +/** + * @internal A {@link Redactor} with no secret yet. + * + * `redact` never changes its input. It returns the same value when no + * secret is in it, so a check of identity downstream stays cheap. */ -export function scrubSecrets( - text: string, - secrets: ReadonlyArray, -): string { - const forms = new Set() - for (const secret of secrets) { - if (secret.length < 6) continue - forms.add(secret) - forms.add(JSON.stringify(secret).slice(1, -1)) - forms.add(encodeURIComponent(secret)) +export function createRedactor() { + const values: Array = [] + /** Each form that `scrubSecrets` replaces: raw, JSON-escaped, URL-encoded. */ + const forms: Array = [] + + /** + * Scrub `value` from now on: a string, or each string in an array or a + * plain object (a secret answer with fields). A string shorter than 6 + * characters is left. + */ + const add = (value: unknown): void => { + if (typeof value === 'object' && value !== null) { + for (const item of Object.values(value)) add(item) + return + } + if (typeof value !== 'string' || value.length < 6) return + if (values.includes(value)) return + values.push(value) + for (const form of [ + value, + JSON.stringify(value).slice(1, -1), + encodeURIComponent(value), + ]) { + if (!forms.includes(form)) forms.push(form) + } + } + + /** `text` without the secrets. The same string when it has none. */ + const scrub = (text: string): string => { + // A quick check first, so a text without a secret makes no copy. + for (const form of forms) { + if (text.includes(form)) return scrubSecrets(text, values) + } + return text + } + + /** `items` with their secrets scrubbed, or `undefined` when none changed. */ + const walkAll = (items: Array) => { + let isChanged = false + const next = items.map((item) => { + const walked = walk(item) + if (walked !== item) isChanged = true + return walked + }) + return isChanged ? next : undefined } - // The longest first, so a secret inside another does not leave a part. - let scrubbed = text - for (const form of [...forms].sort((a, b) => b.length - a.length)) { - scrubbed = scrubbed.split(form).join('[redacted]') + + /** `value` with its secrets scrubbed. A changed container is a copy. */ + const walk = (value: unknown): unknown => { + if (typeof value === 'string') return scrub(value) + if (typeof value !== 'object' || value === null) return value + if (Array.isArray(value)) return walkAll(value) ?? value + // Each entry is a fresh `[key, value]` array, so an entry that did not + // change comes back as it is. + if (value instanceof Map) { + const entries = walkAll([...value]) + return entries ? new Map(entries as Array<[unknown, unknown]>) : value + } + if (value instanceof Set) { + const items = walkAll([...value]) + return items ? new Set(items) : value + } + // ponytail: other class instances (`Uint8Array`, `Date`, an app class) + // stay as they are, since a copy could break them. Walk them if a + // plugin keeps a secret in one. + const proto: unknown = Object.getPrototypeOf(value) + if (proto !== Object.prototype && proto !== null) return value + const record = value as Record + let copy: Record | undefined + for (const key of Object.keys(record)) { + const item = record[key] + const next = walk(item) + if (next === item) continue + copy ??= Object.assign(Object.create(proto) as object, record) + copy[key] = next + } + return copy ?? value + } + + return { + /** No secret to scrub yet. Then `redact` and `scrub` do no work. */ + get isEmpty(): boolean { + return forms.length === 0 + }, + add, + scrub, + /** + * `value` with each string in it scrubbed, in arrays, plain objects, + * `Map`s, and `Set`s at any depth. A changed container is a copy. Other + * values (numbers, class instances like `Uint8Array`) stay as they are. + */ + redact: (value: T): T => + forms.length === 0 ? value : (walk(value) as T), + } +} + +/** + * @internal The model gets the context without the secrets of `redactor`. + * Put it after `mediaMiddleware`, so no later middleware sets the messages + * again. The transcript itself is not changed here: the log view scrubs it. + */ +export function redactMiddleware(redactor: Redactor): ChatMiddleware { + return { + name: 'harness:redact', + onConfig(_ctx, config) { + if (redactor.isEmpty) return undefined + const messages = config.providerMessages ?? config.messages + const providerMessages = redactor.redact(messages) + const systemPrompts = redactor.redact(config.systemPrompts) + if ( + providerMessages === messages && + systemPrompts === config.systemPrompts + ) { + return undefined + } + return { providerMessages, systemPrompts } + }, } - return scrubbed } diff --git a/packages/ai-harness/src/boundary-waits.ts b/packages/ai-harness/src/boundary-waits.ts new file mode 100644 index 0000000000..a48d8a6d54 --- /dev/null +++ b/packages/ai-harness/src/boundary-waits.ts @@ -0,0 +1,167 @@ +import { defineInterrupt, validateJsonSchema } from '@tanstack/ai' +import { isRecord } from './utils' +import type { + AnyChatMiddleware, + StandardJSONSchemaV1, + StandardSchemaV1, +} from '@tanstack/ai' + +/** + * A plain JSON Schema as a Standard Schema with a JSON Schema export, so a + * core interrupt can name it as a schema. `validateJsonSchema` checks values. + */ +function standardJsonSchema( + json: Record, +): StandardSchemaV1 & StandardJSONSchemaV1 { + return { + '~standard': { + version: 1, + vendor: 'tanstack-harness', + validate: (value) => validateJsonSchema(json, value), + jsonSchema: { input: () => json, output: () => json }, + }, + } +} + +/** The id of the interrupt that asks the user to approve one tool call. */ +export const APPROVAL_INTERRUPT_ID = 'tanstack/harness:approval' + +/** The answer to an approval: once, always, or reject with a message. */ +export interface ApprovalAnswer { + answer: 'once' | 'always' | 'reject' + /** With `reject`: what the model should do instead. */ + message?: string +} + +/** + * The answer shape of an approval, for hosts that render a form. A yes or + * no (`true` or `false`) works too, as for every approval, so a client + * answers each approval the same way. + */ +export const APPROVAL_ANSWER_SCHEMA: Record = { + anyOf: [ + { + type: 'boolean', + description: 'true: allow this call. false: refuse it.', + }, + { type: 'string', enum: ['once', 'always', 'reject'] }, + { + type: 'object', + properties: { + answer: { + type: 'string', + enum: ['once', 'always', 'reject'], + description: + 'once: allow this call. always: allow it from now on in this project. reject: refuse it.', + }, + message: { + type: 'string', + description: 'With reject: what the model should do instead.', + }, + }, + required: ['answer'], + }, + ], +} + +/** + * @internal The interrupt that a turn stops with, before its tools run, for + * each tool call that needs the user's approval (see `permissions()`). Its + * `key` is the tool call id. Every chat call of a turn registers it. + */ +export const approvalInterrupt = defineInterrupt({ + id: APPROVAL_INTERRUPT_ID, + payloadSchema: standardJsonSchema({ + type: 'object', + properties: { + toolCallId: { type: 'string' }, + tool: { type: 'string' }, + }, + required: ['toolCallId', 'tool'], + }), + responseSchema: standardJsonSchema(APPROVAL_ANSWER_SCHEMA), +}) + +/** The id of the interrupt that asks one question of the `question` tool. */ +export const QUESTION_INTERRUPT_ID = 'tanstack/harness:question' + +/** + * @internal The interrupt that a turn stops with, before its tools run, for + * each question of a `question` tool call. Its `key` is + * `${toolCallId}:${index}`. The payload has the answer shape of the + * question, for hosts that render a form. The answer is one label, a list + * of labels, or the user's own text, so the tool call gets it as its result. + */ +export const questionInterrupt = defineInterrupt({ + id: QUESTION_INTERRUPT_ID, + payloadSchema: standardJsonSchema({ + type: 'object', + properties: { + toolCallId: { type: 'string' }, + index: { type: 'integer' }, + schema: { type: 'object' }, + }, + required: ['toolCallId', 'index', 'schema'], + }), + responseSchema: standardJsonSchema({ + anyOf: [ + { type: 'string' }, + { type: 'number' }, + { type: 'array', items: { type: ['string', 'number'] } }, + ], + }), +}) + +/** The interrupts that every chat call of a turn registers. */ +export const BOUNDARY_INTERRUPTS = [approvalInterrupt, questionInterrupt] + +const BOUNDARY_IDS = new Set( + BOUNDARY_INTERRUPTS.map((definition) => definition.id), +) + +/** + * `middleware` that does not stop the run for an approval or a question. + * For a tool-mode child of a turn with `durability.agentToolWaits: false`: + * it does not ask, and a call that needs an approval is refused. Other + * interrupts stay. + */ +export function withoutBoundaryWaits( + middleware: ReadonlyArray, +): Array { + return middleware.map((entry) => { + const boundary = entry.onInterruptBoundary + if (!boundary) return entry + return { + ...entry, + onInterruptBoundary: async (ctx) => { + const result = await boundary(ctx) + const interrupts = result?.interrupts.filter( + (request) => !BOUNDARY_IDS.has(request.definition.id), + ) + return interrupts?.length ? { interrupts } : undefined + }, + } satisfies AnyChatMiddleware + }) +} + +/** + * The user's reply to an approval: `{ answer, message? }`, the answer as + * text, or a yes or no (`true` is `once`). Anything that is not `once` or + * `always` rejects. + */ +export function readApproval(value: unknown): { + choice: ApprovalAnswer['answer'] + message: string | undefined +} { + if (value === true) return { choice: 'once', message: undefined } + const raw = isRecord(value) ? value.answer : value + const choice = typeof raw === 'string' ? raw.trim().toLowerCase() : '' + const text = + isRecord(value) && typeof value.message === 'string' + ? value.message.trim() + : '' + return { + choice: choice === 'once' || choice === 'always' ? choice : 'reject', + message: text === '' ? undefined : text, + } +} diff --git a/packages/ai-harness/src/build.ts b/packages/ai-harness/src/build.ts index 7210fb7453..41ee2decc8 100644 --- a/packages/ai-harness/src/build.ts +++ b/packages/ai-harness/src/build.ts @@ -245,6 +245,8 @@ const FORWARDED = new Set([ interface Worker { send: (frame: object) => void frames: (listener: (frame: HostFrame) => void) => () => void + /** True for the `requestId` of a frame that was sent to this worker. */ + sent: (requestId: string) => boolean kill: () => void } @@ -257,10 +259,16 @@ function startWorker( const child = spawn(node, [bundle], { stdio: ['pipe', 'pipe', 'inherit'] }) const listeners = new Set<(frame: HostFrame) => void>() // A worker that fails to start or stops fails the streams that wait on it. + const requestIds = new Set() const fail = (message: string) => { onExit() for (const listener of listeners) - listener({ type: 'harness.error', message }) + listener({ + type: 'harness.error', + message, + code: 'internal', + requestId: 'worker', + }) } child.on('error', (error) => fail(`The worker failed: ${error.message}`)) child.on('exit', (code, signal) => @@ -275,8 +283,11 @@ function startWorker( // A line that is not a frame (for example a stray log) is skipped. } }) - const send = (frame: object) => + const send = (frame: object) => { + if ('requestId' in frame && typeof frame.requestId === 'string') + requestIds.add(frame.requestId) child.stdin.write(`${JSON.stringify(frame)}\n`) + } send({ type: 'harness.subscribe', threadId }) return { send, @@ -284,6 +295,7 @@ function startWorker( listeners.add(listener) return () => listeners.delete(listener) }, + sent: (requestId) => requestIds.has(requestId), kill: () => { child.stdin.end() child.kill() @@ -394,7 +406,15 @@ export async function artifactText( } const frame = queue.shift() if (!frame) continue - if (frame.type === 'harness.error') throw new Error(frame.message) + if (frame.type === 'harness.error') { + // The error of another input of this worker, for example the + // cancel of an older turn, is not about this turn. + if (frame.requestId !== requestId && worker.sent(frame.requestId)) + continue + throw new Error(frame.message, { + cause: { code: frame.code, requestId: frame.requestId }, + }) + } if ( frame.type === 'harness.receipt' && frame.requestId === requestId diff --git a/packages/ai-harness/src/client.ts b/packages/ai-harness/src/client.ts index f260b1b218..aa4ddc4c75 100644 --- a/packages/ai-harness/src/client.ts +++ b/packages/ai-harness/src/client.ts @@ -1,12 +1,14 @@ // Browser-safe: this module has type-only imports from the rest of the -// package, plus `media-ref`, which has type-only imports itself. So no server -// code (adapters, stores, secrets) reaches a client. +// package, plus `media-ref` and `utils`, which have no runtime imports +// themselves. So no server code (adapters, stores, secrets) reaches a client. import { isMediaRecord } from './media-ref' -import type { ModelMessage, RunAgentResumeItem } from '@tanstack/ai' +import { isRecord } from './utils' +import type { ModelMessage } from '@tanstack/ai' import type { LogRecord, SessionIndexEntry, SessionIndexPage, + SessionMember, } from '@tanstack/ai-persistence' import type { AgentInputOf } from './agents' import type { @@ -17,17 +19,23 @@ import type { SessionMetadataOf, } from './define' import type { HostEvent } from './host' +import type { HarnessErrorCode } from './protocol' import type { SessionDescription, SessionSnapshot } from './session/index' import type { SessionEntryChange } from './session-entry' import type { BusyPolicy, + CancelOptions, + CommandOptions, + ConfigureOptions, Cursor, ForkPoint, HarnessInput, MediaRecord, + MediaUpload, Receipt, + ResetOptions, + ResolveOptions, SessionEvent, - ThreadSettingsChange, UserInput, WaitingInput, } from './types' @@ -40,6 +48,7 @@ export { mediaPart, } from './media-ref' export type { ForkPoint, MediaKind, MediaRecord } from './types' +export type { HarnessErrorCode } from './protocol' export type { HostEvent, HostSessionDeletedEvent, @@ -47,6 +56,36 @@ export type { HostStatusEvent, } from './host' +/** + * A `harness.error` frame that ended a stream of the client: `code` says + * what failed. `events()` and `hostEvents()` throw it. A `slow_consumer` + * error is not thrown: they connect again. + */ +export class HarnessProtocolError extends Error { + readonly code: HarnessErrorCode + constructor(code: HarnessErrorCode, message: string) { + super(message) + this.name = 'HarnessProtocolError' + this.code = code + } +} + +/** + * The error of a `harness.error` frame. Undefined for another frame and + * for `slow_consumer`: the stream ends, and the client connects again. + */ +function protocolErrorOf(frame: unknown): HarnessProtocolError | undefined { + if (!isRecord(frame) || frame.type !== 'harness.error') return undefined + if (frame.code === 'slow_consumer') return undefined + const code = + typeof frame.code === 'string' + ? (frame.code as HarnessErrorCode) + : 'internal' + const message = + typeof frame.message === 'string' ? frame.message : 'Harness stream failed' + return new HarnessProtocolError(code, message) +} + export interface HarnessClientOptions { /** The base URL of `createHarnessHandler`, for example `/api/harness`. */ url: string @@ -54,8 +93,20 @@ export interface HarnessClientOptions { /** Extra headers, for example `Authorization`. */ headers?: Record | (() => Record) fetch?: typeof fetch - /** Wait before a reconnect of `events()`. Default 1000 ms. */ + /** + * The first wait before a reconnect of `events()`. Default 1000 ms. Each + * failed try after it doubles the wait, up to 30 s, with a random part. + * A read from the server sets the wait back to this value. + */ reconnectDelayMs?: number + /** + * A connection of `events()` that gets no bytes for this long is dead, + * for example after a laptop sleeps or a NAT drops the mapping. The + * client closes it and connects again from its last cursor. Default + * 60 000 ms. Keep it larger than the `stream.heartbeatMs` of the server + * (default 20 000 ms), or the client reconnects in a loop. + */ + idleTimeoutMs?: number } type StartArgs = @@ -71,20 +122,36 @@ export type ClientAgentHandles = { } /** The options of `HarnessClient.send`. */ -export type ClientSendOptions = { inputId?: string } & ( +export type ClientSendOptions = { + inputId?: string + /** + * A W3C `traceparent` of the caller, sent as the `traceparent` header. + * The span of the turn on the server links it. + */ + traceparent?: string +} & ( | { to?: undefined mode?: 'prompt' busy?: BusyPolicy ephemeral?: ReadonlyArray + /** `'operator'`: an operator reply. See `SendOptions.as`. */ + as?: 'operator' } | { to?: undefined mode: 'steer' | 'followUp' busy?: never ephemeral?: never + as?: never + } + | { + to: string + mode?: 'steer' | 'followUp' + busy?: never + ephemeral?: never + as?: never } - | { to: string; mode?: 'steer' | 'followUp'; busy?: never; ephemeral?: never } ) export interface HarnessClient { @@ -95,72 +162,152 @@ export interface HarnessClient { * run's agent in `expose.agents`. `inputId` is an id you choose: a retry * with the same id and message gets the first receipt and does not run * again. `ephemeral` are messages for the model calls of a prompt only. + * `as: 'operator'` is the reply of an operator while a handoff to a + * person is open: no model runs. The sender needs the `operator` action. */ send: (message: UserInput, options?: ClientSendOptions) => Promise /** * Start a turn from the stored transcript, with no new message. See - * `session.continue`. + * `session.continueWithoutMessage`. */ - continue: (options?: { + continueWithoutMessage: (options?: { inputId?: string ephemeral?: ReadonlyArray }) => Promise - resolve: (resume: Array) => Promise - cancel: (operationId?: string) => Promise + /** + * Answer the waits of `snapshot().pendingWaits` that have an + * `interrupt`, as `session.resolve` does. The waits of the last turn (the + * approvals, the `question` tool calls, the client tools, and the + * sign-ins) go in one resume. A wait of a background agent run takes a + * resolve of its own. A wait with no interrupt takes `answer`. + * + * @example + * ```ts + * await client.resolve({ resume: [{ interruptId, status: 'resolved' }] }) + * ``` + */ + resolve: (options: ResolveOptions) => Promise + /** + * Stop the operation `operationId`, or the running turn without it, as + * `session.cancel` does. + * + * @example + * ```ts + * await client.cancel({ operationId }) + * ``` + */ + cancel: (options?: CancelOptions) => Promise /** * Cancel an input that waits (see `snapshot().waitingInputs`). An input * that started is refused with `not_waiting`: use `cancel`. + * + * @example + * ```ts + * await client.cancelInput({ inputId }) + * ``` */ - cancelInput: (inputId: string) => Promise + cancelInput: (options: { inputId: string }) => Promise /** * Move an input that waits: `'steer'` joins the running turn, `'queue'` * runs it as its own turn later. */ - setDelivery: (change: { + changeInputDelivery: (change: { inputId: string delivery: WaitingInput['delivery'] }) => Promise agents: ClientAgentHandles - /** Answer a question from a command or a plugin. */ + /** + * Answer a wait of `snapshot().pendingWaits` that no turn holds, as + * `session.answer` does: a `step.ask` wait of a command, a durable tool, + * or an agent run, an MCP question, or a `human` wait. The waits of the + * stopped turn take `resolve`. + */ answer: (answer: { questionId: string; value: unknown }) => Promise - /** Run a plugin command. Its result arrives as a `harness.command.result` event. */ - command: (name: string, input?: unknown) => Promise /** - * Run a harness tool with no model turn, as `session.tool` does. The + * Run the plugin command `name`. Its result arrives as a + * `harness.command.result` event. + * + * @example + * ```ts + * await client.runCommand({ name: 'mode', input: 'plan' }) + * ``` + */ + runCommand: (options: CommandOptions) => Promise + /** + * Run a harness tool with no model turn, as `session.runTool` does. The * harness must list it in `expose.tools`. `args` is typed from the * tool's input schema. + * + * @example + * ```ts + * await client.runTool({ name: 'sync_inbox', args: { folder: 'INBOX' } }) + * ``` */ - tool: ['name']>( - name: TName, - args?: HarnessToolInput, - options?: { inputId?: string; meta?: Record }, - ) => Promise + runTool: ['name']>(options: { + name: TName + args?: HarnessToolInput + meta?: Record + inputId?: string + }) => Promise /** - * Add host records to the session log, as `session.append` does. The - * harness must list each record type in `expose.records`. + * Add host records to the session log, as `session.appendRecords` does. The + * harness must list each record type in `expose.records`. With `inputId`, + * a second append with the same id is a duplicate and writes nothing. + * + * @example + * ```ts + * await client.appendRecords({ records: [{ type: 'app.signal', text: 'Saved.' }] }) + * ``` */ - append: (records: ReadonlyArray) => Promise + appendRecords: (options: { + records: ReadonlyArray + inputId?: string + }) => Promise /** Change a session setting. */ - setConfig: (change: { key: string; value: unknown }) => Promise - /** Change the stored settings of the thread. See `session.configure`. */ - configure: (settings: ThreadSettingsChange) => Promise + setPluginSetting: (change: { + key: string + value: unknown + }) => Promise + /** + * Change the stored settings of the thread. See `session.updateThreadSettings`. + * + * @example + * ```ts + * await client.updateThreadSettings({ settings: { model: 'strong' } }) + * ``` + */ + updateThreadSettings: (options: ConfigureOptions) => Promise /** * Start a fresh model context from the next turn. The model sees `note` * first, when you give one. The transcript keeps every message. + * + * @example + * ```ts + * await client.reset({ note: 'The user wants a new plan.' }) + * ``` */ - reset: (note?: string, options?: { inputId?: string }) => Promise + reset: (options?: ResetOptions) => Promise /** * Go back to the message `messageId`, as `session.revert` does. The * harness must list `'undo'` in `expose.commands`. + * + * @example + * ```ts + * await client.revert({ messageId }) + * ``` */ - revert: (messageId: string) => Promise + revert: (options: { messageId: string }) => Promise /** End the revert that stands, as `session.unrevert` does. */ unrevert: () => Promise /** - * The session events from `from` (exclusive). Reconnects after a network + * The session events from `from` (exclusive). Without `from`, the events + * start at the cursor of the snapshot: only new events come. Pass + * `from: '0'` to read the events from the start. Reconnects after a network * error and resumes from the last cursor. Ends when `signal` aborts. * `onConnection` reports `'open'` for each connection and `'reconnecting'` - * before each new try. + * before each new try. When the log dropped the events after `from`, the + * first event is a `harness.gap` (`HARNESS_EVENTS.gap`): load the snapshot + * and the transcript again. A session view does this itself. */ events: (options?: { from?: Cursor @@ -179,10 +326,18 @@ export interface HarnessClient { transcript: () => Promise> /** * The saved messages of a child agent run of the thread, by its - * `subagentRunId`. Empty for an unknown id. A child with no messages has - * one message with no content, with its card. See `session.agentTranscript`. + * `subagentRunId` or by its `runId`. Empty for an unknown id. A child with + * no messages has one message with no content, with its card. See + * `session.agentTranscript`. + * + * @example + * ```ts + * const messages = await client.agentTranscript({ subagentRunId }) + * ``` */ - agentTranscript: (subagentRunId: string) => Promise> + agentTranscript: ( + run: { subagentRunId: string } | { runId: string }, + ) => Promise> /** * The commands, settings, and tools of the session. Only the commands in * `expose.commands` and the config keys in `expose.config`. @@ -192,18 +347,52 @@ export interface HarnessClient { * Store a file in the media store of the thread. Send the record in a * prompt with `mediaPart(record)`. Throws when the handler refuses the * file, for example `413` for a file over the size limit. + * + * @example + * ```ts + * const record = await client.putMedia({ body: file, mimeType: file.type, name: file.name }) + * ``` */ - upload: ( - body: Blob | ArrayBuffer | Uint8Array, - info: { name: string; mimeType: string }, + putMedia: ( + file: MediaUpload, ) => Promise /** * A signed URL for a media file, for ``, `