Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
69 changes: 51 additions & 18 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,25 @@
# @tangle-network/agent-runtime

The engine Tangle's AI agents run on. It runs an agent a **chat turn**, a **one-shot task**, or a **team of agents** working toward a goal records every run, and uses those records to **measure and improve** agents against real pass/fail checks.
The engine Tangle's AI agents run on. It runs an agent as a **chat turn**, a **one-shot task**, or a **team of agents** working toward a goal, records every run, and uses those records to **measure and improve** agents against real pass/fail checks.

One loop, used three ways. Domain behavior (models, tools, knowledge) plugs in as adapters; the scoring statistics and the ship decision come from [`@tangle-network/agent-eval`](https://www.npmjs.com/package/@tangle-network/agent-eval); sandboxed execution from [`@tangle-network/sandbox`](https://www.npmjs.com/package/@tangle-network/sandbox).
One loop, used four common ways. Domain behavior (models, tools, knowledge) plugs in as adapters; the scoring statistics and the ship decision come from [`@tangle-network/agent-eval`](https://www.npmjs.com/package/@tangle-network/agent-eval); sandboxed execution from [`@tangle-network/sandbox`](https://www.npmjs.com/package/@tangle-network/sandbox).

```bash
pnpm add @tangle-network/agent-runtime @tangle-network/agent-eval @tangle-network/sandbox
```

**See it run in 30 seconds** (offline, no keys) — the one move everything else builds on, a driver reading a worker's output and composing the next step from it:
## Contents

- [What you do with it](#what-you-do-with-it)
- [Run a chat turn](#run-a-chat-turn)
- [Supervise a team of agents](#supervise-a-team-of-agents)
- [Improve an agent](#improve-an-agent)
- [Improve a knowledge base](#improve-a-knowledge-base)
- [How it works](#how-it-works-the-short-version)
- [Examples](#examples)
- [Where to go next](#where-to-go-next)

**See it run in 30 seconds** (offline, no keys): the one move everything else builds on, a driver reading a worker's output and composing the next step from it:

```bash
pnpm tsx examples/driver-loop/driver-loop.ts
Expand All @@ -18,9 +29,10 @@ pnpm tsx examples/driver-loop/driver-loop.ts

| You want to… | Call |
|---|---|
| Run a **chat turn** — what every product agent does in production | `handleChatTurn(...)` |
| Run a **chat turn** for a production product agent | `handleChatTurn(...)` |
| Have one agent **supervise a team of agents** toward a goal | `supervise(profile, task, opts)` |
| **Improve** an agent and prove the gain on fresh tasks | `improve(profile, findings, opts)` |
| **Improve** a knowledge base with agents, checks, and safe promotion | `runKnowledgeImprovementJob(...)` |

### Run a chat turn

Expand Down Expand Up @@ -56,7 +68,7 @@ const result = await supervise(

### Improve an agent

`improve` optimizes one part of an agent (its prompt, skills, or code) and **only ships a change if it beats the current agent on tasks it never practiced on** — so registering an agent for self-improvement can never make it worse.
`improve` optimizes one part of an agent (its prompt, skills, or code) and **only ships a change if it beats the current agent on tasks it never practiced on**. Registering an agent for self-improvement cannot ship a worse candidate unless the caller supplies a bad measurement.

```ts
import { improve } from '@tangle-network/agent-runtime'
Expand All @@ -68,37 +80,58 @@ const { profile, shipped, lift } = await improve(baseProfile, findings, {
})
```

### Improve a knowledge base

`runKnowledgeImprovementJob` is the runtime-owned front door for KB, wiki, memory-backed, and RAG improvement jobs. It creates a candidate copy, runs supervised agents against it, checks readiness through `@tangle-network/agent-knowledge`, measures spend and timing, and promotes only when the candidate passes.

```ts
import { runKnowledgeImprovementJob } from '@tangle-network/agent-runtime/knowledge'

const result = await runKnowledgeImprovementJob({
root: './kb',
goal: 'Improve support refund-policy knowledge',
readinessSpecs,
budget: { maxIterations: 8, maxTokens: 120_000, maxUsd: 10 },
backend,
})

console.log(result.promoted, result.measurement.supervisedSpent)
```

Use it when the product needs one knob for "make this knowledge base better" instead of wiring `improveKnowledgeBase`, a runtime supervisor, candidate workspaces, readiness checks, and promotion tracking by hand.

## How it works (the short version)

- **One agent, run two ways.** The same agent runs at "do the task" speed and at "get better at the task" speed. "Driver", "worker", and "coordinator" aren't separate types — they're roles one agent plays.
- **Everything is measured.** Every run is a trace: tokens, dollars, time, and a pass/fail score from a real check. "Better" is a number with a denominator, not a vibe and "equally good but cheaper" is a result you can prove.
- **Improvement is gated.** A change ships only after it beats the current agent on fresh tasks no tuning step ever saw, with a statistical test not a single lucky run.
- **The grader is honest.** Whatever gives feedback never sees the answer key, and scores are recomputed from the attempts actually run — an agent can't fabricate its own win.
- **One agent, run two ways.** The same agent runs at "do the task" speed and at "get better at the task" speed. "Driver", "worker", and "coordinator" are roles one agent plays, not separate types.
- **Everything is measured.** Every run is a trace: tokens, dollars, time, and a pass/fail score from a real check. "Better" is a number with a denominator, not a vibe, and "equally good but cheaper" is a result you can prove.
- **Improvement is gated.** A change ships only after it beats the current agent on fresh tasks no tuning step ever saw, with a statistical test, not a single lucky run.
- **The grader is honest.** Whatever gives feedback never sees the answer key, and scores are recomputed from the attempts actually run. An agent cannot fabricate its own win.

## Examples

Runnable, grouped by what they show — copy the one nearest your task:
Runnable, grouped by what they show. Copy the one nearest your task:

| Do this | Example |
|---|---|
| Run a product chat turn | [`chat-handler`](./examples/chat-handler) |
| Drive a team of agents to a goal | [`supervise`](./examples/supervise) · [`recursive-supervisor`](./examples/recursive-supervisor) |
| Benchmark strategies on your own domain | [`coding-benchmark`](./examples/coding-benchmark) |
| Benchmark **harnesses × models** over a real task suite (the real WebCode dataset) | [`webcode-matrix`](./examples/webcode-matrix) |
| Render a **multi-profile leaderboard** ranked board + score matrix + SVG/HTML charts, any domain | `leaderboard(records)` → `renderLeaderboardMarkdown` / `Svg` / `Html` |
| Render a **multi-profile leaderboard** with ranked board, score matrix, and SVG/HTML charts | `leaderboard(records)` → `renderLeaderboardMarkdown` / `Svg` / `Html` |
| Trace + bill + effort-gate the WebCode benchmark (the Intelligence SDK) | [`intelligence-webcode`](./examples/intelligence-webcode) |
| Self-improve an agent, gated on a held-out set | [`improve`](./examples/improve) · [`self-improving-coder`](./examples/self-improving-coder) |
| Improve a KB, wiki, or RAG corpus with runtime agents | [`docs/canonical-api.md`](./docs/canonical-api.md) |
| Study coordination vs raw compute | [`ablation-suite`](./examples/ablation-suite) |

All 28 live in [`examples/`](./examples).
All 29 live in [`examples/`](./examples).

## Where to go next

- New here? [`docs/concepts.md`](./docs/concepts.md) the mental model in plain terms.
- [`docs/canonical-api.md`](./docs/canonical-api.md) find the primitive: "I want to ___ → use ___".
- [`docs/api/primitive-catalog.md`](./docs/api/primitive-catalog.md) every export in one generated, never-stale list with its import path. Check it before building anything new.
- Import subpaths: the root export is the product surface (`handleChatTurn`, `improve`); deeper capabilities ship as subpaths`/loops` (multi-agent + the loop kernel), `/mcp` (tool servers), `/intelligence` (observability drop-in), `/lifecycle`, `/agent`, `/profiles`, `/platform`, `/analyst-loop`, `/environment-provider`.
- [`docs/architecture.md`](./docs/architecture.md) the design, end to end.
- [`bench/HARNESS.md`](./bench/HARNESS.md) the experiment harness and how to run a benchmark.
- New here? [`docs/concepts.md`](./docs/concepts.md), the mental model in plain terms.
- [`docs/canonical-api.md`](./docs/canonical-api.md), find the primitive: "I want to ___ → use ___".
- [`docs/api/primitive-catalog.md`](./docs/api/primitive-catalog.md), every export in one generated, never-stale list with its import path. Check it before building anything new.
- Import subpaths: the root export is the product surface (`handleChatTurn`, `improve`); deeper capabilities ship as subpaths: `/loops` (multi-agent + the loop kernel), `/knowledge` (KB improvement), `/mcp` (tool servers), `/intelligence` (observability drop-in), `/lifecycle`, `/agent`, `/profiles`, `/platform`, `/analyst-loop`, `/environment-provider`.
- [`docs/architecture.md`](./docs/architecture.md), the design, end to end.
- [`bench/HARNESS.md`](./bench/HARNESS.md), the experiment harness and how to run a benchmark.

**Contributing:** `pnpm i && pnpm test` gets you running; the full local gate is the [`package.json`](./package.json) scripts (`lint`, `typecheck`, `docs:check`).
2 changes: 1 addition & 1 deletion docs/canonical-api.md
Original file line number Diff line number Diff line change
Expand Up @@ -111,7 +111,7 @@ A general "loop" primitive is the single most common modelling error in this rep
| Stand up a vertical agent in the eval loop | `defineAgent(manifest)` + `createSurfaceImprovementAdapter` — `/agent` | a per-vertical manifest parser, surface-validator, or bespoke `ImprovementAdapter` |
| Turn intelligence/observation OFF (prove inference-only billing) | `withTangleIntelligence(agent, { effort: 'off' })` — `/intelligence` | a custom trace-wrapper or hand-rolled effort/tier config |
| Fold **certified prompt additions into a system prompt you assemble yourself** (product chat routes) | `createCertifiedPromptSource({ target })` → `source.compose(base)` — `/intelligence` (cached, coalesced, fail-closed; `withCertifiedDelivery` rides the same source) | a module-scope cache + refresh-window + keep-last-known loop around `pullCertified` in product wiring |
| Improve a KB with runtime agents, candidate workspaces, readiness checks, and measured supervised spend | `runKnowledgeImprovementJob(options)` `/knowledge` | hand-wiring `improveKnowledgeBase` + a supervised updater + a readiness callback in every product |
| Improve a KB with runtime agents, candidate workspaces, readiness checks, and measured supervised spend | `runKnowledgeImprovementJob(options)` from `/knowledge` | hand-wiring `improveKnowledgeBase` + a supervised updater + a readiness callback in every product |

For the full export inventory (every primitive, its import path, its summary — generated, never stale), see `docs/api/primitive-catalog.md`; for per-symbol signatures, the per-module `docs/api/` pages. For the recursive atom (recursion · isolated-or-collaborative artifact · conserved budget · analysts) and the two-timescale architecture, see `docs/architecture.md`. For the genome→run→optimize→ship spine in depth, `docs/concepts.md` + `docs/learning-flywheel.md`. For the Intelligence SDK (Observe + the provable-OFF billing boundary), `docs/intelligence-sdk.md`.

Expand Down
Loading