Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 24 additions & 7 deletions docs/api/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -2813,13 +2813,30 @@ The byte-producing seam — the ONE thing that differs between the cheap

Defined in: [improvement/improvement-driver.ts:37](https://github.com/tangle-network/agent-runtime/blob/main/src/improvement/improvement-driver.ts#L37)

##### proposesWithoutFindings?

> `optional` **proposesWithoutFindings?**: `boolean`

Defined in: [improvement/improvement-driver.ts:48](https://github.com/tangle-network/agent-runtime/blob/main/src/improvement/improvement-driver.ts#L48)

Whether this generator can produce a candidate from an EMPTY findings set
and no phase-2 report — i.e. it draws its change signal from the repo and
the raw-trace filesystem context on disk, not only from pre-summarized
findings. An agentic coder (`agenticGenerator`) sets this: the seed repo +
raw traces ARE the signal, so it must still run the full `populationSize`
when the distiller yielded nothing (this is the meta-harness contract — the
agent diagnoses from the raw traces itself). A patch-applier
(`reflectiveGenerator`) leaves it unset — with no findings there is no
patch to draft, so the driver short-circuits rather than spin up worktrees
for a guaranteed no-op. Default `false`.

#### Methods

##### generate()

> **generate**(`args`): `Promise`\<\{ `applied`: `boolean`; `summary`: `string`; \}\>

Defined in: [improvement/improvement-driver.ts:38](https://github.com/tangle-network/agent-runtime/blob/main/src/improvement/improvement-driver.ts#L38)
Defined in: [improvement/improvement-driver.ts:49](https://github.com/tangle-network/agent-runtime/blob/main/src/improvement/improvement-driver.ts#L49)

###### Parameters

Expand Down Expand Up @@ -2868,27 +2885,27 @@ DEPTH: max iterations the generator may take (agentic uses this; the

### ImprovementDriverOptions

Defined in: [improvement/improvement-driver.ts:54](https://github.com/tangle-network/agent-runtime/blob/main/src/improvement/improvement-driver.ts#L54)
Defined in: [improvement/improvement-driver.ts:65](https://github.com/tangle-network/agent-runtime/blob/main/src/improvement/improvement-driver.ts#L65)

#### Properties

##### worktree

> **worktree**: `WorktreeAdapter`

Defined in: [improvement/improvement-driver.ts:55](https://github.com/tangle-network/agent-runtime/blob/main/src/improvement/improvement-driver.ts#L55)
Defined in: [improvement/improvement-driver.ts:66](https://github.com/tangle-network/agent-runtime/blob/main/src/improvement/improvement-driver.ts#L66)

##### generator

> **generator**: [`CandidateGenerator`](#candidategenerator)

Defined in: [improvement/improvement-driver.ts:56](https://github.com/tangle-network/agent-runtime/blob/main/src/improvement/improvement-driver.ts#L56)
Defined in: [improvement/improvement-driver.ts:67](https://github.com/tangle-network/agent-runtime/blob/main/src/improvement/improvement-driver.ts#L67)

##### baseRef?

> `optional` **baseRef?**: `string`

Defined in: [improvement/improvement-driver.ts:58](https://github.com/tangle-network/agent-runtime/blob/main/src/improvement/improvement-driver.ts#L58)
Defined in: [improvement/improvement-driver.ts:69](https://github.com/tangle-network/agent-runtime/blob/main/src/improvement/improvement-driver.ts#L69)

Base ref candidate worktrees fork from. Default `main`.

Expand Down Expand Up @@ -8460,7 +8477,7 @@ Full-agentic `CandidateGenerator` (the `shots=N, sandbox=on` setting): run a rea

> **commandVerifier**(`command`, `args?`, `timeoutMs?`): [`Verifier`](#verifier)

Defined in: [improvement/agentic-generator.ts:237](https://github.com/tangle-network/agent-runtime/blob/main/src/improvement/agentic-generator.ts#L237)
Defined in: [improvement/agentic-generator.ts:247](https://github.com/tangle-network/agent-runtime/blob/main/src/improvement/agentic-generator.ts#L247)

A `Verifier` that runs a command in the worktree: exit 0 ⇒ ok, any other
exit ⇒ failed with stdout+stderr as feedback. The common case — verify by
Expand Down Expand Up @@ -8585,7 +8602,7 @@ Optimize the system prompt, default holdout gate:

> **improvementDriver**(`opts`): `SurfaceProposer`\<`AnalystFinding`\>

Defined in: [improvement/improvement-driver.ts:62](https://github.com/tangle-network/agent-runtime/blob/main/src/improvement/improvement-driver.ts#L62)
Defined in: [improvement/improvement-driver.ts:73](https://github.com/tangle-network/agent-runtime/blob/main/src/improvement/improvement-driver.ts#L73)

The one reflective/agentic improvement proposer (`SurfaceProposer`): owns the candidate worktree lifecycle and delegates HOW a change is produced to a pluggable `CandidateGenerator`.

Expand Down
45 changes: 28 additions & 17 deletions docs/api/mcp.md
Original file line number Diff line number Diff line change
Expand Up @@ -2512,7 +2512,7 @@ Default `[]` (no circular check unless the consumer declares its kinds).

### RunLocalHarnessOptions

Defined in: [mcp/local-harness.ts:108](https://github.com/tangle-network/agent-runtime/blob/main/src/mcp/local-harness.ts#L108)
Defined in: [mcp/local-harness.ts:129](https://github.com/tangle-network/agent-runtime/blob/main/src/mcp/local-harness.ts#L129)

**`Experimental`**

Expand All @@ -2522,15 +2522,15 @@ Defined in: [mcp/local-harness.ts:108](https://github.com/tangle-network/agent-r

> **harness**: [`LocalHarness`](#localharness)

Defined in: [mcp/local-harness.ts:109](https://github.com/tangle-network/agent-runtime/blob/main/src/mcp/local-harness.ts#L109)
Defined in: [mcp/local-harness.ts:130](https://github.com/tangle-network/agent-runtime/blob/main/src/mcp/local-harness.ts#L130)

**`Experimental`**

##### cwd

> **cwd**: `string`

Defined in: [mcp/local-harness.ts:111](https://github.com/tangle-network/agent-runtime/blob/main/src/mcp/local-harness.ts#L111)
Defined in: [mcp/local-harness.ts:132](https://github.com/tangle-network/agent-runtime/blob/main/src/mcp/local-harness.ts#L132)

**`Experimental`**

Expand All @@ -2540,7 +2540,7 @@ Working directory for the subprocess (typically a worktree path).

> **taskPrompt**: `string`

Defined in: [mcp/local-harness.ts:113](https://github.com/tangle-network/agent-runtime/blob/main/src/mcp/local-harness.ts#L113)
Defined in: [mcp/local-harness.ts:134](https://github.com/tangle-network/agent-runtime/blob/main/src/mcp/local-harness.ts#L134)

**`Experimental`**

Expand All @@ -2550,7 +2550,7 @@ Prompt forwarded as the harness CLI's task argument.

> `optional` **invocation?**: `object`

Defined in: [mcp/local-harness.ts:121](https://github.com/tangle-network/agent-runtime/blob/main/src/mcp/local-harness.ts#L121)
Defined in: [mcp/local-harness.ts:142](https://github.com/tangle-network/agent-runtime/blob/main/src/mcp/local-harness.ts#L142)

**`Experimental`**

Expand All @@ -2568,11 +2568,22 @@ is used unchanged.

> **args**: readonly `string`[]

##### dangerouslySkipPermissions?

> `optional` **dangerouslySkipPermissions?**: `boolean`

Defined in: [mcp/local-harness.ts:145](https://github.com/tangle-network/agent-runtime/blob/main/src/mcp/local-harness.ts#L145)

**`Experimental`**

Allow autonomous Claude edits without an interactive permission prompt.
Use only when `cwd` is an isolated candidate worktree.

##### timeoutMs?

> `optional` **timeoutMs?**: `number`

Defined in: [mcp/local-harness.ts:123](https://github.com/tangle-network/agent-runtime/blob/main/src/mcp/local-harness.ts#L123)
Defined in: [mcp/local-harness.ts:147](https://github.com/tangle-network/agent-runtime/blob/main/src/mcp/local-harness.ts#L147)

**`Experimental`**

Expand All @@ -2582,7 +2593,7 @@ Wall-clock kill deadline (ms). Default 5 min. Subprocess SIGTERMed on expiry.

> `optional` **signal?**: `AbortSignal`

Defined in: [mcp/local-harness.ts:125](https://github.com/tangle-network/agent-runtime/blob/main/src/mcp/local-harness.ts#L125)
Defined in: [mcp/local-harness.ts:149](https://github.com/tangle-network/agent-runtime/blob/main/src/mcp/local-harness.ts#L149)

**`Experimental`**

Expand All @@ -2592,7 +2603,7 @@ Caller cancellation. SIGTERM is sent on abort.

> `optional` **env?**: `ProcessEnv`

Defined in: [mcp/local-harness.ts:127](https://github.com/tangle-network/agent-runtime/blob/main/src/mcp/local-harness.ts#L127)
Defined in: [mcp/local-harness.ts:151](https://github.com/tangle-network/agent-runtime/blob/main/src/mcp/local-harness.ts#L151)

**`Experimental`**

Expand All @@ -2602,7 +2613,7 @@ Override env (defaults to inheriting from the parent).

> `optional` **spawn?**: (`command`, `args`, `opts`) => `ChildProcess`

Defined in: [mcp/local-harness.ts:132](https://github.com/tangle-network/agent-runtime/blob/main/src/mcp/local-harness.ts#L132)
Defined in: [mcp/local-harness.ts:156](https://github.com/tangle-network/agent-runtime/blob/main/src/mcp/local-harness.ts#L156)

**`Experimental`**

Expand Down Expand Up @@ -2641,7 +2652,7 @@ readonly `string`[]

### LocalHarnessResult

Defined in: [mcp/local-harness.ts:144](https://github.com/tangle-network/agent-runtime/blob/main/src/mcp/local-harness.ts#L144)
Defined in: [mcp/local-harness.ts:168](https://github.com/tangle-network/agent-runtime/blob/main/src/mcp/local-harness.ts#L168)

**`Experimental`**

Expand All @@ -2651,7 +2662,7 @@ Defined in: [mcp/local-harness.ts:144](https://github.com/tangle-network/agent-r

> **exitCode**: `number` \| `null`

Defined in: [mcp/local-harness.ts:146](https://github.com/tangle-network/agent-runtime/blob/main/src/mcp/local-harness.ts#L146)
Defined in: [mcp/local-harness.ts:170](https://github.com/tangle-network/agent-runtime/blob/main/src/mcp/local-harness.ts#L170)

**`Experimental`**

Expand All @@ -2661,7 +2672,7 @@ OS exit code. `null` when killed before exit.

> **stdout**: `string`

Defined in: [mcp/local-harness.ts:148](https://github.com/tangle-network/agent-runtime/blob/main/src/mcp/local-harness.ts#L148)
Defined in: [mcp/local-harness.ts:172](https://github.com/tangle-network/agent-runtime/blob/main/src/mcp/local-harness.ts#L172)

**`Experimental`**

Expand All @@ -2671,7 +2682,7 @@ Concatenated stdout.

> **stderr**: `string`

Defined in: [mcp/local-harness.ts:150](https://github.com/tangle-network/agent-runtime/blob/main/src/mcp/local-harness.ts#L150)
Defined in: [mcp/local-harness.ts:174](https://github.com/tangle-network/agent-runtime/blob/main/src/mcp/local-harness.ts#L174)

**`Experimental`**

Expand All @@ -2681,7 +2692,7 @@ Concatenated stderr.

> **killedBySignal**: `Signals` \| `null`

Defined in: [mcp/local-harness.ts:152](https://github.com/tangle-network/agent-runtime/blob/main/src/mcp/local-harness.ts#L152)
Defined in: [mcp/local-harness.ts:176](https://github.com/tangle-network/agent-runtime/blob/main/src/mcp/local-harness.ts#L176)

**`Experimental`**

Expand All @@ -2691,7 +2702,7 @@ Set when the process exited via signal (timeout / abort).

> **durationMs**: `number`

Defined in: [mcp/local-harness.ts:154](https://github.com/tangle-network/agent-runtime/blob/main/src/mcp/local-harness.ts#L154)
Defined in: [mcp/local-harness.ts:178](https://github.com/tangle-network/agent-runtime/blob/main/src/mcp/local-harness.ts#L178)

**`Experimental`**

Expand All @@ -2701,7 +2712,7 @@ Wall-clock duration ms (spawn → exit).

> **timedOut**: `boolean`

Defined in: [mcp/local-harness.ts:156](https://github.com/tangle-network/agent-runtime/blob/main/src/mcp/local-harness.ts#L156)
Defined in: [mcp/local-harness.ts:180](https://github.com/tangle-network/agent-runtime/blob/main/src/mcp/local-harness.ts#L180)

**`Experimental`**

Expand Down Expand Up @@ -7165,7 +7176,7 @@ then any consumer judges, returning on the first veto.

> **runLocalHarness**(`options`): `Promise`\<[`LocalHarnessResult`](#localharnessresult)\>

Defined in: [mcp/local-harness.ts:180](https://github.com/tangle-network/agent-runtime/blob/main/src/mcp/local-harness.ts#L180)
Defined in: [mcp/local-harness.ts:204](https://github.com/tangle-network/agent-runtime/blob/main/src/mcp/local-harness.ts#L204)

**`Experimental`**

Expand Down
10 changes: 10 additions & 0 deletions src/improvement/agentic-generator.ts
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,12 @@ export function agenticGenerator(opts: AgenticGeneratorOptions = {}): CandidateG

return {
kind: `agentic:${harness}`,
// The seed repo + (in rawTraceContext mode) the raw-trace filesystem context
// are the change signal — an agentic coder proposes from them even when the
// distiller yielded zero findings. Without this, the improvementDriver's
// empty-findings guard short-circuits and generates ZERO candidates on the
// first (and, for a single-generation run, only) proposal round.
proposesWithoutFindings: true,
async generate({ worktreePath, report, findings, maxShots, signal }) {
const basePrompt = buildPrompt({ report, findings })
const needsRawTraceEvidence = requiresRawTraceEvidence(findings)
Expand All @@ -98,6 +104,10 @@ export function agenticGenerator(opts: AgenticGeneratorOptions = {}): CandidateG
harness,
cwd: worktreePath,
taskPrompt: attemptNote ? `${basePrompt}\n\n${attemptNote}` : basePrompt,
// The candidate worktree is isolated and must be editable without an
// interactive permission prompt. Other runLocalHarness callers remain
// permission-safe by default.
dangerouslySkipPermissions: harness === 'claude',
timeoutMs: opts.timeoutMs,
signal,
})
Expand Down
27 changes: 25 additions & 2 deletions src/improvement/improvement-driver.ts
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,17 @@ import type {
* adapter's `finalize`. */
export interface CandidateGenerator {
kind: string
/** Whether this generator can produce a candidate from an EMPTY findings set
* and no phase-2 report — i.e. it draws its change signal from the repo and
* the raw-trace filesystem context on disk, not only from pre-summarized
* findings. An agentic coder (`agenticGenerator`) sets this: the seed repo +
* raw traces ARE the signal, so it must still run the full `populationSize`
* when the distiller yielded nothing (this is the meta-harness contract — the
* agent diagnoses from the raw traces itself). A patch-applier
* (`reflectiveGenerator`) leaves it unset — with no findings there is no
* patch to draft, so the driver short-circuits rather than spin up worktrees
* for a guaranteed no-op. Default `false`. */
proposesWithoutFindings?: boolean
generate(args: {
/** The candidate worktree — a fresh checkout of baseRef. Write changes here. */
worktreePath: string
Expand Down Expand Up @@ -66,8 +77,20 @@ export function improvementDriver(opts: ImprovementDriverOptions): SurfacePropos
kind: `improvement:${opts.generator.kind}`,
async propose(ctx: ProposeContext<AnalystFinding>) {
const findings = resolveFindings(ctx)
// No signal to act on — propose nothing rather than spin up worktrees.
if (findings.length === 0 && ctx.report === undefined) return []
// No findings AND no report AND a generator that can only act on findings
// (the reflective patch-applier) — propose nothing rather than spin up
// worktrees for a guaranteed no-op. An agentic coder draws its signal from
// the repo + raw traces on disk, so it opts in via `proposesWithoutFindings`
// and still runs the full populationSize even on an empty findings set —
// otherwise the FIRST generation (whose seed findings are empty and whose
// rawTraceDistiller has not run yet) would always generate ZERO candidates.
if (
findings.length === 0 &&
ctx.report === undefined &&
!opts.generator.proposesWithoutFindings
) {
return []
}

const surfaces: CodeSurface[] = []
for (let i = 0; i < ctx.populationSize; i++) {
Expand Down
27 changes: 27 additions & 0 deletions src/improvement/raw-trace-distiller.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -218,4 +218,31 @@ describe('rawTraceDistiller', () => {
const findings = await rawTraceDistiller({ fallbackFindings: seed })(input)
expect(findings).toEqual(seed)
})

it('emits the default raw-trace instruction (not []) when the fallback seed is EMPTY on a clean generation', async () => {
// The runner passes findings:[] as the seed, so fallbackFindings is []. An
// empty fallback must NOT wipe the steering context to nothing — it must fall
// through to the default raw-trace instruction so the meta-harness discipline
// (and the agenticGenerator diagnosis-evidence gate keyed on the
// `raw-trace-context` area) stays armed for the next generation.
const input = {
generation: 0,
runDir: join(root, 'gen-0'),
candidates: [
{
surfaceHash: 'perfect',
composite: 1,
campaign: {
runDir: join(root, 'gen-0', 'candidate-0'),
cells: [{ cellId: 's:0', scenarioId: 's', judgeScores: { j: { composite: 1 } } }],
},
},
],
history: [],
} as unknown as AnalyzeInput

const findings = await rawTraceDistiller({ fallbackFindings: [] })(input)
expect(findings).toHaveLength(1)
expect((findings[0] as unknown as { area: string }).area).toBe('raw-trace-context')
})
})
37 changes: 22 additions & 15 deletions src/improvement/raw-trace-distiller.ts
Original file line number Diff line number Diff line change
Expand Up @@ -111,22 +111,29 @@ export function rawTraceDistiller<TScenario extends Scenario = Scenario, TArtifa
const totalFailingCells = ranked.reduce((n, c) => n + c.cells.length, 0)

// A clean generation: keep the proposer's steering context rather than
// wiping it (parity with the default distiller's static-seed fallback).
// wiping it (parity with the default distiller's static-seed fallback). An
// EMPTY fallback array means there is no STATIC seed to preserve — it must NOT
// wipe the context to nothing. Fall through to the default raw-trace
// instruction so the meta-harness discipline stays live (the agent still
// inspects the on-disk traces next round, and the finding's `raw-trace-context`
// area keeps the agenticGenerator's diagnosis-evidence gate armed). A bare
// `??` would return the empty array and silently disable both.
if (totalFailingCells === 0) {
return (
options.fallbackFindings ?? [
makeFinding({
analyst_id: ANALYST_ID,
severity: 'info',
area: 'raw-trace-context',
confidence: 1,
claim: `Generation ${input.generation} had no failing cells. The full raw run traces are on disk under ${genRoot}.`,
recommended_action: `To keep improving, grep/cat the raw traces under ${genRoot} (per-cell spans.jsonl + cached-result.json) to find the weakest passing runs, then make a targeted harness-code edit.`,
evidence_refs: [{ kind: 'artifact', uri: genRoot }],
metadata: { generation: input.generation, runDir: genRoot, failingCells: 0 },
}),
]
)
if (options.fallbackFindings && options.fallbackFindings.length > 0) {
return options.fallbackFindings
}
return [
makeFinding({
analyst_id: ANALYST_ID,
severity: 'info',
area: 'raw-trace-context',
confidence: 1,
claim: `Generation ${input.generation} had no failing cells. The full raw run traces are on disk under ${genRoot}.`,
recommended_action: `To keep improving, grep/cat the raw traces under ${genRoot} (per-cell spans.jsonl + cached-result.json) to find the weakest passing runs, then make a targeted harness-code edit.`,
evidence_refs: [{ kind: 'artifact', uri: genRoot }],
metadata: { generation: input.generation, runDir: genRoot, failingCells: 0 },
}),
]
}

const findings: AnalystFinding[] = []
Expand Down
Loading
Loading