Skip to content

supervisor-run: read agent-runtime's observer projection instead of re-deriving spend and placement #711

Description

@drewstone

Follow-up to #708. The four gaps in that issue are fixed in the PR that cites this one; this issue is the structural version of the same problem.

The two derivations

agent-runtime now writes observer.jsonl beside spawn-journal.jsonl — a digest-chained FileObserverJournal — and ships projectPursuit(records) in @tangle-network/agent-runtime/durable. That projection already produces, per node, the facts src/supervisor-run re-derives from the spawn journal:

  • PursuitNodeProjection.spent / ownInference — per-node spend, with Runtime's own usdKnown / tokensKnown flags intact.
  • PursuitNodeProjection.placement — the execution binding the supervisor-run reader cannot recover at all today.
  • materialization, executionBindings, providerModel, trace, model, backend — evidence the reader reports as unavailable.
  • Inclusive/exclusive telescoping across the tree, plus the run-level spendGaps naming each unaccounted node.

So the same run has two spend projections: Runtime's, and this package's. #708 was four instances of the second one being wrong where the first one is right. That is a design fault, not four bugs.

Why this is not a small change

Eval must not depend on agent-runtime, and agent-runtime does not depend on eval. The projection and the record types live entirely in agent-runtime:

  • ObserverRecord, ObserverJournal, FileObserverJournalagent-runtime/src/durable/observer-journal.ts
  • RuntimeHookEvent, RuntimeDecisionPointagent-runtime/src/runtime-hooks.ts
  • projectPursuit, PursuitProjection, PursuitNodeProjectionagent-runtime/src/durable/observer-projection.ts (~28 KB)

Reading the file is allowed; re-implementing the projection to read it is not — a second implementation of a 28 KB reducer is the fork this repository's rules exist to prevent, and it would drift from Runtime's on the first schema change.

The record shape, for whoever takes this

One line of observer.jsonl, from ~/code/discovery-lab/pursuits/runner-smoke-20260901/observer.jsonl:

{"schemaVersion":1,"pursuitId":"runner-smoke-20260901","sequence":3,"kind":"event",
 "observedAt":1788275579122,"previousDigest":"7564ed56…",
 "event":{"id":"runner-smoke-20260901:s0:settled","runId":"runner-smoke-20260901",
   "target":"agent.child","phase":"after","timestamp":1788275579115,"stepIndex":0,
   "parentId":"runner-smoke-20260901",
   "payload":{"childId":"runner-smoke-20260901:s0","status":"done","score":1,"valid":true,
     "spent":{"iterations":1,"tokens":{"input":31608,"output":221,"freshInput":5496,
       "cacheRead":26112,"cacheWrite":0},"usd":0,"usdKnown":false,"ms":17995},
     "runtime":"sandbox","materialization":{},"executionBindings":[{}]},
   "pursuitId":"runner-smoke-20260901"},
 "digest":"a606b25c…"}

Envelope: schemaVersion, pursuitId, sequence, kind (event | decision), observedAt, previousDigest, digest, and one of event / decision. Targets seen on a five-record smoke run: agent.run (phase: before / after, payload.status), agent.spawn (payload.childId, label, assignmentId, runtime, identity, budget, depth, attemptId), agent.child (as above), agent.turn (payload.spend, role, runtime). previousDigest + digest chain the file, so deletion, reordering, and mutation are detectable — which the spawn journal is not.

The options, ranked

  1. Move the contract down. Publish ObserverRecord, the hook-event types, projectPursuit, and PursuitProjection from @tangle-network/agent-interface (or a small agent-observer package), and have agent-runtime and agent-eval both consume it. One owner of the projection, no dependency edge from eval to runtime. Preferred.
  2. A narrow port. agent-runtime exports a pure projectPursuit with no runtime imports, published as its own entry point that eval may depend on for types plus that one function. Smaller, but it makes eval depend on runtime.
  3. Do nothing and keep two derivations. The status quo, and the reason fix(supervisor-run): the runtime reader ignores result.kind and a settled verdict, and collapses totals on one unknown #708 existed.

Until one of these lands, the supervisor-run reader stays on the spawn journal, and every fact only the observer plane carries — placement above all — stays unavailable.

A consumer already has both halves in one file and can be used as the acceptance test: discovery-lab/runner/journal.mjs imports analyzeSupervisorRunSources from agent-eval AND projectPursuit from agent-runtime, and branches on whether observer.jsonl exists. When this issue is closed, that branch collapses.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions