Skip to content

Reconciliation never recovers a sandbox child: 83% of lost children end there, 0 of 35 came back, and fleet yield is reported nowhere #1247

Description

@drewstone

The measurement

Across every retained pursuit on one operator host (385 runs, deduplicated across seven discovery-lab checkouts):

child runtime spawned never settled
tangle-sandbox 1,173 223 (19.0%)
cli 132 2 (1.5%)

Of the 223 lost sandbox children, 185 (83%) stop at exactly one lifecycle stage: reconciled. The rest stop at execution-admitted (31) or execution-result (7).

In capability-per-parameter-cpp-glm-20260915c, 0 of 35 children that reached reconciled ever settled. Not one.

This is not a budget bug

src/runtime/supervise/scope.ts states the contract, and it is correct:

The child-work floor a RETAINED-PENDING child's reservation was reconciled at. Its cursor slot stays open, so the settle path journals this as a reconciled record in place of the settled record it cannot write — otherwise every journal reader charges the ceiling the pool just refunded.

The pool refunds. I checked a reservation-leak hypothesis against this and it is wrong; spentTotal stays honest.

What is lost is the child's work. One sample record: spent: { iterations: 0, tokens: { input: 0, output: 0 }, ms: 19636 } — admitted, ran twenty seconds, produced nothing, reconciled, gone. Its transcript goes with it (#1244), and the director that spawned it simply never hears back.

Why it is invisible

Three views disagree and none of them shows the number that matters:

  • the budget pool says everything is fine, because it is;
  • disco report counts these as "down", which reads like a failure with a cause;
  • the journal says reconciled, a word that promises recovery that never arrives.

Fleet yield — settled over spawned — is reported nowhere. Measured on the two runs from tonight:

run spawned settled yield
capability-per-parameter-cpp-ds-20260915c 48 38 79%
capability-per-parameter-cpp-glm-20260915c 69 34 49%

A director lost half its fleet and nothing in the operator's view said so.

Ask

  1. Make reconciliation recover, or stop calling it that. If a retained-pending execution is unobservable, the child should reach a terminal state with a stated reason rather than an open cursor slot that never closes. 0 of 35 is not a recovery path.
  2. Report fleet yield. Settled over spawned, per run, in disco report and on the settle record. It is one division and it is the number that tells an operator their fleet is half working.
  3. A reconciled child and a down child are different facts; a reader that renders both as "down" hides which one an operator can act on.

Related: #1244 (a dropped child's transcript dies with it), #1245 (a sandbox-placed root cannot use its coordination tools).

🤖 Generated with Claude Code

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions