feat(orchestrator): distinguish a mount serving empty trees from one timing out (#351)

docs/deployed-diagnostics.md

@@ -185,6 +185,27 @@ logic of its own by design: the boundary lives in one place, in this repo, with
   not running the loop at all, or it predates #351 — on a current build a hung call fails within
   `relayfileOperationTimeoutMs`.
 
+- **`treeReads` / `emptyTreeReads`** — the case a timeout cannot catch. A mount that starts serving
+  *empty* trees instead of hanging raises no timeout, no failure and no `lastError`: the sweep
+  completes `healthy` and dispatches nothing, which on every other field is indistinguishable from a
+  workspace that simply has no ready work.
+
+  Read them as a pair, against `candidates`. An empty read on its own is ordinary — a healthy sweep
+  lists two path forms per repo and only one of them exists. The fault is
+  `emptyTreeReads === treeReads` with `treeReads > 0`: the mount served nothing at all. So
+  `candidates: 0, treeReads: 3, emptyTreeReads: 1` is an empty workspace, and
+  `candidates: 0, treeReads: 3, emptyTreeReads: 3` is a silent mount.
+
+  `factory diagnose --deployed` makes that reading for you: it renders a `tree reads` line and
+  folds the verdict into the "Last enumerating sweep" sentence, so an all-empty pass says
+  `the mount served nothing at all` rather than leaving a zero `candidates` to speak for itself.
+
+  Both numbers count only the reads the readiness sweep's own discovery pass issued. In live mode
+  event drains and completion timers list trees too, and a populated lookup landing in the
+  denominator would make `emptyTreeReads < treeReads` on a sweep whose every discovery read was
+  empty — masking the fault. So a sweep whose roots all came from the discovery cache reports
+  `treeReads: 0`, which claims nothing in either direction.
+
 ### Why `ok` stays `true` while `status` goes amber
 
 `/healthz` is the Cloudflare **Container ping endpoint** (`pingEndpoint = 'localhost/healthz'` in the

src/cli/diagnose.test.ts

@@ -123,6 +123,74 @@ describe('formatSweepOutcome (#359)', () => {
     )
     expect(outcome).not.toContain('nothing has enumerated successfully yet')
   })
+
+  // MUST-FIRE (#363 review, codex P1). The numbers reaching `/healthz` is only
+  // half of getting the signal to the operator: `candidates: 0` with every
+  // tree read empty and `candidates: 0` with content served are opposite
+  // diagnoses, and this sentence is where a deployed operator meets them. It
+  // is also the sentence the verdict line embeds, so the reading travels with
+  // the "dispatching" claim it qualifies.
+  it('names a silent mount rather than leaving a zero candidate count to speak for itself', () => {
+    const outcome = formatSweepOutcome({
+      state: 'healthy',
+      consecutiveFailures: 0,
+      failureThreshold: 3,
+      candidates: 0,
+      dispatched: 0,
+      skipped: 0,
+      treeReads: 3,
+      emptyTreeReads: 3,
+    })
+
+    expect(outcome).toContain('0 candidate(s)')
+    expect(outcome).toContain('every one of 3 tree read(s) came back empty')
+    expect(outcome).toContain('the mount served nothing at all')
+  })
+
+  // MUST-NOT-FIRE, and it is the reason this is a pair rather than a count: a
+  // healthy sweep lists two path forms per repo and only one exists, so an
+  // empty read is ORDINARY. A renderer that shouted on any empty read would
+  // shout on every healthy instance, and an operator would learn to ignore it.
+  it('stays quiet when the mount served content, and when no read was made', () => {
+    const emptyWorkspace = formatSweepOutcome({
+      state: 'healthy',
+      consecutiveFailures: 0,
+      failureThreshold: 3,
+      candidates: 0,
+      dispatched: 0,
+      skipped: 0,
+      treeReads: 3,
+      emptyTreeReads: 1,
+    })
+    expect(emptyWorkspace).not.toContain('served nothing at all')
+    expect(emptyWorkspace).toContain('1/3 tree read(s) empty')
+
+    // An incremental sweep that answered every root from the discovery cache
+    // issued no read at all. A ratio over zero reads is not a fact about the
+    // mount, so it claims neither direction.
+    const noReads = formatSweepOutcome({
+      state: 'healthy',
+      consecutiveFailures: 0,
+      failureThreshold: 3,
+      candidates: 0,
+      dispatched: 0,
+      skipped: 0,
+      treeReads: 0,
+      emptyTreeReads: 0,
+    })
+    expect(noReads).toBe('0 candidate(s), 0 dispatched, 0 skipped')
+
+    // A producer that predates the pair says nothing about it either.
+    const olderProducer = formatSweepOutcome({
+      state: 'healthy',
+      consecutiveFailures: 0,
+      failureThreshold: 3,
+      candidates: 0,
+      dispatched: 0,
+      skipped: 0,
+    })
+    expect(olderProducer).toBe('0 candidate(s), 0 dispatched, 0 skipped')
+  })
 })
 
 describe('factory diagnose --deployed (#295)', () => {

src/cli/diagnose.ts

@@ -366,6 +366,7 @@ export function formatSweepOutcome(
   }
   return `${readiness.candidates} candidate(s), ${readiness.dispatched ?? 0} dispatched, ` +
     `${readiness.skipped ?? 0} skipped` +
+    formatTreeReadReading(readiness) +
     (readiness.discoveryDeferred
       // Name the instant, not just "an earlier pass" (#359 review): retained
       // counts sit beside an ever-fresh `lastCompletedAtMs`, so without this
@@ -378,6 +379,32 @@ export function formatSweepOutcome(
       : '')
 }
 
+/**
+ * The tree-read pair, turned into the sentence an operator needs (#363 review).
+ *
+ * The numbers reaching `/healthz` is half the fix; the other half is that
+ * `candidates: 0, treeReads: 3, emptyTreeReads: 3` and
+ * `candidates: 0, treeReads: 3, emptyTreeReads: 1` mean opposite things and
+ * nothing on this surface said which was which. The first is a mount serving
+ * nothing at all — dispatch is dead upstream of eligibility; the second is a
+ * workspace with no ready work.
+ *
+ * Silent about a zero `treeReads`: a sweep that issued no enumerating read
+ * (deferred, or shed) has nothing to report here, and a ratio over zero reads
+ * is not a fact about the mount.
+ */
+export function formatTreeReadReading(
+  readiness: FactoryPublicReadinessReconcileHealth | undefined,
+): string {
+  const treeReads = readiness?.treeReads
+  const emptyTreeReads = readiness?.emptyTreeReads
+  if (treeReads === undefined || emptyTreeReads === undefined || treeReads === 0) return ''
+  return emptyTreeReads === treeReads
+    ? ` — every one of ${treeReads} tree read(s) came back empty: the mount served nothing at all,` +
+      ' so a zero candidate count here is not evidence the workspace is empty'
+    : ` (${emptyTreeReads}/${treeReads} tree read(s) empty — the mount served content)`
+}
+
 /** The skip breakdown, ordered as the record carries it. */
 export function formatSkipReasons(counts: Record<string, number>): string {
   const entries = Object.entries(counts).filter(([, count]) => count > 0)
@@ -549,6 +576,17 @@ export function renderDeployedDiagnosis(diagnosis: DeployedFactoryDiagnosis): st
           }`,
         )
       }
+      // Rendered whenever the producer reports the pair, zeroes included: the
+      // raw numbers are what an operator diffs across two samples, and the
+      // reading beside them is what they act on (#363 review).
+      if (readiness.treeReads !== undefined && readiness.emptyTreeReads !== undefined) {
+        lines.push(
+          `    tree reads         : ${readiness.treeReads} served, ${readiness.emptyTreeReads} empty` +
+            (readiness.treeReads > 0 && readiness.emptyTreeReads === readiness.treeReads
+              ? ' — SILENT MOUNT: it served nothing at all'
+              : ''),
+        )
+      }
     }
     const capacity = health.dispatchCapacity
     if (capacity) {