[factory] Unit tests write the real daemon's heartbeat and registry at /tmp/factory-run, clobbering live state

[khaliqgant]

Summary

The shared config() helper in src/orchestrator/factory.test.ts sets no loop.heartbeatPath and no
loop.registryPath:

const config = (overrides: FactoryConfigOverrides = {}): FactoryConfig => FactoryConfigSchema.parse({
  workspaceId: 'factory-test',
  repos: { ... },
  triage: { maxImplementers: 4 },
  batchSize: 2,
  ...
})

So every test built on it falls back to the schema defaults (src/config/schema.ts):

/tmp/factory-run/factory-loop-heartbeat.json
/tmp/factory-run/factory-loop-registry.json

Those are the real daemon's paths, not test paths. Any test that starts a factory in a live mode writes
them.

Demonstration

Running only the three terminal Slack receipt lease tests, watching the host's real run directory:

before:    Aug 22 15:59:10  1975 bytes   (factory-loop-registry.json)
hb before: Aug 22 15:59:10  1989 bytes   (factory-loop-heartbeat.json)

after:     Aug 22 15:59:20   169 bytes
hb after:  Aug 22 15:59:19  1653 bytes

The unit test truncated both files. The registry it left behind:

{
  "pid": 46378,
  "heartbeatPath": "/tmp/factory-run/factory-loop-heartbeat.json",
  "updatedAt": "1970-01-01T00:00:20.000Z",
  "updatedAtMs": 20000,
  "agents": []
}

updatedAtMs: 20000 is the test's ManualClock, so the file is stamped 1970.

Why this matters

0. The mechanism is a synthesised path, not an empty value. The absent key does not leave the runtime
with nothing — schema.ts:169-170 carries .default('/tmp/factory-run/factory-loop-heartbeat.json') and
.default('/tmp/factory-run/factory-loop-registry.json'), so FactoryConfigSchema.parse synthesises a
fixed absolute path for every test that omits it. Reading the schema rather than the test is what makes
this visible.

1. It corrupts a live daemon's state. On any host running a real Factory — a developer laptop, a shared
runner — running the unit suite overwrites the daemon's heartbeat with a 1970 timestamp and its registry with
an empty agent list. A crash reaper or /healthz reading that file sees a heartbeat ~56 years stale:
checkFactoryLoopLiveness returns heartbeat stale, and the live daemon is declared dead. The registry's
empty agents: [] is the same hazard for anything that reads in-flight agents.

Because those paths are fixed and absolute, the collision is not merely cross-test or cross-worker — it
is cross-process. The full chain, none of it speculative:

a test run and a live Factory daemon on the same host write the same two files
→ the daemon's heartbeat is corrupted by a process that is not the daemon
→ before #324 that collision produced a torn read
→ a torn read is swallowed into undefined by readFactoryLoopHeartbeat
→ undefined is indistinguishable from "file absent"
→ checkFactoryLoopLiveness returns heartbeat missing
→ a live daemon reads as dead, and the crash reaper reads exactly this file to decide that.

#324 breaks that chain at the third step: a reader now sees a whole document rather than a spliced one. It
does not break it at the first step. A test that writes a 1970 timestamp over a live daemon's heartbeat
produces a whole, valid, and wrong document, and heartbeat stale is the same verdict as heartbeat missing
for the reaper's purposes.

Observed live on a developer host

The cross-process arm above is not hypothetical. It was caught in the act on 2026-08-22, twice, by two
people looking independently.

Observation 1 (~14:02 UTC). /tmp/factory-run/factory-loop-heartbeat.json contained "pid": 51121.
PID 51121 was a node process under
.../AgentWorkforce/.lanes/factory-lead/node_modules/.bin/vitest, child of vitest run at PID 51114.
The file names its own writer, and the writer is a test runner. No inference required. The registry's
mtime was sampled twice 6s apart and moved (16:02:52 → 16:02:57 local), so the writing was ongoing.

Observation 2 (~14:11 UTC), a separate run. The same file had already been overwritten again, by a
different process:

{
  "pid": 64478,
  "status": "stopping",
  "updatedAt": "2026-08-22T14:11:02.864Z",
  "eventListener": { "state": "not-listening", "reason": "live daemon heartbeat is inactive" }
}

PID 64478 no longer exists. So the file currently advertises status: "stopping" and a not-listening
event listener — state authored by a test process that has since exited, left standing as the daemon's
published health. Anything reading that path now (the crash reaper, /healthz, factory diagnose) is told
the daemon is stopping.

To be explicit about my own part in this: unit-test runs from this lane are among the processes that
overwrote that file while investigating. That is the defect, not a side effect of investigating it.

Precision, held deliberately. This proves the collision mechanism is live on this host. It does
not prove that any specific past /healthz reading, or any specific confusion on any lane, was caused by
it. No historical reading is attributed here. What is recorded is what was observed, with pids and
timestamps; the inference is the reader's to draw.

Operational note. While a test suite runs on the same box as a daemon, a heartbeat read from
/tmp/factory-run may be describing the test runner rather than the daemon. Anyone diagnosing from that
file during a test run should confirm the pid in it belongs to the daemon they mean.

On file modes. The heartbeat and registry are 0644, while their siblings in the same directory
(factory-cloud-events.json, github-issue-comment-watches.json) are 0600. The 0644 is consistent with
files intended for out-of-process readers, and #324 preserves the destination's existing mode rather than
imposing one.

2. It is cross-worker interference in CI. npm run test runs vitest with a parallel worker pool inside
one container, and /tmp/factory-run is shared across those workers. Tests in different workers write and
read the same two paths concurrently, with no isolation between them.

3. It is a plausible cause of the lease-test flake. The three terminal Slack receipt lease tests fail
together with expected false to be true and have failed Publish twice. I instrumented their fake-timer
budget: the receipt write is normally entered after 5-11 ticks of a 200 budget, but in a failing run the
loop burns all 200 and never enters it — a bimodal failure, not a budget that is marginally too small.
That shape is consistent with cross-test interference on shared global state, and is not explained by tick
accounting.

Stated carefully: this is a hypothesis for the flake, not a demonstrated cause, and it is a hypothesis
that has already been tried once. An earlier lane blamed this shared state for the lease flake and ran an A/B
that did not support it. Note also that A/Bs on this test file have very low power — the counts involved
(1-of-6 vs 0-of-6) are well within chance — so that result does not settle the question in either direction.
Anyone picking this up should not re-run the same A/B expecting it to decide anything.

What is demonstrated above is the shared-state write itself, which is a defect on its own terms regardless of
the flake. That is the reason to fix it; the flake connection is unproven and should not be the
justification.

Relationship to #323

Separate defect. #323 (non-atomic write) is about the file being torn while read; this is about the file
being the wrong file. Fixing #323 removes the corruption that this collision can produce — a concurrent
reader no longer sees a spliced document — but it does not stop a test from clobbering a live daemon's
heartbeat with a 1970 timestamp, which is a whole, valid, and wrong document either way.

Suggested fix

Give the test config() helper per-run temp paths for loop.heartbeatPath and loop.registryPath, so no
test touches /tmp/factory-run unless it explicitly opts in. Individual tests that already pass explicit
paths (join(root, 'heartbeat.json') and friends) are unaffected and show the intended pattern.