feat(reliability): measure observer overhead with a synthetic DSH session

loopx/capabilities/reliability_diagnostics/README.md

@@ -209,6 +209,33 @@ covers legacy readback and an explicit offline upgrade. Legacy files remain
 readable, but block new appends until reconciled; mixed ownership must hold
 deletion. Upgrade CLI and plugin together with writers frozen.
 
+## Measure Observer Overhead
+
+`packages/dsh-loopx-plugin/bench/observer-overhead.ts` measures the observer
+component cost on a deterministic synthetic DSH event stream: observer off/on
+arms, CPU (user+system), RSS, ledger append through the shipped writer, ledger
+bytes per envelope and stats bytes per flush, per-trial event loss and buffer
+peaks, and the clock-source mix with full denominators. It also checks that
+every planned record is observed with the planned envelope-kind mix. A trial
+that drops or rejects events, fails an append, persists fewer envelopes than it
+accepted, or misses that record mix is excluded from the timing summary; the
+report is marked not usable and lists warnings, so a lossy or incompletely
+consumed run cannot read as a clean result. It writes a public-safe
+JSON + Markdown report and states its limits. It does not measure a live DSH
+host, model, network, or Cordis dispatch, and it does not qualify C1. See the
+[bench README](../../../packages/dsh-loopx-plugin/bench/README.md).
+
+Run it from `packages/dsh-loopx-plugin`:
+
+```bash
+node --experimental-strip-types bench/observer-overhead.ts --events 10000
+```
+
+`--events` is normalized down to a whole six-event turn cycle (5,000 → 4,998;
+10,000 → 9,996); the report states both counts. The bench uses a 4,096 buffer
+bound for the synthetic burst; pass `--buffer-bound 256` to mirror the observer's
+product default.
+
 ## Validation
 
 ```bash
@@ -231,5 +258,6 @@ No dashboard surface, no L2 recommendations, no automatic recovery, no
 writeback into goals, todos, gates, or session runtime, and no change to the
 `loopx status` first screen. The adapter requires an externally pinned
 goal/session/run identity; it does not discover bindings through the Driver or
-LoopX CLI. This prototype also does not provide matched native/L1 execution,
-observer CPU/I/O/latency/storage measurement, or an eligible C1 run.
+LoopX CLI. This prototype also does not provide matched native/L1 execution, a
+live DSH host measurement, or an eligible C1 run. The bench tool measures
+observer component cost on a synthetic stream; it is not a live qualification.

loopx/capabilities/reliability_diagnostics/README.zh-CN.md

@@ -177,6 +177,28 @@ live 部署验收。操作 regular file 前会验证 provider 目录与 CLI 的
 旧文件仍可只读回查，但必须离线核对后才能恢复追加；方案说明了升级和回退步骤，
 归属不明确时须暂停删除。冻结 writer 后一起升级 CLI 与插件。
 
+## 测量 observer 开销
+
+`packages/dsh-loopx-plugin/bench/observer-overhead.ts` 在确定性的 synthetic DSH 事件流上
+测量 observer 组件开销：observer 关闭/开启两臂、CPU（user+system）、RSS、经 shipped writer
+的 ledger append 延迟、每条 envelope 的 ledger 字节与每次 flush 的 stats 字节、逐试验的
+event loss 与 buffer 峰值，以及 clock source 构成和完整分母；并校验计划记录与 envelope kind
+构成是否被完整消费。丢弃或拒绝事件、append 失败、persisted 少于 accepted、或未覆盖计划记录构成
+的试验会从计时汇总中排除；报告标记为不可用并列出警告，因此有丢失或未完整消费的运行不会被读成
+干净结果。输出 public-safe 的 JSON + Markdown 报告，并声明限制；不测量真实
+DSH host、模型、网络或 Cordis dispatch，也不构成 C1 资格。见
+[bench README](../../../packages/dsh-loopx-plugin/bench/README.md)。
+
+在 `packages/dsh-loopx-plugin` 下运行：
+
+```bash
+node --experimental-strip-types bench/observer-overhead.ts --events 10000
+```
+
+`--events` 会向下归一化为 6 事件 turn cycle 的整数倍（5,000 → 4,998；10,000 → 9,996），
+报告同时给出请求值与归一化值。bench 对 synthetic 突发使用 4,096 buffer bound；传
+`--buffer-bound 256` 可对齐 observer 的产品默认值。
+
 ## 验证
 
 ```bash
@@ -195,5 +217,5 @@ projection 报告 `read` 上的重复、一次已恢复的错误、无 stall。
 
 不做 dashboard、不做 L2 建议、不做自动恢复、不回写 goal / todo / gate / session runtime、
 不改 `loopx status` 首屏。适配器要求外部固定 goal/session/run identity；不会通过 Driver
-或 LoopX CLI 发现绑定。本原型也不提供 matched native/L1 执行、observer CPU/I/O/延迟/
-存储开销测量，或一次合格 C1 实跑。
+或 LoopX CLI 发现绑定。本原型也不提供 matched native/L1 执行、真实 DSH host 测量，或一次合格 C1 实跑。
+bench 工具只在 synthetic 事件流上测量 observer 组件开销，不构成 live 资格。

packages/dsh-loopx-plugin/bench/README.md

@@ -0,0 +1,120 @@
+# Observer overhead bench
+
+Measure the DSH shadow observer's component cost on a deterministic synthetic
+event stream. This is a development tool for the P0 "measured overhead" gap in
+[#5211](https://github.com/loopx-project/loopx/issues/5211); it is not a live
+DSH host, model, or C1 qualification.
+
+## What it measures
+
+- **Two arms**: a baseline driver that walks the same planned stream without an
+  observer (no serialization or appends), and the real `ShadowObserver`.
+- **CPU**: `process.cpuUsage()` user + system delta.
+- **RSS**: process RSS before/after each trial.
+- **Ledger append**: p50/p95 latency through the shipped `appendLedgerLines`
+  writer, envelope bytes per persisted envelope, and stats bytes per flush.
+- **Event accounting**: observed, accepted, persisted, dropped (total and max),
+  rejected, peak buffer, and flush attempts. Counts must satisfy
+  `observed = accepted + rejected + dropped`.
+- **Clock**: the harness-time vs wall-clock envelope mix and the maximum
+  declared uncertainty.
+- **Integrity**: no event loss, persisted envelopes equal accepted envelopes,
+  every planned record observed with the planned envelope-kind mix, no outbound
+  endpoints, no worker-context influence, and no scheduler-input influence.
+
+A trial that drops or rejects events, fails an append, persists fewer envelopes
+than it accepted, or does not consume the planned record and envelope-kind mix
+is excluded from the arm summaries. The report is then marked `NOT USABLE`,
+prints the per-trial loss and peaks, and lists warnings. Do not use such a run
+as an overhead sample: rerun with a larger `--buffer-bound` or fewer `--events`
+(or fix the bench mix or observer when planned kinds stop being consumed).
+
+The report is public-safe: it contains aggregate numbers, the Node/platform
+version, and limits. It does not contain goal ids, session ids, ledger paths, or
+raw event content.
+
+## Run
+
+From `packages/dsh-loopx-plugin`:
+
+```bash
+node --experimental-strip-types bench/observer-overhead.ts \
+  --events 10000 \
+  --json ../../.local/observer-overhead.json \
+  --markdown ../../.local/observer-overhead.md
+```
+
+Defaults are 5,000 events, 3 repeats, and a buffer bound of 4,096. The bench
+uses a larger bound than the observer's product default (256) because the
+synthetic producer emits events far faster than a real session; pass
+`--buffer-bound 256` to mirror the product default and exercise backpressure
+drops. `--events` is normalized down to a whole six-event turn cycle: 5,000
+becomes 4,998 and 10,000 becomes 9,996, and the report prints both counts. The
+tool runs a short warmup, then alternates baseline and observer trials. Use
+`--json -` or `--markdown -` to print a report to stdout.
+
+The observer requests a flush at every turn end, but an append still in flight
+coalesces later requests, so the number of append batches (and the peak buffer)
+depends on I/O latency. The report prints the actual flush attempts and peak
+buffer; treat the per-event numbers as a signal, not a workload forecast.
+
+Example output from one Windows/Node 26 run of `--events 10000` (numbers are
+machine-specific and are not a qualification):
+
+```text
+- Status: **usable**
+- Events per trial: 9,996 (requested 10,000; normalized to whole 6-event turn cycles)
+- Buffer bound: 4,096
+
+| Metric | Baseline | Observer | Delta |
+| --- | ---: | ---: | ---: |
+| CPU total, median (ms) | 0.00 | 906.00 | 906.00 |
+| CPU, median (ms / 1k events) | 0.00 | 90.64 | 90.64 |
+| RSS delta, median | 448.0 KiB | 13.1 MiB | 12.7 MiB |
+| Ledger append p50 (ms) | n/a | 1.49 | n/a |
+| Ledger append p95 (ms) | n/a | 2.09 | n/a |
+| Ledger bytes / envelope | n/a | 490.38 | n/a |
+| Ledger stats bytes / flush | n/a | 1326.25 | n/a |
+| Persisted envelopes, median | n/a | 9,998 | n/a |
+| Clock source, median (harness / wall-clock) | n/a | 9,996 / 2 | n/a |
+| Clock uncertainty max, median (ms) | n/a | 50.00 | n/a |
+
+| Observed | Accepted | Persisted | Dropped (total) | Dropped (max) | Rejected | Flush attempts | Peak buffer (max) |
+| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
+| 9,998 | 9,998 | 9,998 | 0 | 0 | 0 | 218 | 84 |
+
+- Planned records not observed: 0
+- Unsupported envelopes: 0
+- No event loss: passed
+- Persisted equals accepted: passed
+- All trials usable: passed
+- Observed matches plan: passed
+- Planned envelope mix complete: passed
+```
+
+The clock row shows the split: workload events carry a harness timestamp, and
+the session-created/disposed records use the observer wall clock with the
+declared 50 ms uncertainty. That mix is the measured timing uncertainty; the
+maximum is the observer's declared bound for those two lifecycle records and does
+not vary with the stream.
+
+## What it deliberately does not measure
+
+- Cordis hook dispatch or DSH scheduling;
+- model, network, or tool latency;
+- a real session's worker outcome;
+- C1 eligibility or P0 exit.
+
+Those require a separately authorized live run and maintainer acceptance. See
+the [capability README](../../../loopx/capabilities/reliability_diagnostics/README.md)
+and the [RFC](../../../docs/architecture/rfcs/long-running-agent-reliability-diagnostics-governed-delivery-v0.md).
+
+## Test
+
+```bash
+pnpm test -- observer-overhead
+```
+
+The test covers the event-plan shape, percentile boundaries, the shipped-writer
+append path and byte split, loss, append-failure, and planned-record-mix
+marking, CLI parsing, and public-safe report rendering.

packages/dsh-loopx-plugin/bench/observer-overhead.ts

@@ -0,0 +1,162 @@
+import { mkdir, writeFile } from 'node:fs/promises'
+import { cpus, totalmem } from 'node:os'
+import { dirname } from 'node:path'
+import { pathToFileURL } from 'node:url'
+import { runBaselineTrial, runObserverTrial, normalizeEventCount } from './overhead-runner.ts'
+import type { TrialMeasurement } from './overhead-runner.ts'
+import { buildOverheadReport, renderMarkdown } from './overhead-report.ts'
+
+export interface CliOptions {
+  readonly events: number
+  readonly repeat: number
+  readonly bufferBound: number
+  readonly json: string | undefined
+  readonly markdown: string | undefined
+  readonly help: boolean
+}
+
+function positiveInteger(value: string | undefined, flag: string): number {
+  const parsed = Number.parseInt(value ?? '', 10)
+  if (!Number.isInteger(parsed) || parsed <= 0) {
+    throw new Error(`${flag} requires a positive integer`)
+  }
+  return parsed
+}
+
+export function parseArgs(args: readonly string[]): CliOptions {
+  let events = 5_000
+  let repeat = 3
+  let bufferBound = 4_096
+  let json: string | undefined
+  let markdown: string | undefined
+  let help = false
+  for (let index = 0; index < args.length; index += 1) {
+    const flag = args[index]
+    switch (flag) {
+      case '--help':
+      case '-h':
+        help = true
+        break
+      case '--events':
+        index += 1
+        events = positiveInteger(args[index], flag)
+        break
+      case '--repeat':
+        index += 1
+        repeat = positiveInteger(args[index], flag)
+        break
+      case '--buffer-bound':
+        index += 1
+        bufferBound = positiveInteger(args[index], flag)
+        break
+      case '--json':
+        index += 1
+        json = args[index]
+        if (json === undefined) throw new Error('--json requires a path or "-"')
+        break
+      case '--markdown':
+        index += 1
+        markdown = args[index]
+        if (markdown === undefined) throw new Error('--markdown requires a path or "-"')
+        break
+      default:
+        throw new Error(`unknown argument: ${flag ?? ''}`)
+    }
+  }
+  return { events, repeat, bufferBound, json, markdown, help }
+}
+
+function printUsage(): void {
+  process.stdout.write(`Usage: node --experimental-strip-types bench/observer-overhead.ts [options]
+
+Options:
+  --events <n>        synthetic session events per trial, normalized down to a
+                      whole 6-event turn cycle (default: 5000)
+  --repeat <n>        measured trials per arm (default: 3)
+  --buffer-bound <n>  observer buffer bound (default: 4096; pass 256 to mirror
+                      the observer's product default and exercise drops)
+  --json <path>       write the JSON report to path; "-" writes stdout
+  --markdown <path>   write the Markdown report to path; "-" writes stdout
+  --help              show this help
+`)
+}
+
+async function main(): Promise<void> {
+  const options = parseArgs(process.argv.slice(2))
+  if (options.help) {
+    printUsage()
+    return
+  }
+  const events = normalizeEventCount(options.events)
+  if (events !== options.events) {
+    process.stderr.write(
+      `note: normalized --events ${options.events} to ${events} (whole 6-event turn cycles)\n`,
+    )
+  }
+  const warmupEvents = Math.min(600, events)
+  runBaselineTrial({ inputEvents: warmupEvents, bufferBound: options.bufferBound })
+  await runObserverTrial({ inputEvents: warmupEvents, bufferBound: options.bufferBound })
+
+  const baseline: TrialMeasurement[] = []
+  const observer: TrialMeasurement[] = []
+  for (let index = 0; index < options.repeat; index += 1) {
+    baseline.push(runBaselineTrial({ inputEvents: events, bufferBound: options.bufferBound }))
+    observer.push(await runObserverTrial({ inputEvents: events, bufferBound: options.bufferBound }))
+  }
+
+  const report = buildOverheadReport({
+    config: {
+      events,
+      requested_events: options.events,
+      repeat: options.repeat,
+      buffer_bound: options.bufferBound,
+      event_mix: 'turn-cycle-v1',
+    },
+    baseline,
+    observer,
+    environment: {
+      node: process.version,
+      platform: process.platform,
+      arch: process.arch,
+      cpus: cpus().length,
+      total_memory_gib: Number((totalmem() / 1_024 ** 3).toFixed(1)),
+    },
+  })
+  const markdown = renderMarkdown(report)
+  const json = `${JSON.stringify(report, null, 2)}\n`
+
+  if (!report.usable) {
+    process.stderr.write('warning: this report is not a usable overhead sample\n')
+    for (const warning of report.warnings) process.stderr.write(`- ${warning}\n`)
+  }
+
+  let printed = false
+  if (options.json === '-') {
+    process.stdout.write(json)
+    printed = true
+  } else if (options.json !== undefined) {
+    await mkdir(dirname(options.json), { recursive: true })
+    await writeFile(options.json, json, 'utf8')
+    process.stderr.write(`wrote ${options.json}\n`)
+  }
+  if (options.markdown === '-') {
+    process.stdout.write(markdown)
+    printed = true
+  } else if (options.markdown !== undefined) {
+    await mkdir(dirname(options.markdown), { recursive: true })
+    await writeFile(options.markdown, markdown, 'utf8')
+    process.stderr.write(`wrote ${options.markdown}\n`)
+  }
+  if (!printed && options.markdown === undefined) {
+    process.stdout.write(markdown)
+  }
+}
+
+if (process.argv[1] !== undefined && import.meta.url === pathToFileURL(process.argv[1]).href) {
+  try {
+    await main()
+  } catch (error: unknown) {
+    process.stderr.write(`${error instanceof Error ? error.message : String(error)}\n`)
+    process.exitCode = 1
+  }
+}