Skip to content

test(driver-sqlite-wasm): run the persistence suites' temp files on /dev/shm, so a flush's fsync stops waiting on the shard's other suites - #27245

Merged
objectstack-fleet[bot] merged 1 commit into
mainfrom
claude/project-thread-9j22mo
Oct 28, 2026
Merged

objectstack-fleet[bot] merged 1 commit into
mainfrom
claude/project-thread-9j22mo

Conversation

@objectstack-fleet

Copy link
Copy Markdown
Contributor

Requested by Leehom · project thread

Fixes #27076

Clause-②: no
Metadata-protocol: none

维护者速读

  • 改了什么:driver-sqlite-wasm 这个包的 vitest 配置把测试进程的 TMPDIR 指到可写的 /dev/shm(内存盘);机器上没有 /dev/shm(macOS、Windows)就保持原样。只动了 packages/drivers/driver-sqlite-wasm/vitest.config.ts 一个文件,产品代码、用例、断言和 5 s 预算都没动。
  • 为什么改:全量测试两次红(10-22 的 39853791549、今天 08:05Z 的 40654333991)都是这个包里写真实数据库文件的落盘用例超时。每次落盘最后都要 fsync,而 fsync 要排在同一台 CI 机器上其他测试套件的磁盘队列后面,所以本地 30 ms 的用例在分片上能拖到 5 s 以上。用例测的是驱动的落盘语义,不是邻居的磁盘有多忙。
  • 风险与回滚:低。代码路径完全一样(sync() 照样调用),只是临时文件落在内存盘上;不发布任何东西(vitest.config.ts 不在包的 files[] 里)。回滚即 revert。
  • 先看:packages/drivers/driver-sqlite-wasm/vitest.config.ts(唯一改动,头部注释写了测量)。
  • 两分钟验证:cd packages/drivers/driver-sqlite-wasm && pnpm exec vitest run,预期 39 个文件 746 条全过;在用例里打印 os.tmpdir() 会看到 /dev/shm。
  • 需要你做的:无,席位落地。下一次 4 小时全量测试转绿即关卡。

What changes

packages/drivers/driver-sqlite-wasm/vitest.config.ts used to be the bare re-export of vitest.preset.mts. It is now a defineConfig with two keys:

  • disableConsoleIntercept: true, restated from the preset: check-console-intercept-disarm.mjs judges the preset in a config's place only for the bare re-export, so a config with its own key carries the disarm itself (node scripts/check-console-intercept-disarm.mjs → OK: 77 vitest-running package(s)).
  • env: { TMPDIR: '/dev/shm' } when /dev/shm is writable, {} otherwise. Every test file in the package reaches its temp directory through os.tmpdir() (mkdtempSync(join(tmpdir(), …)), 9 files), and os.tmpdir() reads TMPDIR on each call, so no test file changes.

Every other key stays on vitest's defaults, as under the re-export.

Why the cases were slow (the cause)

The failing case, read from the full job log by the maintainer (card comment 6468136240): SqliteWasmDriver (file persistence) > should persist data on-write, 5,026 ms, Test timed out in 5000ms, on Test Core (6/8); its sibling on-disconnect took 4,236 ms. Run 39853791549 timed out five cases, all on on-write or flush paths (background-flush-failure ×3, returning-persist UPDATE … RETURNING, durability atomic write).

Each of those flushes ends in WasmSqliteConnection.atomicWriteFile: write a sibling temp file, handle.sync(), rename. The case itself is about 30 ms on an idle disk (step probe: create 3, insert 1, flush 4, reopen + find 3, raw 8 KiB sync() 1 ms). The time goes to sync() waiting behind other processes' writes on the same disk:

Disk state (4-core box; the 4 persistence files, 29 cases) on-write case Slowest case Cases over 1 s Timeouts
idle, TMPDIR on the disk (2 runs, base 24351cf) 15–39 ms 0.26 s 0 0
4 dd … conv=fsync writers, TMPDIR on the disk (3 runs) 1,388–3,630 ms 3.6 s 7–11 per run 0
4 writers, TMPDIR=/dev/shm (3 runs) 12–28 ms 0.54 s 0 0

The same set of cases that went red on CI is the set that inflates under disk load, and only those. CPU load alone does not do it: 8 runs of the whole suite beside 3 busy-loop processes kept every case under 0.6 s.

Why this is the test's correctness, not a budget

A case whose wall time is set by unrelated suites' disk I/O measures the runner, not the driver. On /dev/shm the same code runs, sync() and rename() included, and every assertion (bytes on disk, atomic rename with no leftover temp file, the serialized flush chain, the failure report and retry) is unchanged. The triage ruling holds: no budget raised, no case skipped, no fixture shrunk, no retry wrapper.

Tests

Red on the base config, green on this head: 8 fsync writers, alternated, 3 rounds (head 2c4bfce)

Base leg: node scripts/ablation-replace.mjs replaced the env: line with env: {} (the pre-change behaviour: anchor 1 → 0, blob 08fbf987a6e2 → ea1e118ddf13; a probe case printed TMPDIR_SEEN=/tmp under it and TMPDIR_SEEN=/dev/shm on head), ran the four persistence files, then restored (ok restored: blob == HEAD (08fbf987a6e2) and git diff HEAD is empty, checked after every round). Head leg: the committed config. All legs under 8 dd if=/dev/zero bs=1M count=512 conv=fsync writers on the same disk, inside one os-verify-lock hold (VERDICT command-exit 0, 236 s).

base-1  Tests 2 failed | 27 passed (29)   returning-persist UPDATE … RETURNING 6061 ms timeout; returning-persist transaction 7227 ms timeout
head-1  Tests 29 passed (29)              slowest 178 ms
base-2  Tests 2 failed | 27 passed (29)   returning-persist INSERT … RETURNING 6641 ms timeout; driver.test.ts "should persist data on-write" 5358 ms timeout
head-2  Tests 29 passed (29)              slowest 223 ms
base-3  Tests 1 failed | 28 passed (29)   returning-persist UPDATE … RETURNING 6223 ms timeout
head-3  Tests 29 passed (29)              slowest 217 ms

Base round 2 reproduces the CI failure exactly (should persist data on-write, Test timed out in 5000ms).

Whole package, head 2c4bfce

pnpm --filter @objectstack/driver-sqlite-wasm test under os-verify-lock: VERDICT command-exit 0, Test Files 39 passed (39), Tests 746 passed (746). /dev/shm holds no leftover after the run (the suites clean their mkdtemp directories as before).

What was tried before the case was named (base 24351cf)

Alone ×2, beside 3 busy-loop CPU burners ×8, and the shard's real whole-package leg (the 10 packages of Test Core (6/8), turbo run test test:repo --concurrency=4, OS_TEST_TIERS=queue) ×3: all 746/746 green. Those loads were CPU-heavy, not fsync-heavy, which is why none reproduced it.

Acceptance notes

  • /dev/shm is writable in the cloud container these readings come from; I did not read a GitHub runner's mounts, so the shard's next run is the proof that it is used there. A host without a writable /dev/shm keeps its own temp directory, so the config never errors and never changes behaviour there.
  • Other packages that fsync real files in tests were not surveyed; this PR is scoped to the package the card names.

Verification

Local, head 2c4bfcec, change set packages/drivers/driver-sqlite-wasm/vitest.config.ts only.

  • node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack derived 42 commands; each run with its exit code captured before any pipe; --ran reconciliation: 42 derived famil(ies) accounted for — 40 run, 2 NOT-MEASURED.
  • NOT MEASURED, reason: pnpm check:dual-build-cjs-loads and pnpm check:lean-entry-closure exit 3 (PREREQUISITE NOT MET: they need a full workspace build, which this card thread did not run). CI runs both.
  • node scripts/check-plugin-teardown-shape.mjs --self-test exited 3 on the shallow clone, then 0 after git fetch --depth=1 of its positive-control commit (48 cases pass).
  • Outside the derivation: node scripts/check-console-intercept-disarm.mjs → exit 0 (OK: 77 vitest-running package(s)); pnpm exec eslint --no-inline-config packages/drivers/driver-sqlite-wasm → exit 0; pnpm --filter @objectstack/driver-sqlite-wasm typecheck → exit 0; node scripts/check-commit-card-trailers.mjs --range origin/main..HEAD → exit 0.
  • Changeset: none. vitest.config.ts is not in the package's files[] (dist, README.md, CHANGELOG.md), so nothing publishes: skip-changeset.

Human Minimum

  • 1. Schemas start as Zod; packages/spec holds no runtime logic. (not touched)
  • 2. Every name I added uses the casing for its kind.
  • 3. No formatter ran; pnpm lint passes. (eslint scoped to the package, in Verification above)
  • 4. One issue, one branch off an up-to-date main, one pull request.
  • 5. A change that publishes carries a changeset, and both declaration lines are filled in here and in the changeset. (publishes nothing: skip-changeset)
  • 6. packages/spec touched: its generated artifacts are regenerated and committed. (not touched)
  • 7. No hand edits to content/docs/references/, packages/*/CHANGELOG.md or content/docs/releases/.
  • 8. Title, body and comments are in English. (the 维护者速读 block is the dispatch protocol's Chinese summary)
  • The Before Submitting list in CONTRIBUTING.md is done.

🤖 Generated with Claude Code

https://claude.ai/code/session_01FEkQ5oKUikM4XcQpd1HJ6U


Generated by Claude Code

…dev/shm, so a flush's fsync stops waiting on the shard's other suites

The on-write and flush cases timed out on Test Core shards because each
flush's fsync queued behind unrelated suites' disk I/O. Point TMPDIR at a
writable /dev/shm in this package's vitest config; the code path, the
assertions and the 5 s budget are unchanged.

Co-authored-by: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FEkQ5oKUikM4XcQpd1HJ6U
@objectstack-fleet objectstack-fleet Bot added the skip-changeset PR has no user-facing published change; bypasses the changeset gate label Oct 28, 2026
@github-actions

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

2 anchor(s) derived from 1 changed package(s); no hand-written page names any of them, so this run has nothing to list — not a clean bill of health. This check sees only pages that NAME a derived anchor: one that documents this change in prose, or enumerates it in an authoring dialect, names none and stays invisible to it on every run.

What this run could not see
  • the SDK route bridge reached 56 of 208 client-bound route-ledger rows — the other 152 have no registrar path: tail to select them, so pages documenting THEIR client methods cannot appear above, on this or any run. Of those 152: 0 are remediable by widening that discovery convention (an in-repo file declares the path; the convention did not scan it); 55 are structural — on a ledger where NOT ONE row is declared in-repo, so no discovery change reaches them at any price; 97 are undecided (no in-repo declaration, on a ledger that has other in-repo registrars — absence and an unreadable spelling are not distinguishable here). The rows themselves: node scripts/docs-audit/affected-docs.mjs --bridge-coverage
  • a page that states a rule by its inputs shares no identifier with the emitter that implements the rule, so an emitter-only diff cannot list it — not on this run and not on any run. Measured on fix(driver-sql): emit varchar(maxLength) for a text field a declared index keys on #11430: content/docs/protocol/objectql/types.mdx documents the text-family column mapping by the ObjectQL type names it maps FROM (text / textarea / html) while the diff changed createColumn; it went unlisted, and it was the page that diff falsified, in four places. No shared token exists to detect this on, so a rule your change carries has to be re-read by hand in the pages that restate it.
  • a key NAME is not a key, so the hand re-read the line above prescribes can land on the wrong schema. The same spelling is authorable on one governed type and a [REMOVED] tombstone on another for each of active, aria, joins, objects, template, tools and version (censused on [finding] tools is a key on BOTH AgentSchema (tombstoned, dead) and SkillSchema (live, cloud-attested), so a name-based search attributes skill examples to the agent key — it produced a false stop-the-line alarm on PR #19059 #19093 over the liveness ledger's governed types, top-level keys); nothing in a search result distinguishes the two, so a grep hit on a LIVE example reads as evidence about the DEAD key. Measured on fix(spec): the agent.tools liveness row says dead — it claimed live on a key the schema tombstoned #19059: content/docs/ai/agents.mdx was reported as contradicting the agent.tools tombstone over its tools: example at :161, which is inside the defineSkill({ block opened at :155 — the page was already correct. Settle ownership by PARSING the value against both schemas, never by the name: that literal PASSES SkillSchema, and as an AgentSchema it FAILS at tools with the tombstone prescription. ⛔ These names are not the whole class — a key retired through a .strict() guidance map leaves no tombstone in the walked shape and none of them here (tool.category, live as AIToolDefinition.category).

Coarse fallback — 7 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json fbe344b3c537d944028e95e67aac2c81a2ae8426 → packageMentionDocs.

Which tree this was computed on

This run read content/docs from b6845716385e11d2620f5be034837d68ba5e9afb — the merge of head 2c4bfcec4122d5a1a77c8e3bf91805c15cc14062 into base fbe344b3c537d944028e95e67aac2c81a2ae8426, which is what actions/checkout gives a pull_request run. Not the PR head.

A worktree cut from an older main holds a different content/docs, so re-deriving there can legitimately return a different list — that is a different tree, not a wrong row. To answer on the same tree:

# while this PR is open — GitHub drops the merge commit once it closes
git fetch origin b6845716385e11d2620f5be034837d68ba5e9afb && git checkout b6845716385e11d2620f5be034837d68ba5e9afb
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin fbe344b3c537d944028e95e67aac2c81a2ae8426 2c4bfcec4122d5a1a77c8e3bf91805c15cc14062 && git checkout -B drift-repro fbe344b3c537d944028e95e67aac2c81a2ae8426 && git merge --no-ff 2c4bfcec4122d5a1a77c8e3bf91805c15cc14062

node scripts/docs-audit/affected-docs.mjs --json fbe344b3c537d944028e95e67aac2c81a2ae8426

⚠️ That checkout carried uncommitted changes, so the commit above does not fully identify what was read.

@objectstack-fleet

Copy link
Copy Markdown
Contributor Author

席位核对

ACCEPT · domain:server seat 2 · os-project-manager · session_0186vfjHHMsRXC9EUT9hTFMo · 2026-10-28T12:49Z · head 2c4bfcec · #27076

The maintainer's standing authorisation for landing reviewed PRs, verbatim: 「按照 pm-dispatch 规则复核过的都授权」. The maintainer pointed the dev thread at the failing test themselves (6468136240).

  1. Shape:
    • A draft against main, assignee os-project-manager, with skip-changeset. The diff is test configuration only, so no package publishes.
    • It says Fixes #27076.
    • check-governed-merges --pr 27245 reads NOT governed, 56 changed lines.
    • The claim (6465085494) names this PR's branch.
  2. Against the order:
    • The failing test is identified. driver-sqlite-wasm "SqliteWasmDriver (file persistence) > should persist data on-write" hit 5,026 ms on run 40654333991 (Test Core 6/8). Run 39853791549 timed out five on-write and flush cases the same way.
    • The cause is measured. Every on-write flush ends in atomicWriteFile's fsync, which queues behind the other suites' disk I/O on the shard.
    • Red first, under load. With 8 dd conv=fsync writers, base and head were alternated 3 times. The base had 5 timeouts, including the CI case at 5,358 ms. The head had 0 failures, and its slowest case took 223 ms.
    • The fix is the test's own correctness, not its budget. vitest.config.ts points TMPDIR at /dev/shm when it is writable, and leaves it unchanged otherwise. The same driver code still runs, sync() included, and every assertion is unchanged.
      • The order's ⛔ lines hold: no raised timeout, no skip, no smaller fixture, no retry wrapper, and no product change.
      • The preset's one key, disableConsoleIntercept, is restated, so check-console-intercept-disarm still judges this config.
  3. Contract review: not owed. There is no packages/spec/src/** path, no governed path and no Clause-② arm, and the diff is test-only.
  4. Gates:
    • Locally, 40 of 42 derived families exit 0. The 2 NOT-MEASURED (dual-build-cjs-loads, lean-entry-closure) need a full build, and CI ran them.
    • CI on head 2c4bfcec: 34 success, 19 skipped, 0 red. Test Core 1–8 are all green, including 6/8. The state is clean.
  5. Overlap: no other open PR touches packages/drivers/driver-sqlite-wasm/**. git merge-tree reports no conflict against main 9ff8154c.
  6. Proof owed after landing: the order closes this card on a green 4-hourly run. The landing record will name the first schedule run after the merge, and if that run is red in this package, the card reopens.

Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/s skip-changeset PR has no user-facing published change; bypasses the changeset gate tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4-hourly full run: red on main (CI)

2 participants