Skip to content

Tests grow 2.5 times faster than source: measure the per-fix test weight and the #25911 pilot, then roll the shared-setup merge out to the heaviest components or retire it #29469

Open

Description

@objectstack-fleet

director#1 · os-support-ai · session_01WGufzg12BtjFc5rHp7YD1J

Ruled: 6583487846 · Q1 A · Q2 A · 2026-11-02T17:17Z

Filing gate ③, a task the maintainer set. After the director's project assessment reported test lines growing 2.5 times faster than source lines, the maintainer answered 「同意」 to filing this card. ⛔ Not a claim. Reader who acts: the domain:devx seat. Follows #25911 (closed), whose own plan said 「先量 setup 与断言的占比再决定推广」.

Measured (origin/main, 2026-10-26 → 11-02, git log --numstat over packages and ui)

class of file added removed net
test files (.test., .spec., __tests__, test/, e2e) 352,859 43,250 +309,609
source 236,290 114,716 +121,574
generated, JSON, fixtures 8,155 3,789 +4,366

Ask

  1. Measure, on the last 7 days' merged fix( PRs: test lines added per source line changed (median and p90), and how many new test files each adds.
  2. Measure the tests: 56.5% of spec files are per-issue pins costing 52.5% of suite time — pilot a shared-setup consolidation on the 95 StudioDesignSurface files and split cli-bin.test.ts (74.6 s) #25911 pilot: suite time of the folded StudioDesignSurface.* files before and after.
  3. Report both on this card. If the pilot saved time without losing a pin, roll the shared-setup merge out to the next four components tests: 56.5% of spec files are per-issue pins costing 52.5% of suite time — pilot a shared-setup consolidation on the 95 StudioDesignSurface files and split cli-bin.test.ts (74.6 s) #25911 named (ListView*, ObjectView*, RecordDetailView*, ObjectGantt*), one PR each; if it did not, close the pattern and say so.
  4. ⛔ No new gate and no ratio ceiling in CI: a guidance line in the test discipline of AGENTS.md is the most this card may propose, and it goes to the maintainer as a proposal, not a change.

Done when

The two measurements are on this card with their commands, and either the four rollout PRs have merged or the card records why the pattern stops.

Activity

  1. objectstack-fleet commented on Nov 2, 2026

    @objectstack-fleet
    ContributorAuthor

    triage#1 · os-zhuang · session_01AavokzJ5DndAwitDXvKy4U

    Path: none

    Triage: p2 · domain:devx · pm:queue · 2026-11-02T15:57Z

    ⛔ Not a claim.


    Generated by Claude Code

  2. objectstack-fleet commented on Nov 2, 2026

    @objectstack-fleet
    ContributorAuthor

    director#1 · os-support-ai · session_01WGufzg12BtjFc5rHp7YD1J

    Claim: maintainer-directed dispatch · director seat · 2026-11-02T16:30Z
    Session: session_01WGufzg12BtjFc5rHp7YD1J
    Account: os-support-ai (the seat's linked user as GET /user answers it; the card's assignee)
    Branch: claude/issue-29469-test-weight-census
    Worktree: objectstack-issue-29469
    Domain: domain:devx
    Seat: director#1 (the director seat dispatching under the maintainer's direct order; it holds no domain:devx seat)
    File surface: none in phase 1 — measurement only, results reported as a comment on this card; ⛔ no source, test or workflow edit in this dispatch (stop on breach; explain in the report)
    Container & model: S (measurement, no build of the product), mode:subagent, model: default tier (dispatch-gates.mjs --tier: 「The tier stays the PM's per-card judgment call (floor sonnet · default opus · ceiling fable)」; no path-derived mandate, no file surface), effort: high
    Clause-②: no
    Metadata-protocol: none
    Responsibility: n/a — not a defect card
    Thread-read: 6581713880
    Premise: holds (the card's figures were measured by this seat at 2026-11-02T15:20Z on origin/main; the dev re-measures them as part of the work)
    Serial constraints cleared: none — the dispatch edits no file

    Provenance (three pieces): the maintainer, in the director session (session_01WGufzg12BtjFc5rHp7YD1J, claude.ai), verbatim: 「普查测试的卡你现在就可以派发」, after 「同意」 to filing it. The director seat reads 「普查测试的卡」 as this card. The pm-dispatch rule that the director seat never claims backlog yields to the maintainer's order (SKILL.md 〈优先级〉: 维护者裁决 > … > 红线). Triage's first touch 6581713880 (p2, domain:devx, pm:queue) is respected; pm:queue → pm:dispatched in one label write, read back.

    Scope of this dispatch: the card's Ask items 1 and 2 (the two measurements) and item 3's verdict, reported on this card. The rollout PRs that item 3 may call for are dispatched afterwards, one per component, by the lane that owns the files (ui/** → domain:ui), each its own claim.

  3. objectstack-fleet commented on Nov 2, 2026

    @objectstack-fleet
    ContributorAuthor

    director#1 · os-support-ai · session_01WGufzg12BtjFc5rHp7YD1J

    Results: per-fix test weight, the #25911 pilot, and the verdict (Ask 1, 2, 3)

    Read at 2026-11-02T16:37Z on objectstack-ai/objectstack at 94d3e2543 (the origin/main tip, committed 2026-11-02T16:10:12Z). Window: first-parent commits on that tip since 2026-10-26T15:20:00Z. That is 1,656 commits, all squash (0 merges). The history is proven present by node scripts/pm/git-history.mjs ensure --since=2026-10-20 (floor 2026-10-17, no fetch needed). Measurement only: no file, PR or label was written.

    0. The card's table, re-measured

    Same path regexes as the card, with the test class taking precedence over generated, over packages/ and ui/:

    class added removed net
    test 359,185 43,250 +315,935
    source (card's definition) 256,853 142,879 +113,974
    generated, JSON, fixtures 8,131 3,789 +4,342

    The removed column matches the card exactly. The added column is 2 % (test) and 9 % (source) higher. I did not chase this; the tip moved after the card was read.

    The classification moves the multiplier but not the conclusion. The card's "source" includes 22,029 added and 54,985 removed lines that are not code: CHANGELOG.md +16,484 (written by the release tooling), .mdx −23,284, .md −21,705, LICENSE +3,051, and .sh. With source limited to code files (.ts .tsx .js .jsx .mjs .mts .cjs .cts) it is +234,824 / −87,894 / net +146,930. Test against source is then 2.15× on net lines (the card says 2.5×; the raw table gives 2.77×) and 1.53× on added lines. Tests still outgrow source by about 2 to 1.

    1. Per-fix test weight (Ask 1)

    Population: the 816 first-parent commits in the window whose subject starts fix( (2 more start fix: or fix! and are not counted). Each file is classified as: test (card regex), generated (card regex), prose (.md, .mdx, .txt, LICENSE*, which includes .changeset/), or source (everything else, any path). Source lines changed = added + removed.

    reading median p90 other
    test lines added ÷ source lines changed (809 fixes with source over 0) 2.28 7.47 p75 4.07, mean 3.59; aggregate 189,425 ÷ 99,540 = 1.90
    test lines added per fix 196 476 total 189,425
    source lines changed per fix 78 266 total 99,540
    new test files per fix 1 2 876 files over 816 fixes, mean 1.07

    2. The pilot (Ask 2)

    Instruments and their radius

    reading before after
    the ten folded pins, CI (a) 10 files, 70.5 s not measured
    the ten folded pins, local (b) 10 files, 76.6 s CPU (38.1 + 38.5) 1 file, 13.8 s CPU (−82 %)
    StudioDesignSurface.* family, local (b) 95 files, 1,022.1 s CPU, 528.4 s wall 86 files, 954.9 s CPU, 493.8 s wall (−6.6 % CPU, −6.5 % wall)

    The CI before (70.5 s) and the local before (76.6 s) agree within 9 %, so the local scale holds. Each folded file saved about 6.5 s of fixed cost, and the folded file's own fixed cost stayed about 7.5 s.

    No pin lost (my own check, from git only). The ten original files, read at the parent of each fold (697a307a2^ and 3249e204a^), hold 50 it/test titles and 153 expect( calls. StudioDesignSurface.railLabels.test.tsx at 3249e204a holds the same 50 titles as an identical set, and 153 expect( calls. At the tip it holds 54 titles and 172 expect( calls, and 0 of the original titles are missing. The 4 extra titles came from later commits that appended to the shared file (for example #27039 and #28825).

    The cli-bin split (#26333) is a split, not a fold. The heaviest file went from 87.2 s to 57.5 s cold wall, the spawn CPU is unchanged, and CI CPU rises by one build when the two files land on different legs. It is not evidence for or against the shared-setup merge.

    Inflow after the pilot. StudioDesignSurface.* test files: 95 before, 86 after #26309, 116 at the tip. That is 30 new per-issue pin files in the 7 days since, and 0 deleted. At about 6.5 s of fixed cost each, that is roughly 195 s, about 3× the 67 s the fold saved.

    The four components #25911 named, counted by test-file basename COMPONENT.*.test.ts(x):

    family files at f143cd88 (pilot base) files at tip CI module time at 2026-10-19 (a)
    StudioDesignSurface 95 116 948.0 s
    ListView 110 128 615.7 s
    ObjectView 131 141 804.8 s
    RecordDetailView 70 99 402.0 s
    ObjectGantt 53 56 274.3 s

    3. Verdict (Ask 3): roll out, and land new pins in the shared file

    Roll out. The pilot cut its ten pins from 76.6 s to 13.8 s of CPU (the family by 6.6 %) and kept 50 of 50 titles and 153 of 153 expect( calls, which meets the card's bar.

    Folding alone does not keep up with the inflow. The same family gained 30 new pin files in the following week, about 3× the time the fold saved, and 78.8 % of fix test lines go into new files. A rollout PR pays off only if later pins for that component go into its shared file.

    So, under Ask 4 (a proposal to the maintainer, not a change, and no gate or ceiling): add one guidance line to the test discipline in AGENTS.md: "A new pin for a component that already has a shared harness file goes in as a top-level describe in that file, not as a new file." The four rollout PRs (ListView*, ObjectView*, RecordDetailView*, ObjectGantt*) are for the owning lane (domain:ui) to dispatch.

    The rollout has no CI after-reading until the per-file timings reporter and its refresh workflow are ported into ci.yml. Today each rollout PR can only cite a local A/B like #26309's.

    Exact commands (scripts are in the dev's scratchpad, read-only over git)
    # SINCE = the window start declared at the top of this comment
    git fetch origin main
    git worktree add --no-track ../objectstack-issue-29469 -b claude/issue-29469-test-weight-census origin/main
    node scripts/pm/git-history.mjs ensure --since=2026-10-20
    # table + per-fix: one numstat pass, parsed in Python (renames resolved to the new path, binary rows dropped)
    git log 94d3e2543 --first-parent --since="$SINCE" --numstat --format='@@%H%x09%cI%x09%s'
    # new files per commit
    git log 94d3e2543 --first-parent --since="$SINCE" --diff-filter=A --name-only --format='@@%H'
    # test regex:  \.test\.|\.spec\.|__tests__|/test/|/tests/|e2e
    # gen regex:   generated|\.snap|\.json|baseline|fixture   (test wins on overlap)
    # pilot, CI before: sha1(ui-relative path)[:16] looked up in ui/scripts/test-shard-timings.json .files
    # pilot, pins kept: git show REV:PATH for the 10 originals at 697a307a2^ / 3249e204a^ and the folded file at 3249e204a / 94d3e2543,
    #   counting it/test titles and expect( calls
    # inflow: git ls-tree -r --name-only REV ui/ | basename COMPONENT.*.test.ts(x), at f143cd88, 3249e204a, 94d3e2543
    git log 3249e204a..94d3e2543 --first-parent --diff-filter=A --name-only --format= -- 'ui/packages/app-shell/src/views/studio-design/StudioDesignSurface.*.test.tsx'
    
  4. objectstack-fleet commented on Nov 2, 2026

    @objectstack-fleet
    ContributorAuthor

    director#1 · os-support-ai · session_01WGufzg12BtjFc5rHp7YD1J
    os-dev-report

    {
    "issue": 29469,
    "status": "done",
    "branch": "claude/issue-29469-test-weight-census",
    "pr": null,
    "session": "session_01WGufzg12BtjFc5rHp7YD1J",
    "premise_still_valid": true,
    "summary": "Measurement only, results comment 6582607393 on #29469. Ask 1: 816 fix( commits; test lines added / source lines changed median 2.28, p90 7.47; new test files per fix median 1, p90 2 (69.7% add one; 78.8% of fix test lines land in new files). Ask 2: the ten folded pins went 76.6 s to 13.8 s CPU (local A/B from PR #26255/#26309 bodies; CI before 70.5 s from ui/scripts/test-shard-timings.json; CI after NOT MEASURED, per-file reporter not ported), family -6.6% CPU, 50/50 titles and 153/153 expect( kept. Verdict: roll out, with a proposed AGENTS.md guidance line that new pins land in the shared file, because StudioDesignSurface.* gained 30 pin files in the week after the fold (about 3x the saving). Card table re-measured: code-only net test:source is 2.15x, not 2.5x (card's source class carries CHANGELOG/md/mdx/LICENSE). Writes: 2 comments via post-stamped relay; mcp_calls 0.",
    "tests": "No product test run (no-build dispatch). Measurements: git log TIP --first-parent --since=SINCE --numstat parsed in Python at 94d3e25 (1,656 commits, 0 merges); history proven by git-history.mjs ensure --since=2026-10-20 (exit 0, floor 2026-10-17). Pin-kept check: 10 originals at 697a307^/3249e204a^ = 50 titles, 153 expect(; folded file at 3249e20 = same 50-title set, 153 expect(; at tip 54/172, 0 originals missing. CI-dataset keying control: cli-bin.test.ts = 74.12 s, matching the 74.1 s #26333 cites. No ablation: no tested behaviour changed.",
    "gates": "No diff, so no derived gate family and no PR check-runs; dispatch-gates.mjs not run (nothing to derive from). post-stamped dry-run exit 2 once (MIXED stamp refusal on a bare --since stamp in the commands block), fixed to a SINCE variable, dry-run exit 0, post exit 0 with byte-identical read-back (8668 bytes).",
    "dogfood_battery": "n/a — no PR, no ! title and no (narrowing) arm",
    "mcp_calls": "0",
    "api_writes": "2 — both through scripts/pm/post-stamped.mjs on the fleet-write relay (repository_dispatch then POST /repos//issues/29469/comments): the results comment 6582607393 and this os-dev-report comment. No git push (no edit, so no empty-branch probe), no PR, no label.",
    "open_questions": [
    {
    "question": "Accept the AGENTS.md test-discipline guidance line proposed under Ask 4 (a new pin for a component that already has a shared harness file goes in as a top-level describe in that file, not a new file)?",
    "options": [
    "A propose it to the maintainer as written, as a separate docs PR after the ruling",
    "B roll out the four folds without the line"
    ],
    "recommendation": "A, because the StudioDesignSurface family gained 30 pin files in the 7 days after the fold, about 3x the CPU the fold saved, so folds without the landing rule are outrun within a week."
    },
    {
    "question": "Port the ui per-file timings reporter and its refresh workflow into ci.yml so a rollout has a CI after-reading?",
    "options": [
    "A file it as a devx card before the rollout PRs",
    "B let each rollout PR cite a local A/B like #26309"
    ],
    "recommendation": "A, because no CI per-file reading has existed since the ui/ import, so neither the pilot nor the rollout can be checked in CI."
    }
    ],
    "security_findings": "none",
    "out_of_scope_findings": [
    "carrier: PM (director#1) · the ui per-file timings reporter was not ported with the ui/ import (ci.yml comment above Run the ui tests says so), so ui/scripts/test-shard-timings.json is frozen at its 2026-10-19 measurement; observation, noted and not filed · dedupe words: test-file-timings reporter, shard timings refresh, ui per-file timings, test-shard-timings.json stale"
    ]
    }

  5. objectstack-fleet commented on Nov 2, 2026

    @objectstack-fleet
    ContributorAuthor

    director#1 · os-support-ai · session_01WGufzg12BtjFc5rHp7YD1J

    Collected and accepted · measurement dispatch (claim 6582445213) · 2026-11-02T16:41Z

    • Verdict: ACCEPT. Results 6582607393 and report 6582630893 are on this card, both through the relay, read back. No PR, no file edit, mcp_calls 0: the dispatch's shape held.
    • Seat spot checks, own instrument: fix( first-parent subjects on origin/main 2026-10-26T15:20Z → 2026-11-02 15:20 UTC = 811 (the dev's 816 runs to the 16:10Z tip); net test lines ÷ net code-source lines (.ts/.tsx/.js/.mjs/.cjs/.jsx, generated and fixtures excluded) over packages + ui = 313,791 ÷ 145,358 = 2.16 (the dev: 2.15). The body's "2.5 times" counted CHANGELOG and prose lines as source; the corrected figure is about 2.15. The conclusion stands.
    • Item 3 verdict: roll out the shared-setup fold to ListView*, ObjectView*, RecordDetailView*, ObjectGantt* (pilot: the ten folded pins 76.6 s → 13.8 s CPU, local A/B; no title or expect( lost). The rollout PRs are domain:ui work and are not dispatched until the two questions below are ruled, because the first decides whether a fold stays folded.

    Two questions for the maintainer (the dev's open_questions, analysed by this seat)

    Q1 — a test-discipline line in AGENTS.md: 「a new pin for a component that already has a shared harness file goes in as a top-level describe in that file, not as a new file.」 Reading behind it: StudioDesignSurface.* went 95 → 86 files with the fold, then 116 at the tip — 30 new per-issue pin files in 7 days, about 3× what the fold saved. 78.8 % of fix test lines land in new files.

    • A add the line (one docs PR, governed text, your approval). B roll out without it.

    Q2 — the ui/ per-file timings reporter, not ported when ui/ was imported: ui/scripts/test-shard-timings.json has been frozen since 2026-10-19, so neither the pilot nor a rollout can be read in CI, and Test UI shard balancing runs on week-old data.

    四轴分析

    • 项目长远合理性:Q1 选 A。按组件共用一个测试文件、用 describe 分块,是 Jest/Vitest 生态的常规组织;不加这条规则,合并一次、一周又长回来,实测已是 3 倍。Q2 选 A。CI 自己测出的逐文件耗时,才是分片均衡和测试瘦身的长期依据;本地 A/B 每次换机器、换口径。
    • 实际业务拉动:有。修复 PR 中位数每改 1 行源码配 2.28 行测试,七成修复新开测试文件;Test Core 现在 12–21 分钟,增长会把它推回去。
    • 防 AI 犯错:Q1 是给 AI 读的约定,告诉它新 pin 写进哪里,AI 最常见的做法正是「每张卡一个新文件」。这类约定不属于「声明即强制」的范围,不需要门禁。Q2 不直接防错,但让回归可见。
    • 创业阶段不扩散:Q1 只加一行规则文字,不加门禁。Q2 恢复一个导入时丢掉的测量工具,不是新增门禁;代价是一次 workflow 改动和您一次批准。

    os-decision-facets

    自检:只看①选 Q1 A、Q2 A;②③④ 是否翻转:否。

    推荐:Q1 A,Q2 A。 Q1 B 不取:实测一周内长回 3 倍。Q2 B 不取:本地 A/B 不能复查,也不喂分片均衡。置信缺口: 报告器移植的工作量未测(原实现在 objectui 仓,导入时未带过来)。

    Analysis-tier: claude-opus-5-5 (served at analysis; not CONTRACT_REVIEW_TIER).

    State: the measurement claim is released. pm:dispatched → needs-user-decision in one label write, assignee cleared; this card is the anchor for the ruling. After the ruling: Q1 A → one docs PR in the skills lane quoting it; Q2 A → one domain:devx card; then the four rollout cards in domain:ui.

  6. objectstack-fleet commented on Nov 2, 2026

    @objectstack-fleet
    ContributorAuthor

    director#1 · os-support-ai · session_01WGufzg12BtjFc5rHp7YD1J

    Brief-claimed-by: session_01WGufzg12BtjFc5rHp7YD1J · batch #458 · 2026-11-02T17:11Z

    ⛔ Not a claim on the work. The two questions in 6582704041 are batch #458 item 1, presented in the director session.

  7. objectstack-fleet commented on Nov 2, 2026

    @objectstack-fleet
    ContributorAuthor

    director#1 · os-support-ai · session_01WGufzg12BtjFc5rHp7YD1J

    Ruling: batch #458 item 1 · Q1 letter A · Q2 letter A · maintainer 「同意」 2026-11-02T17:15Z

    • Record: the director presented the two questions of 6582704041 in the director session with A recommended for both; the maintainer answered 「同意」. Brief-claim 6583378464.
    • Q1 A: one line in the test discipline of AGENTS.md: a new pin for a component that already has a shared harness file goes in as a top-level describe in that file, not as a new file. Workload: domain:skills · 1 docs PR (governed text, the maintainer's approval) · no precedent.
    • Q2 A: port the ui/ per-file timings reporter and its dataset refresh into ci.yml, so CI can read the rollout. Workload: domain:devx · 1 PR touching .github/workflows/** (Tier H, the maintainer's approval) · effort unmeasured (the original lives in the objectui repository and was not carried over).
    • Then: the shared-setup fold rolls out to ListView*, ObjectView*, RecordDetailView*, ObjectGantt*, one PR each, domain:ui.
    • Exit stroke on this card (needs-user-decision → working state) and the Q2 card are the domain:devx seat's; the Q1 PR is the domain:skills seat's. The director files both cards in this summon and knocks both lanes.
  8. objectstack-fleet commented on Nov 2, 2026

    @objectstack-fleet
    ContributorAuthor

    domain:devx#1 · os-warren · session_01JQwxbaBjisxkQo9hht9bFB

    Exit stroke for ruling 6583487846 (Q1 A, Q2 A), done by the domain:devx seat the ruling names. needs-user-decision → pm:blocked in this stroke.

    Ruling-ref: 6583487846
    Blocked-by: #29517
    Blocked-by: #29516

    #29517 (Q2: port the Test UI per-file timings reporter and dataset refresh into ci.yml, domain:devx, Tier H) and #29516 (Q1: the shared-harness line in AGENTS.md, domain:skills, governed) are the two cards the director filed. This card stays open as the anchor for the rollout that follows them: the shared-setup fold to ListView*, ObjectView*, RecordDetailView*, ObjectGantt*, one PR each in domain:ui, read in CI through #29517's timings.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions