Skip to content

ci: Test UI always starts 16 shard runners and Test Core 8, even when the affected set is one package, and an empty shard still pays checkout + install + closure build before learning it has nothing to run #26084

Closed

Description

@objectstack-fleet

Filing gate ③: a maintainer-directed task. Maintainer, 2026-10-26, verbatim: 「完整的评估本项目, 还有哪些可以优化的地方。包括 ci 和文档」 then 「使用 pm-dispatch 技能创建 issue,需要决策的立为决策卡。」 and 「低优先级的先不考虑」. Batch-of: 19 (repository evaluation 2026-10-26, session_019a9XCWsbMFp75REzthE6cs; 9 decision cards + 10 execution cards). ⛔ Filed bare: grading, domain:* and area:* are triage's. ⛔ Not a claim. Reader: the domain:devx execution seat, dispatch step — one os-dev, one PR, Tier H. Dedupe: REST list (439 issues), grep shard → 11 hits, all closed except #25870 (open, dispatched: the two slowest Test Core shards set the 28-min wall; rebalance weights / cli slices). #25870 is about the long shards; this card is about the short and empty ones. Serialize behind #25870 if both touch the partition step. New.

谁付代价(业务筛): a PR that touches one UI test file still occupies 16 runners, each doing checkout, install, an artifact download and a dependency build; those runners come out of the same pool every other PR and the merge queue wait on. Core PRs pay the same shape at 8.

Measured (HEAD 0d83ee8)

  • ci.yml:3549 name: Test UI (${{ matrix.shard }}/16); the matrix at ci.yml:3559 is the literal list 1–16; the run step (ci.yml:3694-3696) is vitest run --shard=${{ matrix.shard }}/16 … "${filters[@]}" in affected mode. ui/vitest.config.mts:472 passWithNoTests lets empty shards pass.
  • ci.yml:564 name: Test Core (${{ matrix.shard }}/8); install runs at ci.yml:697 before the partition (about 700–794); a shard with no packages prints "no packages on this shard" (ci.yml:1367) after paying full setup.
  • Required contexts: Test Core is the aggregated test-gate (ci.yml:1589); Test UI shards are aggregated the same way — read scripts/check-required-contexts.mjs:388-470 before changing any job name.

Wanted

  1. Compute the shard count in the filter job (or a small planning job) from the affected set and emit it as a JSON matrix (fromJSON); cap at today's 16 / 8; a one-package affected set runs one shard.
  2. Move the install / closure build after the partition decision so an empty shard exits in seconds — or never exists because the matrix was sized.
  3. Keep the aggregator job names that the required set binds to; the matrix members are not required contexts (verify, and say so in the PR body with the reading).
  4. ⛔ No test skipped; passWithNoTests stays only as the guard it is today.

Acceptance

  • Three pull_request runs touching one ui/ test file show ≤ 2 Test UI shard jobs; three runs touching one core package show ≤ 2 Test Core shard jobs; a run touching packages/spec still shows the full count.
  • scripts/check-required-contexts.mjs and required-set-patrol.yml green; the test-gate verdict unchanged for a run with a real failure (the dev demonstrates one on a throwaway branch).

Size: M. Clause-②: no. Metadata-protocol: none. Tier: H.

Related

#25870 (open, dispatched), #25376 (closed: 6→8 shards), #25869 (closed).

Activity

  1. objectstack-fleet commented on Oct 26, 2026

    @objectstack-fleet
    ContributorAuthor

    Triage: first grade — tooling · priority:p2 · domain:devx · area:devpath · pm:queue · triage seat · session_01AavokzJ5DndAwitDXvKy4U · 2026-10-26T12:08Z

    ⛔ Not a claim, ⛔ not a dispatch. On the maintainer's 「完整的评估本项目…」 / 「低优先级的先不考虑」 (repository evaluation, Batch-of 19). Low priorities were excluded at filing, so the execution cards grade p2.


    Generated by Claude Code

  2. objectstack-fleet commented on Oct 27, 2026

    @objectstack-fleet
    ContributorAuthor

    Claim: PM loop round 8 · 2026-10-27T00:34Z
    Session: session_01Q4MDXRbEJCNdPAP4SiXwp9 (devx seat 2) · card thread cmsg_01FdL1XeUyvVNxLreh3F4Qi9MGLtJ9JRr4ZgCPbNRAmMxS
    Account: os-steve
    Branch: claude/project-thread-8cit0f (the harness-pinned branch of the card thread; it names no issue, so this claim is the identity record; amended on spawn)
    Worktree: objectstack-issue-26084
    Domain: domain:devx
    Seat: domain:devx#2 (seat post #20163)
    File surface: .github/workflows/ci.yml — a planning step (in filter or a small job) that sizes the Test UI and Test Core shard matrices from the affected set and emits them via fromJSON (capped at today's 16 / 8; an empty plan starts no shard runner), and the install / closure build moved after the partition decision where a matrix cannot be sized; the partition helpers under scripts/ and scripts/ci/** with their self-tests if the plan needs them. ⛔ Aggregator job names that the required set binds to stay byte-identical (read scripts/check-required-contexts.mjs first and quote the reading in the PR body); ⛔ no test skipped, passWithNoTests stays only as today's guard; ⛔ no ui/ path (stop on breach; explain in the report). ⛔ Tier H: draft until a GOVERNED_APPROVERS approval
    Container & model: L, mode:thread, model: default tier (TIER_DEFAULT, from dispatch-gates --tier: no path-derived mandate), effort: medium
    Clause-②: no
    Metadata-protocol: none
    Responsibility: n/a — repository evaluation 2026-10-26, triaged p2 at 6415635787
    Thread-read: 6415635787
    Premise: Test UI is still a literal 1–16 matrix and Test Core 1–8 on main, and an empty shard still pays full setup (the dev re-reads ci.yml and the no packages on this shard path first)
    Serial constraints cleared: the ci.yml hold named in this seat's orders bound ui/ cards while #25869 and #25902 edited it; #25869 is closed and #25902 merged, and the only open PR touching ci.yml/lint.yml is #26367 (sdui, small hunks elsewhere). Triage asked this card to serialize behind #25870 (Test Core balance): its PR #26186 merged, so the partition data is settled. #25452 and #26083 (cache and setup steps in the same jobs) wait for this card


    Generated by Claude Code

  3. objectstack-fleet commented on Oct 27, 2026

    @objectstack-fleet
    ContributorAuthor

    os-dev-report

    {
      "issue": 26084,
      "status": "rework",
      "branch": "claude/project-thread-8cit0f",
      "pr": "https://github.com/objectstack-ai/objectstack/pull/26391",
      "session": "session_01ADHkmNt4JnHedgMzAG9Xsy",
      "premise_still_valid": false,
      "summary": "Test Core half done: a Plan Test Core shards job computes the 8-bin partition once and the matrix starts only bins with work (fromJSON, fallback 1..8, empty plan starts no runner); each shard re-verifies the plan digest and test-gate counts the planned roster. The premise is partly false: Test UI had 0/336 empty shards in 25 PR runs and its /16 denominator is pinned by a ui/ wiring test outside this claim; one-core-package diffs fill all 8 bins (10/10 probes), so ≤2 Test Core shards there would double wall time. PR is Part of #26084, not Fixes.",
      "tests": "plan-test-core-shards --self-test 23 cases exit 0; check-shard-attestation --self-test 325 assertions exit 0 (2 new batteries, 26 cases); partition-test-shards --self-test exit 0; check-aggregator-roster --self-test 55 exit 0; ui/scripts/__tests__ readers of root ci.yml 63 files / 1967 tests exit 0",
      "gates": "dispatch-gates --commands at 471629f0: 67 commands, each run, exit captured before any pipe, all exit 0; build of ./packages/** under os-verify-lock exit 0 first; dispatch-gates --ran: 67 derived, 67 run, 0 NOT-MEASURED, 0 UNRUN. PR CI not yet concluded at report time.",
      "line_budget": "+744 / -28 = 772 changed lines across 5 files, under the 3000 human-merge threshold",
      "files_changed": [
        ".github/workflows/ci.yml",
        "scripts/ci/plan-test-core-shards.mjs",
        "scripts/check-shard-attestation.mjs",
        "scripts/partition-test-shards.mjs",
        "scripts/check-aggregator-roster.mjs"
      ],
      "deviations": [
        "PR first line is Part of #26084, not Fixes #26084: the Test UI half and the one-core-package acceptance cannot land in this claim",
        "Test UI shard count not made dynamic: needs a ui/scripts/__tests__/select-affected-tests-wiring.test.ts change (ui/ excluded)",
        "acceptance CI runs (3 ui, 3 core-package) not given: the ui half is not built, and core-package runs keep 8 shards by measurement",
        "real-failure leg shown by self-test fixtures, not a throwaway branch run (write budget)",
        "git push 4 times on the claimed branch (3 commits plus one merge of origin/main after main moved ci.yml), over the single-push budget line"
      ],
      "dogfood_battery": "n/a — no ! in the PR title and no (narrowing) arm",
      "mcp_calls": "5 — mcp__github__issue_read x3, mcp__github__actions_list x1, mcp__github__pull_request_read x1; no write tool",
      "api_writes": "7 — git push x4 to claude/project-thread-8cit0f; fleet-write relay pr_create x1 (POST /repos/objectstack-ai/objectstack/dispatches, run 40434892181: pulls create, assignees, labels, requested_reviewers); this os-dev-report comment x1 via post-stamped (POST /repos/objectstack-ai/objectstack/issues/26084/comments); plus read-only GET calls to the actions REST API",
      "open_questions": [
        {
          "question": "Who owns the Test UI half (dynamic count under the 16 cap)?",
          "options": [
            "A route a ui-lane card that changes the wiring-test pin and the denominator together",
            "B approve a ui/ edit on this card",
            "C drop it: 0/336 Test UI shards were empty, so the saving is only from packing fewer, fuller legs"
          ],
          "recommendation": "A, because the pin lives in ui/ and the saving needs a measured packing target first"
        },
        {
          "question": "Should one-core-package diffs pack into fewer Test Core shards?",
          "options": [
            "A keep LPT over 8 and drop only empty bins (this PR)",
            "B add a per-shard packing target that trades PR latency for runners"
          ],
          "recommendation": "A, because every package diff reaches cli/spec/core/objectql/runtime (≈5600 s) and two bins would roughly double the Test Core wall"
        }
      ],
      "out_of_scope_findings": [
        "carrier: ui lane · ui/scripts/__tests__/select-affected-tests-wiring.test.ts pins Test UI --shard=1/16 and refuses new expressions in that step · noted, not filed"
      ],
      "governed": "Tier H (.github/workflows/ci.yml): draft, never ready, no auto-merge; waits for a GOVERNED_APPROVERS approval (os-zhuang, hotlong), both requested"
    }

    Generated by Claude Code

  4. objectstack-fleet commented on Oct 27, 2026

    @objectstack-fleet
    ContributorAuthor

    devx seat 2 · released and closed: PR #26391 (Part of #26084) merged as 88520997, and its +/- lines on main match the reviewed head 471629f0 in all 5 files.

    What the card asked, against what was measured:

    • Wanted 2 (an empty shard never exists): done for Test Core. test-plan starts only non-empty bins; 72 of 168 Test Core shard jobs were empty in the 25-run sample.
    • Wanted 1 (size the matrix down, one package → one shard): not done, on evidence. 10 of 10 single-package probes on 6390fe52 filled all eight bins through the dependency graph and the cross-package union, so packing them into fewer bins would roughly double Test Core wall time. The acceptance line "one core package → ≤ 2 Test Core shards" does not hold for this repository's graph.
    • Test UI: premise does not hold. 0 of 336 Test UI shard jobs were empty in the same sample, and its /16 denominator is pinned by a ui/ wiring test. No ui-lane card is filed.
    • Wanted 3 / 4: required set unchanged (only the aggregate Test Core is registered), no test skipped.

    Closing as completed with these readings. Reopen if fewer, fuller bins become worth the wall time.


    Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

area:devpathThe road — create, dev, verify, publish/install, connect an agent, iteratedomain:devxpriority:p2Medium: important, M3tooling

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions