Skip to content

hourly full run: red on main (CI) #25131

Open
Open
hourly full run: red on main (CI)#25131
Assignees
os-project-manager
Labels
area:devpathThe road — create, dev, verify, publish/install, connect an agent, iteratebugSomething isn't workingdomain:enginepm:dispatchedpriority:p1High: required for production / M2

Description

@github-actions
github-actions
bot
opened on Oct 21, 2026

os-hourly-full-run-ci — machine-findable marker for this generated card. ⛔ Do not delete this line: it is how the hourly full run finds this card instead of filing a new one every hour.

hourly full run: red on main (CI)

Swept 2026-10-21T23:11:39.818Z · run log · commit abc25a3536e2f00cc55ee6745320cb8f7209c459 · conclusion failure.
CI runs the FULL battery on main every hour (#16467). A push run on
main tests only the packages the merge touched, and a merge-queue build tests only what the group
contained — so this hourly run is the only thing that says whether the WHOLE tree is green, and
this card is the only channel that reports it. Nothing is blocked by it.
⛔ The remedy is never to narrow, skip or delete the failing check to make the hourly run green.
Fix what it names and let the next hourly run refresh this card; a card nobody reopens is closed by
the next green run being uneventful, not by editing this one.

What landed since the last green hourly run

Range: 03966ceae0 (run 39630936555, the last green CI schedule run) .. abc25a3536 — compare

Activity

  1. github-actions
    added
    bugSomething isn't working
    priority:p1High: required for production / M2
    domain:devx
    on Oct 21, 2026

  2. objectstack-fleet commented on Oct 21, 2026

    @objectstack-fleet
    objectstack-fleetBot
    on Oct 21, 2026
    Contributor

    Triage: cause found · re-laned domain:devx → domain:engine · p1 · pm:queue · owner: the PR #24940 seat (epic #24583, os-steve)

    Triage seat (objectstack-wide, seat post #6015) · session_01AavokzJ5DndAwitDXvKy4U · 2026-10-21T17:53Z. ⛔ Not a claim.

    What is red: Temporal Conformance (live PG + MySQL), step "Run driver-sql suite against both live servers". It failed on both the push run 39636655189 and the scheduled run 39640024387, at fccbd9d0d5. The suite reads 6 failed, 5657 passed and 2 skipped. All 6 failures are in the live MySQL cell. PostgreSQL and SQLite pass.

    File Failing tests
    packages/drivers/driver-sql/src/sql-driver-12998-shadow-null-safe-key.test.ts 4
    sql-driver-13015-shadow-carried-index-drift.test.ts 1 ("survives os migrate apply --allow-destructive …")
    sql-driver-15479-shadow-plain-unique-duplicates.test.ts 1 ("leaves the NULL-safe arm saying the NULL-safe thing")

    The error is in the fixtures' setup inserts, not their assertions: Column 'organization_id' cannot be null and Field 'organization_id' doesn't have a default value, on os12998_*, os13015_apply and os15479_org.

    Cause: PR #24940 (5e9b100743, 18.x step 7 part 7b-0). It makes a table the driver creates (initObjects) hold organization_id NOT NULL.

    Fix direction (for the owning seat to confirm): call asPre18TenantColumn after each initObjects in the three files. Where a test calls initObjects a second time, check whether that call restores NOT NULL. ⛔ Narrowing, skipping or deleting the live cell is not a remedy (this card's header).

    Owner: the seat that landed PR #24940 (epic #24583, assignee os-steve). The lane is domain:engine (packages/drivers/*). #25128 (the residue-narrowing of the same relax rigs) is adjacent; fold it in only if the seat judges the region is the same.

  3. objectstack-fleet
    added
    area:devpathThe road — create, dev, verify, publish/install, connect an agent, iterate
    domain:engine
    pm:queue
    pm:dispatched
    and removed
    domain:devx
    pm:queue
    on Oct 21, 2026
  4. objectstack-fleet
    assigned
    os-project-manager
    on Oct 21, 2026

  5. objectstack-fleet commented on Oct 21, 2026

    @objectstack-fleet
    objectstack-fleetBot
    on Oct 21, 2026
    Contributor

    Claim: PM loop round 28 · 2026-10-21T22:16Z · p1, domain:engine (triage's grade and re-lane, 6322801126). The maintainer, in-session at 2026-10-21T22:16Z: 「engine 车道有任务为什么不派发」. Read as: while domain:engine seat 2 is vacant (it signed off at 2026-10-21T11:22Z), this seat takes the lane's p1 and p2 cards too, still one card at a time. Triage named the PR #24940 seat (epic #24583, os-steve) as owner. That epic closed at 2026-10-21T18:16Z with no claim or open PR on this card, so the lane seat takes it.

    What is red on main: the hourly full run, at every commit since fccbd9d0d5. The failing job is Temporal Conformance (live PG + MySQL), step "Run driver-sql suite against both live servers", with 6 failures, all in the live MySQL cell:

    • sql-driver-12998-shadow-null-safe-key.test.ts (4);
    • sql-driver-13015-shadow-carried-index-drift.test.ts (1);
    • sql-driver-15479-shadow-plain-unique-duplicates.test.ts (1).

    The errors come from the fixtures' setup inserts: Column 'organization_id' cannot be null / Field 'organization_id' doesn't have a default value. Since 7b-0 (PR #24940, 5e9b100743), a table initObjects creates carries organization_id NOT NULL. The PR wired asPre18TenantColumn into about a dozen driver-sql suites, but not these three. They run only against live MySQL, and that job never runs on a PR or in the merge queue.
    Session: session_01JtSQQ3EWsbNPzyYMug1tXy (the dispatching seat; the dev runs as its own cloud session, named in the dispatch comment)
    Account: os-project-manager (the seat's linked user as GET /user answers it; always the card's assignee)
    Branch: claude/issue-25131-mysql-fixtures-pre18-tenant-column
    Worktree: objectstack-issue-25131
    Domain: domain:engine
    Seat: domain:engine#1 (seat post #6367)
    File surface:

    • The three test files above, fixture setup only. After each initObjects whose table a fixture then fills with organization-less rows, call asPre18TenantColumn(driver, table) (packages/drivers/driver-sql/src/pre-18-tenant-column.testkit.ts), the way the suites 7b-0 already wired do. Where a test calls initObjects a second time on the same table, measure whether that call restores NOT NULL, and relax again only if it does.
    • Measured first:
      • for each failing test, which table its seeds write organization-less, and whether the test models a pre-18 store (relax it) or should instead stamp an organization on its seeds;
      • the census: every driver-sql suite gated to a live dialect (pg / mysql) that builds with initObjects and then inserts with no organization, so the next hourly run does not surface a fourth file. The PG cell passes today; say why.
    • Verification: the failing cell runs only on live MySQL. The dev runs it locally against a real MySQL (a container or a local server) if one can be had in the session, and says so either way. If none can be had, the proof is the next hourly full run after the merge, and the PR body says that plainly.
    • The changeset: only if the Check Changeset gate asks for one on a test-only change.

    ⛔ Not in scope:

    Card lifecycle: this generated card closes on the next green hourly run, so the PR body says Refs #25131, not Fixes.

    Container & model: S, mode:cloud (fixture setup in three test files, plus a census), model: TIER_DEFAULT (this run's dispatch-gates.mjs --tier: "no path-derived mandate")
    Clause-②: no (test fixtures only: no response, status, error, outcome or export moves)
    Responsibility: anyone reading the hourly full run, and the release gate that reads the full battery | three live-MySQL fixtures insert organization-less rows into tables 7b-0 now creates NOT NULL | main's full battery, red at every commit since fccbd9d0d5
    Thread-read: 6322801126
    Serial constraints cleared: the 9 open PRs' file lists were read at this claim, on origin/main 60b42cd0b7. None touches packages/drivers/driver-sql or a pre-18 testkit.

    Generated by Claude Code

  6. objectstack-fleet commented on Oct 21, 2026

    @objectstack-fleet
    objectstack-fleetBot
    on Oct 21, 2026
    Contributor

    Dispatch · domain:engine seat 1 · session_01JtSQQ3EWsbNPzyYMug1tXy · 2026-10-21T22:19Z: mode:cloud. The dev session is session_018gewcgUaNUX9hMkxeP2HE2. It builds at the default tier on branch claude/issue-25131-mysql-fixtures-pre18-tenant-column, under claim 6327765912.

    • Read first:
      • reproduce the 6 failures against a real MySQL 8.0 (CI's temporal-conformance recipe, in a container) before any edit;
      • for each failing test, which table its seeds write organization-less, and whether a second initObjects restores NOT NULL;
      • why the PG cell passes;
      • a census of every live-dialect driver-sql suite with the same pattern.
    • What lands:
      • asPre18TenantColumn after each initObjects whose table a fixture fills organization-less, as the suites 7b-0 already wired do;
      • the same live run green, and the default suite still green;
      • Refs #25131 (the card closes on the next green hourly run), with the Clause-②: and Metadata-protocol: none lines.
    • If no live MySQL can be had: the dev says exactly why. The proof then falls to the next hourly full run after the merge, and the PR body says so.
    • Not built: no test, cell or assertion is narrowed, skipped or deleted; the NOT NULL itself; test(plugin-security,plugin-auth,plugin-approvals): narrow the blanket pre-18 tenant-column relax rigs to the residue tables they model #25132.
    • Owed: the seat's review of record before the queue.

    Generated by Claude Code

  7. objectstack-fleet commented on Oct 21, 2026

    @objectstack-fleet
    objectstack-fleetBot
    on Oct 21, 2026
    Contributor

    os-dev-report

    {
  "issue": 25131,
  "status": "done",
  "branch": "claude/issue-25131-mysql-fixtures-pre18-tenant-column",
  "pr": "https://github.com/objectstack-ai/objectstack/pull/25142",
  "session": "session_018gewcgUaNUX9hMkxeP2HE2 (dev cloud session; dispatched by domain:engine seat 1, session_01JtSQQ3EWsbNPzyYMug1tXy, seat post #6367)",
  "premise_still_valid": true,
  "summary": "Reproduced on live MySQL 8.0 and PostgreSQL 16 in local docker containers, with CI's recipe. There are 7 failures, not 6, at abc25a35; the hourly log at 60b42cd0b7 (job 119511306969) lists the same 7. The card's 6 was counted at fccbd9d0d5, before #24893 (7b-1 + 7b-2) landed. Five come from 7b-0's NOT NULL rejecting the NULL-organization seeds. Two (12998 'collides two NULL-organization rows…' and 13015 'reports and repairs a shadow that hashes the raw columns') come from 7b-2: a unique: 'organization' declaration now keys the plain tenant column. So asPre18TenantColumn alone cannot green them. The fixture-only edit, in the claim's three files: the org-scoped index these tests use is declared in its 18.0 form (fields [organization_id, v], unique: true, nullSafeColumns [organization_id]; honoured verbatim, the spelling #24893 adopted in the driver-turso suites), and asPre18TenantColumn follows each initObjects whose table then gets NULL-organization rows (7 call sites). No assertion or test was changed, skipped or deleted. Live driver-sql suite after the edit: 242/242 files, 5688 passed, 2 skipped. Default suite and typecheck are green.",
  "tests": "Live (CI env: TZ=America/New_York, OS_TEST_MYSQL_URL=mysql://root:root@127.0.0.1:3306/conformance with time_zone +08:00, OS_TEST_POSTGRES_URL=postgres://postgres:postgres@127.0.0.1:5432/postgres with timezone Asia/Shanghai, OS_EXPECT_LIVE_DIALECT_MATRIX=1). BEFORE, head abc25a35, the three files: 'Test Files 3 failed (3) · Tests 7 failed | 12 passed (19)', VERDICT command-exit 1. The 7, named: 12998 collides two NULL-organization rows under an org-scoped shadow unique (expected 'unhex(sha2(concat(`organization_id`,0…' to contain 'coalesce'); 12998 diagnoses existing NULL-org duplicates… (Column 'organization_id' cannot be null); 12998 names a NULL-organization duplicate in declared terms… (Field 'organization_id' doesn't have a default value); 12998 counts the sixth conflicting group… (cannot be null); 13015 survives 'os migrate apply --allow-destructive'… (cannot be null); 13015 reports and repairs a shadow that hashes the raw columns (a raw-column shadow is real drift and must be reported: expected undefined to be truthy); 15479 leaves the NULL-safe arm saying the NULL-safe thing (cannot be null). AFTER, head e4c6eaf7c: the three files 'Test Files 3 passed (3) · Tests 19 passed (19)', VERDICT command-exit 0. Full `pnpm --filter @objectstack/driver-sql test`, live PG + MySQL: 'Test Files 242 passed (242) · Tests 5688 passed | 2 skipped (5690)', VERDICT command-exit 0. Default run, no live servers: 'Test Files 230 passed | 12 skipped (242) · Tests 3742 passed | 212 skipped (3954)'; `pnpm --filter @objectstack/driver-sql typecheck` exit 0 in the same locked command, VERDICT command-exit 0; `tsc --listFiles` counts all 3 edited files. MEASUREMENT (scratch probe, built driver, live MySQL, IS_NULLABLE of organization_id): NO after initObjects; YES after asPre18TenantColumn; YES after a second initObjects on the same driver; YES after a second initObjects on a new driver; detectManagedDrift then reports [], and applyMigrationEntries with allowDestructive leaves it YES. So no second relax is needed. Ablation: n/a (no tested production behaviour changed; fixture-only).",
  "gates": "On e4c6eaf7c: `node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack` (no paths; change set 3 paths, +67/-8 vs merge base abc25a353) derived 54 commands. Each ran with its exit code captured before any pipe: 52 exit 0. `pnpm check:dual-build-cjs-loads` and `pnpm check:lean-entry-closure` exited 3 (PREREQUISITE NOT MET); after `pnpm exec turbo run build --concurrency=2` (Tasks 77 successful, 77 total, VERDICT command-exit 0), both re-ran to exit 0. `--ran` with 'COMMAND :: exit N' records: 'Run reconciliation — 54 derived, 54 run, 0 NOT-MEASURED, 0 UNRUN' and '✓ … a DERIVED zero'. The repo was deepened first (`git fetch --shallow-since=2026-07-13`). origin/main merged before pr_create: already up to date at abc25a35. NUL/control-byte self-scan of the 3 files: 0 hits. PR check-runs at report time (head e4c6eaf7c): Check Changeset = success; 'No other open PR may claim the same issue' = success; Lint · * (8 jobs), Type Check · * (4 jobs), Governed Surface Queue Guard, 'Part-of PR must not also close its card', 'The card this PR closes must claim this branch', single-writer path, Check PR Size, Check Documentation Links, docs-affected, filter = in_progress; opt-in jobs and Auto Label = skipped. Temporal Conformance (live PG + MySQL) does not run on PRs: its proof is the local live run above plus the next hourly full run.",
  "dogfood_battery": "n/a — no `!` in the PR title and no (narrowing) arm (Clause-②: no)",
  "mcp_calls": "8 — all reads: mcp__github__issue_read ×2 (get, get_comments on #25131), mcp__github__actions_list ×1 (list_workflow_jobs, run 39683348463), mcp__github__get_job_logs ×1 (job 119511306969), mcp__github__pull_request_read ×2 (get, get_check_runs on #25142), plus ToolSearch loads (not GitHub calls). No MCP write tool.",
  "api_writes": "3 — (1) `git push` of claude/issue-25131-mysql-fixtures-pre18-tenant-column (empty branch, then e4c6eaf7c; two pushes to the one branch); (2) relay pr_create, run 39689778307, request fw-20261021T225403Z-c011a5: POST /repos/objectstack-ai/objectstack/pulls (draft) + POST /issues/25142/assignees [os-project-manager] + POST /issues/25142/labels [skip-changeset], read back as 8202 bytes sent / 8202 stored, assignee os-project-manager, label skip-changeset, draft true; (3) relay comment: POST /repos/objectstack-ai/objectstack/issues/25131/comments (this report).",
  "files_changed": [
    "packages/drivers/driver-sql/src/sql-driver-12998-shadow-null-safe-key.test.ts (+23/-4)",
    "packages/drivers/driver-sql/src/sql-driver-13015-shadow-carried-index-drift.test.ts (+25/-2)",
    "packages/drivers/driver-sql/src/sql-driver-15479-shadow-plain-unique-duplicates.test.ts (+19/-2)"
  ],
  "line_budget": "none set; diff +67/-8 over 3 files (per file above)",
  "deviations": [
    "Beyond the claim's stated route: the claim named asPre18TenantColumn alone. Measured, that greens 5 of 7 at most: two failures assert the NULL-safe (COALESCE) shadow form, which since 7b-2 (#24893) a unique: 'organization' declaration no longer produces. Per the dispatch's 按实测修 clause, the fix also declares the org-scoped index in its 18.0 form (nullSafeColumns, honoured verbatim). It stays inside the same three files and inside fixture setup, with no assertion touched.",
    "Failure count: 7, not the 6 in the dispatch and the card; the extra one is 13015 'reports and repairs a shadow that hashes the raw columns'. Measured locally and in the hourly log at 60b42cd0b7.",
    "Base: today's origin/main was abc25a35, not the dispatch's 60b42cd0b7. The one intervening commit (#25141, plugin-auth) touches nothing in driver-sql.",
    "Commit trailers: the commit carries the model-free pair (Claude-Session + 'Co-authored-by: Claude'), per AGENTS.md and the pre-push hook. The harness's attribution reminder asked for a model-named Co-Authored-By; AGENTS.md wins, and that reminder itself yields to repo rules.",
    "PR body footer: the AGENTS.md session-URL footer form only. The harness's separate '🤖 Generated with' PR-body line was not added (os-dev 字节与 sanitizer 纪律 governs).",
    "Platform PR-subscription posture: per os-dev (本派发契约压过平台注入的 PR 订阅姿态), I did not subscribe to or watch #25142, and did not wait on CI.",
    "Infrastructure started in this container: dockerd (its PID recorded in the scratchpad) and two containers, os25131-mysql and os25131-pg; all three are torn down after this report."
  ],
  "open_questions": [
    {
      "question": "How should the 7 live-MySQL tests that pin the NULL-safe hash-shadow arm model their store after 7b-0 + 7b-2? (Settled by me; recorded for the seat.)",
      "options": [
        "A — the 18.0 store: index input carrying nullSafeColumns (honoured verbatim) + asPre18TenantColumn. Fixture-only; every assertion kept; this is #24893's own Turso spelling for the retained path.",
        "B — rewrite the assertions to the new plain semantics (NULL-organization rows do not collide). Changes assertions, which the card forbids, and drops the pins on a driver path (nullSafeColumns input, the COALESCE readers) that still exists.",
        "C — stamp an organization on the seeds. Removes the NULL-organization rows that are the subject of every one of these tests, so they would measure nothing."
      ],
      "recommendation": "A, taken, by the four axes. Real need: the NULL-safe arm is still live code, reached by any input carrying nullSafeColumns and by an 18.0-built store, so its pins guard real behaviour. Long-term soundness: it mirrors #24893's own precedent and needs no production change or workaround. Preventing AI mistakes: every assertion stays strict and nothing gets more lenient. Startup focus: no new surface or gate; +67/-8 in three test files. If the NULL-safe input path is ever retired, these suites retire with it."
    }
  ],
  "out_of_scope_findings": [
    "carrier: none (承接者:无) — noted in the PR's Acceptance notes only. Card/triage arithmetic: #25131's body and triage count 6 failures, read at fccbd9d0d5; at 60b42cd0b7 and later it is 7. Not a defect class a/b/c; no card filed."
  ]
}

    Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

area:devpathThe road — create, dev, verify, publish/install, connect an agent, iteratebugSomething isn't workingdomain:enginepm:dispatchedpriority:p1High: required for production / M2

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions