feat(driver-sql,objectql,cli)!: the ADR-0104 file-family column step and its kernel→driver supply (#15989) - #17859
feat(driver-sql,objectql,cli)!: the ADR-0104 file-family column step and its kernel→driver supply (#15989)#17859os-musk wants to merge 9 commits into
Conversation
Claude-Session: https://claude.ai/code/session_01RuoNSXUbBoWHkNS4AknTrM Co-authored-by: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RuoNSXUbBoWHkNS4AknTrM Co-authored-by: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RuoNSXUbBoWHkNS4AknTrM Co-authored-by: Claude <noreply@anthropic.com>
…tion Claude-Session: https://claude.ai/code/session_01RuoNSXUbBoWHkNS4AknTrM Co-authored-by: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RuoNSXUbBoWHkNS4AknTrM Co-authored-by: Claude <noreply@anthropic.com>
…and its kernel→driver supply The column step retypes and rewrites media columns on PostgreSQL and SQLite after backfill + verify report zero blocking rows, aborting loudly on any cell the move would not preserve; the kernel→driver seam reads sys_migration.columns_moved_at and arms SqlDriverConfig.fileColumnsMoved. Claude-Session: https://claude.ai/code/session_01RuoNSXUbBoWHkNS4AknTrM Co-authored-by: Claude <noreply@anthropic.com>
📓 Docs Drift CheckThis PR changes 5 package(s): 10 hand-written doc(s) NAME something this change touched and may need an implementation-accuracy re-verification:
⛔ 1 release-owned page(s) also name something this change touched. These are read-only:
What this run could not see
Coarse fallback — 142 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): Which tree this was computed onThis run read A worktree cut from an older # while this PR is open — GitHub drops the merge commit once it closes
git fetch origin 7a15e797af58633ad7813cec02abb6dcae8ee34d && git checkout 7a15e797af58633ad7813cec02abb6dcae8ee34d
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin ed8dea17bd510100320ab42dbac6ec2a78e99deb 303d7946e4830f8721851fc565dea41275c117d0 && git checkout -B drift-repro ed8dea17bd510100320ab42dbac6ec2a78e99deb && git merge --no-ff 303d7946e4830f8721851fc565dea41275c117d0
node scripts/docs-audit/affected-docs.mjs --json ed8dea17bd510100320ab42dbac6ec2a78e99deb
|
- check:doc-authoring — no tracker id in a runtime string an operator reads; the engine's log literal stays written once rather than duplicated - check:engine-double-contract / check:objectql-double-limit / check:where-matcher — the new fakes route update() through assertEngineUpdateDispatch, hold the caller's limit by presence, and refuse a combinator they do not implement instead of reading it as a field name Claude-Session: https://claude.ai/code/session_01RuoNSXUbBoWHkNS4AknTrM Co-authored-by: Claude <noreply@anthropic.com>
…mmands oclif's command table globs **/*.js under dist/commands, so a module there with no default-exported Command makes every CLI invocation warn `findCommand … not found` on stderr — measured, and it broke `os validate --json` for a consumer that reads stdout and stderr together. Also records the ruling, not the zero-hit reading, as why the kernel→driver wiring is owed: asked identically, `sqliteJournalMode` reads zero too and is no defect. Claude-Session: https://claude.ai/code/session_01RuoNSXUbBoWHkNS4AknTrM Co-authored-by: Claude <noreply@anthropic.com>
…m the diff 13 new declarations reaching a package entry, not five, and 3 new public methods on exported classes, not two — each named, so a reader can recount them off the three-dot diff. The disposition, the level and the BREAKING banner are unchanged; the clause the counts support is more true at 13/3. Claude-Session: https://claude.ai/code/session_01RuoNSXUbBoWHkNS4AknTrM Co-authored-by: Claude <noreply@anthropic.com>
Contract review
席内默认判断档复核,由本卡派发席(
① Derived judgments —— 逐条点名公开面(本席自量,⛔ 非转述 PR 正文)
裁决
|
| 裁决条 | 要求 | 核对 |
|---|---|---|
| ① keying | 走 sys_migration.columns_moved_at,⛔ 永不单凭 adr-0104-file-references flag |
✅ 且直接钉住:adr0104-file-columns-moved-supply.test.ts 有 ⛔ a VERIFIED flag with NO stamp is "not moved" — the flag alone may never key the arm |
| ② SQL | ⛔ 不用 addendum 的 USING (col #>> '{}') 裸用,改用实测的大声中止子句 |
✅ 见下「消融」 |
| ③ MySQL | 出范围,归 #17788 | ✅ §5 — MySQL is refused by NAME, not attempted,mediaColumnMoveDialect('mysql') 答 null |
⭐⭐ 消融 —— 本轮最要紧的一条,本席逐行读过,⛔ 不采信转述
sql-driver-15989-file-column-move.test.ts §1 真的执行了被推翻的那条子句(:208 就是 alter table … using ("cover" #>> '{}')),而不是在注释里论证它不安全:
- 被推翻的形式:ACCEPTED,无抛出、无告警,落在它被要求拒绝的那一行上;
- 本 PR 的形式:前置检查答 1 ⇒ 一条语句都不跑,且拒绝是整体的、不是逐行的(已转换那格也原样)。
⭐ 而最见功力的是它拒绝把两个方言平均掉:PG 上 json 逐字存储输入文本、#>> '{}' 原样交还 ⇒ 字节存活、类型被毁(expect(after).toBe(before) 且 typeAfter 不再含 json)—— 这正是该缺陷对字节 diff 隐形的原因;SQLite 上没有类型可丢 ⇒ 损失在字节(not.toBe(before),缩进没了)。⛔ 平均成一句话就会只断言其中较弱的那个。
控制项 §2:同一前置检查在无未转换行的 fixture 上答 0,移动照常进行 ⇒ §1 那个 1 是读数,不是一个永远拦路的前置检查。文件里自己写着「Without this the §1 reading is void」。
「每一种不知道都答『未迁移』」—— 两侧都带发火的控制项
driver 侧 12 例 · engine 侧 13 例,涵盖:选项缺省 · resolver 抛出 / 拒绝 / 答非 true / 从不运行 · driver 无该缝 · 无 sys_migration 对象 / 无行 / 表不可读 / 空串戳 / 未验证行上的戳 / 带阻塞发现的戳。
⭐ 两侧各有控制项:driver 侧 ⑥ …a resolver that answers true DOES move the arm — the control;engine 侧 ⭐ CONTROL — a verified row WITH a stamp answers true, so the falses above are readings。⇒ 每一个 false 都是读数,⛔ 不是被焊死的臂。
⭐ 另两条本席认为判得对:宿主在自己配置里声明的 fileColumnsMoved 双向皆胜(覆盖 false 会把裸 id 写进 JSON 列;覆盖 true 会把 JSON 写进已retype 的列),以及首次 initObjects 之后到达的 resolver 被拒(此时 isJsonField 已冻结,晚到的 resolver 是一个 driver 兑现不了的承诺)。
② Semver 定级
- changeset:
driver-sql/objectql/platform-objects/spec/cli五个包minor,带发布窗**BREAKING**banner(已发布存储行为变化),ADR-0087 disposition =not-required (no-migration-prescription)。 - ✅
minor正确、⛔ 非major:判据是上面 ①.3 的「删除的export行 = 0」,纯增。 - ✅ 五个被打分的包每一个都真的发布了变化(见 ①.1–①.2)⇒ level 轴由真正加宽的那个包满足,⛔ 不是蹭到的。
- ✅
Check Changeset的 level 轴:yes需至少一个minor,此处有五个。
③ 边界旗处置 —— 逐旗
| 旗 | 处置 |
|---|---|
已立 #17869 —— src/commands/ 下的非 Command 模块让每次 os 调用都在 stderr 上告警,并污染机读输出 |
✅ 接受。本席已复核:卡面裸立不分级、footer 在(该轮用 REST 建卡,未撞上 MCP issue_write 吞 footer 那个坑)。⛔ 该实例在本 PR 内已由「把 helper 移到 src/utils/」修掉,卡承载的是缺失的门禁 |
@see SqlDriver.setFileColumnsMoved 悬空链接 |
✅ 接受,本 PR 就地修(链接想指的方法正是本 PR 新增的那个),无需承接者 |
readDataMigrationFlag 丢掉 columns_moved_at |
✅ 接受,本 PR 修。⭐ 在范围内:那是本臂的证据路径,而列步骤是第一个必须区分「已迁移/未迁移」的调用者 |
os migrate plan 的散文仍描述 JSON 终态 |
✅ 接受,承接者 #16184(开着的卡,确实重写那条 drift-message 臂) |
SQLite :memory: 两实例=两数据库的 fixture 陷阱 |
✅ 接受,⛔ 不立卡 —— 已写在该文件自己的注释里,正是会撞上它的作者会读到的地方;本 PR 改用文件型 SQLite |
| tsup 不删已移除源文件的产物 | ✅ 接受,AGENTS.md §9 既有类别 |
⭐ 一条本席不接受作为「转交」的,本席自己承担
生成器 TS 与 SQL 两半在本家族宽度上仍不一致 —— generateMigrationSql 出 VARCHAR(2048),generateMigrationTs 出 knex 的 varchar(255)(generate.ts:1338 / :1461 / :1490-1491 / :1546)。该条已被转交两轮(#17403 → #15989),而它点名的「generate.ts 卡」不存在。
generate.ts:1375 那句「⛔ NOT in scope here, and filed rather than mirrored」—— 它指的是本卡 #15989 自己(把该家族移出 JSON_COLUMN_TYPES),⛔ 不是宽度卡。⇒ 宽度分歧确实无卡。
Dedup 已跑(537 张开着的卡,body 截断 6000 字符 —— 这是读数的边界):VARCHAR(2048) 2 命中、generateMigrationTs 1 命中,逐条看过都是 #15989 与 #16184 自己;控制项 generate.ts 2 / knex 5 发火,反控制项 0。
⇒ ⭐ 本席在本 PR 落地后立卡(裸立不分级,packages/cli/** ⇒ 路由是分诊的写权)。⛔ 现在不立:那条分歧的前提在本 PR 落地时才定型(本 PR 把驱动侧 retype 到 2048,分歧才收敛成「生成器自己两半之间」)。记在此处而非记在本席脑子里 —— ⛔ 席位在 GitHub 之外不维护任何跟踪状态。
⚠️ 一条读数纪律,本轮差点让本席误诊
本席的 CI 监视器在 8e5e7627a4 上报出 Test Core 与 TypeScript Type Check 红。⛔ 那不是缺陷 —— 是本席自己那条计数更正请求触发了推送(303d7946e4 @ 16:12:50Z),GitHub 取消了在飞的作业,而聚合门因其依赖被取消而报 failure。
⇒ ⭐ 识别特征:聚合门 failure,而其子作业是 cancelled 且没有任何一个是 failure ⇒ 这是被取代的运行,⛔ 不是红。先读 PR 的当前 head,再去诊断代码。 基线对照同样说了话:origin/main 7c2c5aedd9 上同名检查 51 success / 0 红。
Implemented-by: claude/issue-15989-file-column-step
Reviewed-by: session_01RuoNSXUbBoWHkNS4AknTrM
(mode:subagent 派发 —— 子代理无自有 session,按细则记其分支。⭐ 独立性对裸写:本席今天在 #17839 上把它写成粗体,机读为 UNREADABLE,已修;contract-review.md 现行本写着「值紧跟冒号,前置词即不可读」。)
PASS
条款②申报 yes 与实际公开面一致(13 条入口声明 + 3 个公开方法,零删除),semver 与 changeset 相符且五个包各自真的发布了变化,裁决三条逐条落实且消融是执行出来的而非论证出来的,六面边界旗逐旗处置、其中一条由本席承担。
⇒ 待 CI 在 303d7946e4 上收敛全绿后,同笔剥双载体、转 ready、挂 auto-merge。⛔ 载体现在不剥 —— 落地前检 ③(全部 check 绿)尚未成立。
Generated by Claude Code
Fixes #15989
Clause-②: yes
Step 2 of the maintainer ruling on #15041, completed. PR #17403 landed the encoding half; this lands the two things it named as still owed — the kernel→driver supply seam and the column step — under the director ruling
5643444495(decision batch #120 item 1).⛔ The ruling supersedes the card body in two places, and both are honoured
1 · Keying. The card body's "Shape of the change" item 1 says the wiring is keyed on the
adr-0104-file-referencesflag. That is overturned. The arm is keyed onsys_migration.columns_moved_at, and it additionally requires the flag to be verified — never the flag alone. The reason is measured, not formal: every creation-attested store since 17.0 holds that flag and JSON-quoted ids in a JSON column, so flag-keying would read every existing deployment as migrated and then write bare ids into a JSON column.adr0104-file-columns-moved-supply.test.tspins exactly that row — verified, unstamped — as "not moved".2 · The migration SQL. The addendum's
USING (col #>> '{}')is not used bare. It is preceded by the pre-check the dev seat measured, which aborts on a non-JSON-string cell. See the ablation below: the superseded form was executed on live PostgreSQL 16.13 in this PR's own suite, and it does not abort.⭐ Worth carrying forward, in the card's own words: "an Execution paragraph is authoritative about intent, not immune to being wrong about SQL."
The ablation — the single most important measurement here
Both clauses, executed on one fixture (one converted cell, one inline blob the backfill has not reached), in
sql-driver-15989-file-column-move.test.ts§1, on every dialect this step serves.ALTER … TYPE varchar(2048) USING (col #>> '{}'), alonejson, so the unconverted object is now a plain string in a column whose declared contents are bare ids.json_typeof(col) IS DISTINCT FROM 'string'json.NULLpreserved. So the1above is a reading, not a pre-check that always blocks.jsontype stores its input text verbatim and#>> '{}'hands that same text back, so the bytes survive and the TYPE is destroyed — which is exactly why the defect is invisible to a byte diff. On SQLite there is no type to lose, so the loss is in the bytes:json_extractre-serialises the blob and its formatting is gone. Either way the superseded form wrote to a row it was required to refuse.Reproduced outside the suite first, against a cluster started for the purpose (PostgreSQL 16.13, port 54331, torn down afterwards); the readings are in the report.
The seam — "every way of not knowing answers not moved"
fileColumnsMovedhad no supplier outsidedriver-sqlon the dispatch base — re-measured here with a same-subject control, asking the identical question of each ofSqlDriverConfig's own added keys over the same corpus and package boundary:schemaMode72 files ·autoMigrate14 ·sqliteAbsentFile6 (the firing control, and it includes a real host wiring site,service-datasource/src/default-datasource-driver-factory.ts) ·sqliteJournalMode0 ·fileColumnsMoved0. ⛔ So the zero proves only that there was no supplier —sqliteJournalModereads zero too and is a perfectly ordinary optional key no host sets. What makes this one owed is the director ruling's item 2, which places the kernel→driver wiring inside this dispatch rather than after it.ObjectQL.registerDriverhands every driver that has the seam a closure over the newObjectQL.haveFileColumnsMoved(). Pinned on both sides — thirteen engine-side cases and twelve driver-side ones:truevalue · resolver never runs (the host never callsinitObjects) · driver has no seam at all;sys_migrationobject · no row · unreadable table · null stamp · empty-string stamp · stamp on an unverified row · stamp on a row with blocking findings;true, so everyfalseabove is a reading rather than a welded-shut arm.⛔ A host that names
fileColumnsMovedin its own config wins, in either polarity. The engine only fills an empty slot. Overruling a declaredfalseis the bare-ids-into-a-JSON-column failure this whole mechanism exists to prevent; overruling a declaredtruewrites JSON into columns already retyped. The host is the more specific authority about its own storage, so the engine never contradicts it.A resolver arriving after the first
initObjectsis also refused:registerObjectMetadatahas already frozenisJsonField's answer for every media column, so a late resolver would be a promise the driver cannot keep.The column step
os migrate files-to-references --applygains it, and it moves nothing until three gates pass: the migration's own gate (zero blocking rows), every pre-check across every planned column, and no refusals. Two-phase on purpose — a step that moved three columns and aborted on the fourth leaves a datastore in a state no flag can describe.The shape is read off the column's physical type, never off the dialect: a
jsoncolumn is retyped; a column alreadyvarchar— the populationos generate migration --format sqlcreates and a JSON-arm driver fills with quoted ids (#15771) — has its values unquoted in place. Classifying by dialect would have left that population full of quoted ids behind acolumns_moved_atstamp claiming it was converted.⛔ MySQL is out of scope and refused by name, per ruling item 3 — #17788 owns it, on a real instance, because the addendum leaves its statement ORDER unsettled.
mediaColumnMoveDialect('mysql')answersnulland the step reports a named refusal. §5 pins that refusal so nobody "completes the matrix" by transcribing a form nobody has run — which is the move that produced the Postgres clause this card had to overturn.Per-dialect pin results
sql-driver-15989-file-column-move.test.ts, both arms, both encodings, across the window:OS_TEST_POSTGRES_URLprovisioned.mysqld/mariadbbinary and no docker daemon in this container; more to the point the step has no MySQL statements to measure. This is a named absence, never a pass and never a fail.§3 is the reverse verification the card asks for: after the move, a driver on the moved arm writes a bare id and reads it back unchanged, and reads the row the move itself converted; §3b holds the other half of the window, where an unmoved deployment still stores and reads its JSON-quoted id.
:memory:, so two driver instances are two separate databases — a §3 written against it would have been green while measuring nothing (the second driver creates its own table, its own writes read back perfectly, and the migrated rows are simply absent). The suite uses a file-backed SQLite database so the two arms share bytes, as the live cells already do.Changeset
.changeset/15989-file-family-column-step.md—minoron the five packages that publish a change, with the launch-window**BREAKING**banner, and the ADR-0087 disposition not-required (no-migration-prescription) derived from this diff: nothing authorable moves.DataMigrationFlagSchemaand itscolumns_moved_atmember landed under #16185 and are read here, not edited; the onepackages/specedit is a new exported predicate function over that existing type. Soos migrate metahas nothing to visit,spec-changes.jsonnothing to project, the upgrade guide no row to gain. The other four categories are closed on facts in the changeset itself.Verification
Every exit code captured by redirect-then-
$?, never across a pipe.Suites — each package's own
testtarget, unnarrowed. ⛔ No--projectfilter anywhere (a--projectfilter silently drops a named file outside that project and reports it as passing — #17853).@objectstack/driver-sql@objectstack/cli@objectstack/objectql@objectstack/spec@objectstack/platform-objectsdriver-sqlran against live PostgreSQL 16.13 with the server atAsia/Shanghaiand the process atAmerica/New_York, matching howci.ymlprovisions the live matrix — the live suites assert a three-way zone skew and are vacuous on a UTC server, which is a guard, not an obstacle.⛔ MySQL: NOT MEASURED. No
mysqld/mariadbbinary and no docker daemon in this container — and the column step has no MySQL statements to measure in the first place (#17788). A named absence: never a pass, never a fail.Gates.
node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstackderived 103 families for this diff; all 103 were run and reconciled with--ran, each carrying its real exit code, so the NOT-MEASURED zero is derived rather than claimed. ⛔ Four hitexit 3(PREREQUISITE NOT MET — never a pass) and went green after a fullpnpm build:check:dual-build-cjs-loads,check:i18n,check:i18n-coverage,check:i18n-walk-parity.Four gates went genuinely RED on my own first draft, and are recorded rather than smoothed over:
check:doc-authoring— a tracker id in two runtime strings an operator reads. Fixed by removing the id from the driver's refusal text and by writing the engine's log literal once instead of duplicating it.check:engine-double-contract/check:objectql-double-limit/check:where-matcher— all three on my new fakes. They now routeupdate()throughassertEngineUpdateDispatch, apply the caller'slimitby presence after the filter, and REFUSE a combinator they do not implement instead of reading it as a field name. The last one matters here more than usual: a matcher that answers "no rows" for a filter it does not understand is indistinguishable from an empty ledger, which is exactly the false "not moved" every case in those files is trying to tell apart from a real reading.Cross-package reverse verification (an additive optional member on a type crossing a package boundary is exactly the shape a cached
.d.tsanswers the same way twice, so both legs were run): plantingMediaColumnMoveScan['plansZZ']in@objectstack/cligoes RED withTS2339: Property 'plansZZ' does not exist on type 'MediaColumnMoveScan'(leg 2 exit 1), while the real member typechecks (leg 1 exit 0) — sotscis readingdriver-sql's rebuilt declaration, not a cached one. Restored under atrap … EXIT INT TERMwith repo-absolute paths viagit checkout HEAD -- path, and the restore proven by state rather than by an exit code:git hash-objectback to88f1ad9944byte-identical to the HEAD blob,git diff HEADempty.All gate figures above are read at the final commit
303d7946e4; the reconciliation and the suites were re-run on the head that produced them rather than on the tree that first did. The last commit is a one-line correction inside the ADR-0087 marker — the seat's review measured my rationale's counts as understated, I re-measured them independently off the three-dot diff and got the same answer, and the marker now NAMES what it counts so a reader can recount it: 13 new declarations reaching a package entry (11 on@objectstack/driver-sql's entry — 6 values and 5 types — plusrecordFileColumnMoveon@objectstack/platform-objects/systemandhasMovedFileColumnson@objectstack/spec) and 3 new public methods on exported classes (SqlDriver.planMediaColumnMove,SqlDriver.setFileColumnsMovedResolver,ObjectQL.haveFileColumnsMoved— none private or protected, on classes exported atsql-driver.ts:4514andengine.ts:2768). The level, the BREAKING banner and the disposition itself are unchanged; the clause those counts support is simply more true at 13/3 than at the 5/2 I first wrote.Control-byte self-scan over all changed files: zero hits, with a control that fires on an in-class byte (0x0B).
pnpm check:nul-bytesexit 0.Acceptance notes
Commandmodule underpackages/cli/src/commands/makes EVERYosinvocation warnfindCommand … not foundon stderr, because the oclif command table is a**/*.jsglob — and it corrupts machine-readable output for any consumer that reads stdout and stderr together (os validate --jsonthen failsJSON.parse). Found because my own first draft put the column-step helper there; that instance is fixed in this PR by moving it tosrc/utils/, where every other CLI helper already lives. The missing GUARD is the card:check:cli-command-idswalks the same population but asks the opposite question, and the trap was caught by exactly one of 3247 CLI cases. Dedup ran before filing, with a firing control.noted, not filed:the docblock onSqlDriverConfig.fileColumnsMovedpointed atSqlDriver.setFileColumnsMoved, a method that did not exist — a dangling link left when the option landed without its supply. Repaired in place by this PR, since the method the link wanted is the one this PR adds. Successor: none needed, it is fixed here.noted, not filed:readDataMigrationFlagdroppedcolumns_moved_aton the way out, so a moved deployment was indistinguishable from an unmoved one to every caller reading the ledger through it. Fixed here because the column step is the first caller that has to tell them apart. Successor: none needed.noted, not filed:the TS and SQL halves of the generator still disagree on this family's width —generateMigrationSqlemitsVARCHAR(2048)whilegenerateMigrationTsemits knex'svarchar(255). Carried from the feat(driver-sql)!: the file family's column holds the baresys_fileid, per deployment (#15989) #17403 round, which named the column-step seat as its successor. This PR retypes to 2048, the SQL half's width and the one the ruling calls the end-state, so the disagreement is now between the generator's two halves alone. Successor: apackages/cli/src/commands/generate.tscard; the file is untouched here.noted, not filed:os migrate planprose for a media column still describes the JSON end-state, generated from the drift entry's message. Correct for an unmoved deployment and wrong once one can be moved — which this PR makes possible for the first time. Successor: the drift-message half of driver-sql(schema-drift): during #15989's dual-encoding window a staticJSON_COLUMN_FIELD_TYPEScannot serve both moved and unmoved deployments — ⚠️ the body's "reports the ruled end-state" framing is MEASURED FALSE, see comment 5588614136 #16184, which rewrites that arm.Authored by the
domain:engineexecution seat, sessionhttps://claude.ai/code/session_01RuoNSXUbBoWHkNS4AknTrM. (Carried in prose: on this repository the PR-body UPDATE channel does not recognise the session-URL footer block and appends a second, bare one — measured on the create/update pair for this very PR, so the platform's own appended footer below is left as the single one.)Generated by Claude Code