Skip to content

A consumer upgrading across a MINOR has no machine-readable answer to "what changed" — spec-changes.json is major-shaped and ships added/removed empty #17080

Description

@os-sam

What happened

I upgraded a downstream metadata project (objectstack-ai/ats) from @objectstack/spec 17.3.0 → 17.4.0. The upgrade itself was clean, but establishing that it was clean took building a second worktree pinned at 17.3.0 and diffing two node_modules trees by hand — because nothing in the installed artifact could answer "what changed between 17.3.0 and 17.4.0".

This is not a documentation complaint. The platform ships genuinely good machine-readable upgrade data; the problem is that all of it is keyed to the protocol major, while breaking changes are shipped in minors.

The three surfaces a consumer can read, and what each says about 17.3 → 17.4

1 · spec-changes.json (ADR-0087 D4). Major-shaped by construction. From the published 17.4.0 tarball:

protocolVersion : "17.0.0"          <- does not move with the package minor
supportFloor    : 10
perMajor[-1]    : { from: 16, to: 17, converted: 57, migrated: 77, added: 0, removed: 0 }
aggregate       : { converted: 67, migrated: 87, added: 0, removed: 0 }

Every conversion entry carries toMajor: 17 and nothing finer. So the finest question this file can answer is "16 → 17", which a project already on 17.3 has answered long ago.

2 · added[] / removed[] — the data exists, it just doesn't ship. packages/spec/scripts/build-spec-changes.ts documents a release-time surface join:

--previous-surface <path> diffs the current committed export surface against a previously published one … The Release workflow runs this against the last published spec tarball and attaches the result to the GitHub Release; the committed copy keeps added/removed empty (registry-derived content only) so it stays deterministic.

Measured on the published 17.4.0 tarball: aggregate.added = 0, aggregate.removed = 0. So the one artifact that is computed per-release reaches the GitHub Release page and never reaches node_modules, which is where a consumer's tooling looks.

3 · validate --jsonspecVersionGap. The right idea, also major-resolution. On a project declaring engines: { protocol: '^17' } against a protocol-17 runtime it is null — correct, and silent about the fact that 17.3 → 17.4 narrowed several accept-sets.

Why this matters more than it looks: 17.4.0 shipped 15 BREAKING entries as a minor

The @objectstack/spec@17.4.0 changelog carries fifteen entries marked BREAKING, each noting it is released as minor "under the repo's launch-window convention for breaking changes". Examples: timeDimensions[].dateRange closing to a preset vocabulary, the api/ duration-key renames, composeStacks({objectConflict:'merge'}) refusing unmergeable pairs, a text operator over a non-string-typed field being refused at the engine door.

That last one is not hypothetical — it is the single behavioural difference I measured across 63 identical API probes on the same tree:

$filter [["headcount","contains","1"]]     17.3.0 -> 200 / 28 rows
                                           17.4.0 -> 200 /  0 rows

A consumer reading 17.3.0 → 17.4.0 has every reason to skip regression testing. Semver is the contract most tooling and most humans act on, and here it says "safe".

A second, smaller trap: the tombstone count is not a retirement count

With no per-release data, the obvious fallback is to diff the tombstone prescriptions in json-schema/**. That is misleading:

17.3.0 : 156 [REMOVED] prescriptions
17.4.0 : 221          (+65)

Of those 65, exactly one names 17.4.0 as the release that retired it (SessionUser.language). The other 64 are protocol-17-era retirements whose fix-it text was newly added to the schema in 17.4. Nothing in the data distinguishes "retired in X" from "prescription written in Y" — you have to parse the English sentence. So the delta cannot be used to size an upgrade, and a consumer who tries will over-estimate by 64×.

Related, and worth a look on its own: perMajor[to=17].converted went 58 → 57 between 17.3.0 and 17.4.0. A conversion left the published chain in a minor, so the replay result is not stable across minors within a major.

What would fix it

In rough order of value per unit of work:

  1. Attribute retirements to the release that made them, in data. A retiredIn: "17.4.0" field on conversion / migration entries (and on the tombstone metadata) turns every question above into a filter. This is the one change that makes the rest unnecessary.
  2. Ship added[] / removed[] in the tarball. The release workflow already computes them. Determinism of the committed copy is a real constraint, but it can be satisfied by writing the join into the published artifact at pack time rather than into the repo — the consumer's copy does not need to be byte-reproducible from the registries, it needs to be true about the release it ships with.
  3. Make the BREAKING marker machine-readable. Today the only signal is the word "BREAKING" in changelog prose, inside a release body that is itself truncated (see below). A breaking: true flag plus a count would let a consumer's CI gate on it.
  4. Give specVersionGap minor resolution, once (1) exists — it is already the right place for a consumer to be told "you are on 17.3, this runtime is 17.4, here are the N entries between you".

Adjacent, filed here because it blocked the same task

The @objectstack/spec@17.4.0 GitHub Release body opens with its own apology:

This release note is truncated. The changelog entry for @objectstack/spec@17.4.0 is 382,656 characters; the GitHub Releases API rejects any body over 125,000.

So the human channel is truncated at exactly the moment the machine channel has no answer either. Happy to split this into its own issue if it is better tracked separately.

Repro

npm pack @objectstack/spec@17.3.0 && npm pack @objectstack/spec@17.4.0
# then compare, in each extracted package/:
#   spec-changes.json  -> protocolVersion, perMajor[].toMajor, aggregate.added/removed
#   json-schema/**     -> grep -rho '\[REMOVED\][^"]*' | sort -u | wc -l

🤖 Generated with Claude Code

https://claude.ai/code/session_01PbJ5Cy9KDAzeQHo8bsMadG

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions