Skip to content

fix(scripts): make the Check Changeset refusal say which refusal it is - #19117

Merged
os-try-charles merged 1 commit into
mainfrom
claude/issue-18263-changeset-refusal-says-its-reason
Sep 18, 2026
Merged

os-try-charles merged 1 commit into
mainfrom
claude/issue-18263-changeset-refusal-says-its-reason

Conversation

@os-try-charles

Copy link
Copy Markdown
Collaborator

Fixes #18263

Clause-②: no

The defect, stated as it measures rather than as the title spells it

The card's title says "an entirely empty check-run output". Measured later on the same card (comment 5705401851, PR #18524, run 104955982460), the failing Check Changeset answers output.title = null with 0-byte summary and text and carries annotations_count = 1 — the runner's own generic exit-code annotation (path .github, level failure, title '', message 'Process completed with exit code 1.'). So the accurate description is not "an empty output"; it is "one generic annotation that states no cause".

That distinction is what makes this fix cheap. The annotation channel is already open and already carried by this job, and a plain run: step owns exactly one way onto it — the ::error:: workflow command, whose sibling ::notice this script has emitted for years. The gate's prose refusals — among the best in the repo, naming the missing reading, quoting the offending line and spelling the remedy down to "this red clears with no push and no re-run" — reach the job log and are then discarded at the check-run boundary. The reason is produced and thrown away; the repair is to say it.

What changed, entirely inside scripts/check-changeset-no-major.mjs

  1. Every near miss now renders its REASON. readClause2Line has always returned spelling / inline-key / describing beside the offending line; this gate printed "a near miss" and dropped the reason. It now prints the reason, the line, and the remedy that reason owes — three different sentences, because check-clause2-carriers.mjs says in its own words that "⛔ The reason changes the sentence, never the state".
  2. Every refusing lane emits exactly one ::error:: annotation, so the diagnosis crosses the check-run boundary onto the channel the run already carries. Greens emit none.

The reason is interpolated, never matched against a list held here. CLAUSE2_NEAR_MISS_REMEDIES is a lookup from whatever the reader produced, and its miss path is loud: a reason this file has never been taught still prints, still names the offending line, and still says where the reason came from. That direction is deliberate — the reader is a live surface (PR #18903 is open on it, +225/-41), and a gate that matched reasons against a frozen list would answer a new one with exactly the silence this card is about.

Exactly one annotation per refusal, deliberately: a check run caps annotations at ten per level, and this script has been past that cap before (its stock-scoped predecessor emitted 171 ::notice lines on PRs that introduced none of them). And the annotation is not conditioned on GITHUB_ACTIONSrender and renderLevel are pure by design, which is what lets the self-test assert the MESSAGE rather than the exit code, and an env read inside them would make the one thing this PR adds the one thing the fixtures cannot see.

What did NOT change

Post-condition 1 — the real predicate drives every row, with a negative control that can fail

Every row below was produced by importing readClause2Line (the same function the gate calls) and running the real judgeLevel / renderLevel. ⛔ No hand-written matcher anywhere.

body line readClause2Line exit reason now in the emitted annotation
## Clause-②: no — … (heading) near-miss / spelling 1 spelling + remedy
`Clause-②: no` · `skip-changeset` near-miss / describing 1 describing + remedy
Domain: `domain:devx` · Clause-②: no near-miss / inline-key 1 inline-key + remedy
Clause-②: no (bare, own line) declared / no 0 no annotation — a green annotates nothing
NEGATIVE CONTROL nothing here null 1 "the PR body carries no Clause-②: line" — and no remedy, because there is no line to remedy

The emitted text, for the heading shape (one line, escaped, abridged):

::error title=Check Changeset (level axis)%3A no readable `Clause-②%3A` declaration%2C and it is
the reading this PR needed::… %0A· declaration line: a near miss, not a declaration — reason
`spelling` — ## Clause-②: no — nothing published moves here%0A· remedy for `spelling`: the line
does not carry `Clause-②:` in the fixed spelling at the start of a line. The reader tolerates a
`- `, `* `, `> ` or `**` prefix and NOTHING else, so a markdown HEADING is a near miss and not a
declaration. Write it bare, on a line of its own.%0ADECLARE IT: …

Post-condition 2 — a reason the code has no message for still prints

Driven through the exported nearMissReadings('a-reason-this-file-has-never-been-taught', …):

· declaration line: a near miss, not a declaration — reason
  `a-reason-this-file-has-never-been-taught` — ## Clause-②: no — the line that would otherwise be lost
· remedy for `a-reason-this-file-has-never-been-taught`: this gate carries no remedy sentence for a
  near miss of reason "a-reason-this-file-has-never-been-taught" — `readClause2Line`
  (scripts/pm/check-clause2-carriers.mjs) reports a reason this file has not been taught, and the
  reason plus the line are printed rather than swallowed. The offending line is: … Add the sentence
  for this reason to CLAUSE2_NEAR_MISS_REMEDIES in scripts/check-changeset-no-major.mjs.

Control: the same call with a KNOWN reason returns a different sentence, so the miss path is not silently borrowing a known remedy — a wrong prescription is worse than a named gap. null and undefined reasons also return a sentence; an empty remedy would be this card's defect moved one function along.

Post-condition 3 — the two refusals, side by side

refusal where it is emitted annotation
Clause-② declaration unreadable (near miss, or absent) this script, level axis ::error title=Check Changeset (level axis)%3A no readable Clause-②%3A declaration… with the reason, the line and the remedy in the message
declared yes, no moved package graded minor+ this script, level axis ::error title=Check Changeset (level axis)%3A clause-② declares YES while no moved package is graded minor or above…
this PR adds no changeset pr-automation.yml, Require a changeset step, unchanged by this PR ::error::This PR adds no changeset. FIRST: …

A near miss and an absent line share a verdict (not-measured-material) and used to share every byte anyone outside the run could read; they now differ in the message, which the self-test pins. The missing-changeset refusal is the workflow's own and already carried an ::error::; the self-test now pins that it keeps one, because without it that refusal and this script's are once again one event from outside.

Post-condition 4 — self-test and battery floor, before and after

reading before (07c6f822e) after
node scripts/check-changeset-no-major.mjs --self-test exit 0, 299 assertions exit 0, 335 assertions
SELF_TEST_BATTERY_FLOOR (pinned roster size) 18 19
declared batteries 18 19
'Missing input is a failure, never a pass (#4690 / #7006)' floor 5 6

Both floor moves are reported rather than absorbed, and neither is a battery shrinking.

  • The roster grows by one because this PR declares one new battery, '#18263: the refusal says its reason, and says it where the API can read it' (35 cases). The floor pin is the roster's own size, so declaring a battery necessarily moves it; leaving it at 18 would let the new battery be deleted later with nothing going red. The mechanism was exercised in the process: the run before the roster entry existed failed with "registered 35 case(s) but is not declared in SELF_TEST_BATTERIES".
  • The #4690 / #7006 battery goes 5 → 6 because one assertion there was split into two. The old one was render(unreadable).stdout.length === 0; what check:react-declaration-parity 是唯一没接进任何 workflow 的源码审计门禁,且无 MANIFEST 时静默 skip 退出 0 —— 它现在永远不可能红 #4690 forbids on stdout is a tick, and the ::error:: annotation is the opposite of one, so the pin is now spelled as what it always meant — every stdout line must start with ::error , and there must be exactly one of them. It is strictly stronger than the line it replaces, not a relaxation.

Reverse verification — three ablations, each proving the new battery can fail

Each leg mutates the committed file, proves the mutation reached disk by an anchor count, runs the self-test, then restores with git checkout HEAD -- … and proves the restore by git hash-object against the HEAD blob. A trap … EXIT INT TERM carries the restore on the crash path. Predicted direction for all three: RED.

leg mutation on-disk proof self-test
A delete the ::error:: annotation from the not-measured-material lane anchor title: 'Check Changeset (level axis): no readable 1 → 0 exit 1, 9 failures
B collapse the three near-miss remedies into one shared sentence anchor Object.prototype.hasOwnProperty.call(CLAUSE2_NEAR_MISS_REMEDIES, reason) 1 → 0 exit 1, 3 failures
C make the unknown-reason path return an empty remedy injected marker 0 → 1, with the not-deleted control return ( held at 1 → 1 exit 1, 2 failures

The failures name themselves. Leg A reds nine cases including "a body that ALMOST declared and a body that never tried produce different text". Leg B reds "the three near-miss reasons owe three DIFFERENT remedies — one shared sentence would pass every assertion above while reading no reason at all". Leg C reds "a reason of null or undefined still returns a sentence — an empty remedy is this card's defect moved one function along".

Leg C's first attempt was a proven no-op and its reading was discarded, not retried quietly. Its anchor was the text the mutation inserts a copy of, so grep -c read 1 before and 1 after and the harness refused to read a self-test result it could not prove had run. It was re-run with an injected unique marker (0 → 1) and a control on the text that must NOT vanish. The first attempt produced no reading at all; the row above is the second.

Restores are proven, not assumed: each leg ends with git checkout HEAD -- … (never a bare git checkout --, which would take the mutation back out of the index) and then git hash-object against the HEAD blob af36b25de84a4e55c34ba323c83097c61a3f474c, plus git diff HEAD empty and the injected marker counted back to 0. All ran in a throwaway detached worktree off this branch's commit, since the file under test is the file the ablation mutates; that worktree is removed.

Scope, changeset and labels

skip-changeset, measured rather than assumed. The diff is one file, scripts/check-changeset-no-major.mjs. Resolving every tracked package.json (83 tracked, 70 publishable — not private and carrying a files[]) and asking which files[] entry would ship that path: zero. Positive controls through the same resolver: packages/spec/dist/index.js resolves to @objectstack/spec's dist entry and packages/cli/dist/index.js to @objectstack/cli's, so the resolver does find a shipped path when one exists; packages/spec/src/index.ts correctly resolves to nothing. The repo-root manifest is private: true with no files. Nothing published moves, so this takes the label and not an empty changeset (workflow route 2), and route 0 does not apply — this PR touches no .changeset/*.md at all.

One measured correction to the card's reproduction table

The card's table gives PR #18959 as `Clause-②: no` on its own line, backtick-wrapped, reading {"kind":"near-miss","reason":"describing"}. Driven against origin/main today, that exact line reads {"kind":"declared","value":"no","arm":null} — a declaration, not a near miss. clause2LineDescribes's QUOTED-AND-CONTINUED tell fires only when the backtick span opened at the key continues past its closing tick; a span that closes with nothing after it is not describing. The five PR bodies have all since been corrected, so the historical bytes are no longer readable through the REST API and the exact line #18959 carried could not be recovered — most likely it carried trailing content after the closing tick, which is the describing shape and is covered by the #18946 row.

This narrows one row of the reproduction table; it moves nothing about the card. Both near-miss spellings the card names are reproduced above from the real reader, the third (inline-key) with them, and the remedy is unchanged.

Out of scope, noted here rather than filed

  • .github/workflows/pr-automation.yml is untouched. Its Require a changeset step's ::error:: is now pinned by this script's self-test, which is a new coupling in the direction the card wants: it is what keeps a missing-changeset refusal distinguishable from this script's.
  • The two other gates that run in the same job — check-empty-changeset.mjs and check-adr-0087-registration.mjs — refuse to the job log with no annotation of their own, exactly as this one did. Same shape, different surface, and out of this card's file surface.

Generated by Claude Code

`check-changeset-no-major.mjs` already computes why it is refusing — a
near-miss `Clause-②:` line, a declaration it could not read at all, a
declared `yes` with no package graded `minor` — and then prints it only to
the job log, which is not the check run. Measured on this repository: a
failing `Check Changeset` answers `output.title` = null with 0-byte
`summary`/`text` and carries one annotation, the runner's generic
`Process completed with exit code 1.` So a near miss and a missing
changeset are the same event from outside.

Two changes, both inside the script:

* every near miss now renders the REASON `readClause2Line` returned, the
  offending line, and the remedy that reason owes. The reason is
  interpolated, never matched against a list here, so a reason the reader
  grows later prints and names itself instead of falling through.
* every refusing lane emits exactly one `::error::` workflow command, so
  the diagnosis crosses the check-run boundary onto the annotation channel
  the run already carries. Greens emit none.

No workflow file is touched, and which bodies are ACCEPTED is unchanged:
`CLAUSE2_KEY_LINE` and `readClause2Line` are not edited and no verdict
moves. Only what the refusal SAYS changes.

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017ef78bLdybu3AffehKkhfk
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

domain:devx size/m skip-changeset PR has no user-facing published change; bypasses the changeset gate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[finding] Check Changeset fails with an entirely empty check-run output, so two different refusals are indistinguishable from the API

2 participants