Skip to content

new(journey-proof): named controls prove each journey's own claim can fail (#159) - #368

Closed
MendixMau wants to merge 2 commits into
masterfrom
feat/named-controls
Closed

new(journey-proof): named controls prove each journey's own claim can fail (#159)#368
MendixMau wants to merge 2 commits into
masterfrom
feat/named-controls

Conversation

@MendixMau

@MendixMau MendixMau commented Oct 14, 2026 •
edited
Loading

Copy link
Copy Markdown
Owner

Closes #159.

Direct lane

What changed and why (one paragraph):
--positive-control proves the harness can fail. It does not prove that the claim a journey exists for can fail. In the issue's field case, a journey passed every rung, its positive control proved 7 of 7, and its claim held only by accident. This PR adds project-tests/e2e/named-controls.js (58 lines), ported from the behaviour the issue specifies and added to MXTK_PROJECT_TESTS so install-tests.sh ships it. It runs control files named *.control.json, each declaring "control": {"of", "mustFail"}, through the runner with --out=<artifacts>/control-<id>.json. A control reads PROVEN only when all three hold: the runner exits 1; every check whose name contains mustFail is FAIL, with at least one; and nothing else is FAIL or INVALID. Two cases are a FAULT and do not run: a persona other than the run user, and a file not named *.control.json. A stale findings file is deleted before each run. The suffix is required because verify-module, report-normalize.js and fixture-manifest.sh all discover journeys as *.journey.json. A control saved under that name would be walked, normalized and seeded as a real journey. journey-runner.js gains --out=<file>, which creates its parent directory. The issue depends on this flag from #148, which is not yet on master, so this PR adds only that flag. The runner also prints a warn-only WARN named control: line for a journey with no claim, or with no *.control.json beside it or in controls/. The same list is written as namedControlGaps. It adds no row and does not change the exit code; it stays warn-only until a field run. skills/journey-proof.md gains the rule "every journey names its claim and ships a control", the naming rule, a one-line failure example and a checklist line.

Field evidence: none yet for this port. The behaviour comes from the issue author's card-disbursement requirements-driven build. There, the project's own named-controls.js found a claim that held by accident, and after the fix it read 5 of 5 PROVEN. The toolkit copy has not been run against a live app.

  • No client data anywhere in the diff (leak guard, private-citation check and word grep are clean)
  • Size cap: 6 files, about 130 lines changed outside fixtures and CHANGELOG
  • Test tier reached: T1.
    • New fixture bash tests/wave2/test-named-controls.sh project-tests/e2e/named-controls.js: 24/24. Its cases are built from the captured walk fixtures/journey-control/walk-run.json.
    • Coverage: the three PROVEN rules, the persona FAULT, the misnamed-control FAULT, the --out= handoff and the stale-file guard.
    • The gap check is covered, including that a control saved as *.journey.json or as a bare .json in controls/ does not count.
    • One case proves that a *.control.json is not discovered as a journey by the patterns verify-module, report-normalize and fixture-manifest use.
    • A real runner preflight abort writes to an --out= path whose directories did not exist, and prints the WARN lines.
    • Hand mutations make it go red: rule 3 removed gives 2 FAIL; --out=/WARN removed gives 3 FAIL; suffix filter and mkdir removed gives 6 FAIL.
    • Existing runner fixtures pass unchanged: control-exit 7/7, persona 4/4, preflight 75/75, liveness 36/36, otel-capture-until 4/4, rung4-scope 28/28.
    • No T2 field run.
  • Golden input is captured, not hand-written (the card-disbursement walk). Paths go through config.js/path.resolve, with no new path roots. spawnSync(process.execPath, …) uses no bare wrapper name.
  • New skill: n/a
  • CHANGELOG.md line appended, credited to MendixMau (card-disbursement requirements-driven build)
  • Bug entries: n/a

Not done: the issue's optional obligations.tsv denominator ("journeys with a PROVEN named control") and the testing-shape.md §6 mention. render-routing --check is clean. The baseline is 75,028 words before and after, because journey-proof.md is on-demand and not counted in it.

🤖 Generated with Claude Code

https://claude.ai/code/session_01VJgWP5vEoAsNsJYqCDMGNw

claude added 2 commits October 14, 2026 14:09
@claude
new(journey-proof): named controls prove each journey's own claim can…
62e3784 
… fail (#159)

--positive-control proves the harness can fail, not that a journey's claim can.
Port named-controls.js: a control file (control.of / control.mustFail) is PROVEN
only when the runner exits 1, every check named by mustFail is FAIL (at least
one), and nothing else is FAIL or INVALID; a persona other than the run user is
a FAULT and is not run. journey-runner.js gains --out=<file> and a warn-only
"WARN   named control:" line for a journey with no claim or no control file.
journey-proof.md gains the rule. Fixture tests/wave2/test-named-controls.sh,
19 cases over the captured card-disbursement walk. No field run of this port.

Closes #159. Source: MendixMau, card-disbursement requirements-driven build.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VJgWP5vEoAsNsJYqCDMGNw
@claude
fix(journey-proof): controls are named *.control.json; --out= creates…
c7c4fd6 
… its directory (#368 review)

A control saved as *.journey.json would be walked, normalized and seeded as a
real journey, because verify-module, report-normalize and fixture-manifest
discover journeys by that suffix. The runner's gap check and named-controls.js
now accept only *.control.json (beside the journey or in controls/);
named-controls.js FAULTs any other name without running it. journey-runner.js
creates the parent directory of an --out= path. journey-proof.md names the
suffix and why. Fixture test-named-controls.sh 19 -> 24 cases: misnamed
controls do not count, a *.control.json is not discovered as a journey, and
--out= into a new nested directory.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VJgWP5vEoAsNsJYqCDMGNw
MendixMau pushed a commit that referenced this pull request Oct 14, 2026
@claude
Merge #368: new(journey-proof): named controls prove each journey's o…
b46e99e 
…wn claim can fail (#159)

Review: independent review MERGE-AFTER-FIX — control-file suffix unspecified (a *.journey.json copy would run as a real journey); --out= did not create its parent dir → controls must be *.control.json (skill, runner findControls, named-controls.js FAULT otherwise), --out= mkdir -p, in c7c4fd6; confirmation MERGE: real journeys cannot be swallowed as controls, existing *.journey.json discovery unaffected. CHANGELOG credit wording checked: generic, already used on master. Fixtures: named-controls 24/0, control-exit 7/0, persona 4/0, preflight 75/0, liveness 36/0. Warn-only gap check; no field run. Closes #159.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VJgWP5vEoAsNsJYqCDMGNw
@MendixMau MendixMau mentioned this pull request Oct 14, 2026
Merged
5 of 7 tasks
MendixMau added a commit that referenced this pull request Oct 15, 2026
@MendixMau @claude
Night batch 2026-10-15: 18 reviewed PRs (stub diet, app-facts sweep, …
868b3cb 
…journey controls, fixes) (#385)

Lands #340 #363 #362 #355 #370 #368 #364 #365 #366 #372 #367 #371 #376 #369 #373 #377 #378 #380.

Review: each constituent PR independently reviewed by a fresh agent that did not write it; merged into integration/2026-10-15 only on a MERGE verdict with green CI (per-PR Review lines in the integration merge commits). After-fix rounds: #373 (env-var cap, rename/skip-loops freshness), #377 (inline URI fallback), #378 (full-pack double count, slice content). Stub diet (#369/#378/#380) behaviour A/B, 5 reps x 2 drafting tasks + 5 retrieval probes x 3 reps: trimmed context no worse than master on every scored metric (planted STOP refused 20/20, parse failures page 2/5 vs 3/5, probes 15/15); mdl Stage 5 56,236 -> 30,647 words, baseline 75,028 -> 73,233. Full suites on c9bc073: run-tests 33/0, wave2 111 fixtures 0 failing. Field evidence: none yet for the instruments (#355 warn-only, #373/#377 read-only).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VJgWP5vEoAsNsJYqCDMGNw
@MendixMau

MendixMau commented Oct 15, 2026

Copy link
Copy Markdown
Owner Author

Landed on master in #385 (squash 868b3cb), after its own independent review on the night integration branch. Closing this PR.

Generated by Claude Code

@MendixMau MendixMau closed this Oct 15, 2026
@MendixMau
MendixMau deleted the feat/named-controls branch October 15, 2026 10:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

journey --positive-control proves the harness can fail, not that the journey's own business claim can: add named controls

2 participants

@MendixMau @claude