Skip to content

chore(context): move stories and history out of always-loaded skills (T4) - #369

Closed
MendixMau wants to merge 3 commits into
masterfrom
chore/stub-diet-moves
Closed

MendixMau wants to merge 3 commits into
masterfrom
chore/stub-diet-moves

Conversation

@MendixMau

Copy link
Copy Markdown
Owner

Direct lane

What changed and why (one paragraph):
T4 from the stub-diet research report. This is a trim PR under CLAUDE.md "Always-loaded skill text — the word budget": it only moves movable text. Incident narratives, old measurements and a duplicated A/B paragraph move verbatim out of four baseline skills into process/baseline-incident-history.md, following the #315 precedent. They land as new sections UI1–UI4, TO1–TO3, MF1 and MP1, and each source keeps a one-line pointer. Rules, STOP rows and their numbers, denominators, completion criteria and the one-line failure examples stay in place. Where I wasn't sure, I left the text where it was. A new bin/check-moved-text.sh makes the mechanical half of the #315 review a command: every line removed from the sources must be found with grep -F in its new home.

Word counts (LC_ALL=C bin/render-routing.sh --check, budget 80,000)

before (d16f578) after
Baseline 75,028 74,223 (−805)
Stage 5 mdl 56,236 55,431
Stage 5 review 44,818 44,492
Stage 5 gate 34,629 34,303
Stage 5 test 33,689 33,363
Stage 5 lead / ba / architect 27,862 / 27,950 / 26,037 27,536 / 27,624 / 25,711
Stages P–3, all helper roles 21,106 20,780

Per file: ui-preflight-pages 3,933 → 3,553 · tool-output-is-not-ground-truth 2,888 → 2,562 · microflow-preflight 1,871 → 1,814 · learned-mdl-preflight 3,876 → 3,834.

Below the ~1.5k target, on purpose. The report's estimates for the mxcli layout block (~250) and the #879 note (~150) counted rule sentences. Once each sentence was classified, only ~57 and ~42 words of story were left in those two blocks. The ui-preflight delivery rule, with its A/B numbers, stays in place as instructed.

No-loss check:
bin/check-moved-text.sh process/baseline-incident-history.md skills/ui-preflight-pages.md skills/tool-output-is-not-ground-truth.md skills/microflow-preflight.md skills/learned-mdl-preflight.md
→ 83 of 83 removed non-blank line(s) found verbatim. Negative control: I deleted one moved line from the destination, and the script exited 1 and named that line.

Two edits that are not byte-verbatim:

  • In TO1 and TO3, the moved ##/### headings are demoted one level so they don't collide with the history file's section ids. This is noted on each section's _From_ line. grep -F still finds them.
  • Where a kept rule sentence shared a wrapped line with a moved sentence, I rewrapped the kept sentence in the source. The original line is in the history file verbatim.

Classification per moved block

ui-preflight-pages.md

Text Class Disposition
Delivery rule at top, with its A/B numbers ("a citation is not a read", dispatch must inline) rule + failure example kept
Step 2: "Measured 2026-09-09: 55 tokens…" paragraph story / old numbers moved → UI1; kept a one-line failure example: "0 of 35 knobs, 0 of 20 classes, while mx check, lint and e2e stayed green"
Step 2 B0 rule, report block with denominators, "binding UNPROVEN… silence is not" rule / denominator kept
Step 2 page column, "measured 2026-08-31: 10/10 drafts shipped full-bleed" one-line failure example kept
Step 5: "Measured, ToeicBuddy field run 2026-08-25/26" paragraph old numbers moved → UI2 (the Step 5 intro "wrong shell on ten pages out of ten" stays as the failure line)
"A fidelity score is not a LOOK pass…", "closes nothing… 1280px and 390px… never in place of one" rule kept
Fidelity vs. element-checklist table and its explanation old numbers moved → UI3; kept the one-liner "two screens scored 100% with 58%/40% of elements in place"
"Measured again, controlled A/B, 2026-08-31" duplicate of the top delivery rule moved → UI4
"The build script… cited this file's Step 3 by name…" story / justification moved → UI4
Stub-page rule, "Every run is stored", the v0.20 layout capability paragraph, report-back block rule / denominator kept

tool-output-is-not-ground-truth.md

Text Class Disposition
Instances 1–3: narratives ("saw 6 microflows instead of 13…", "kept flipping…", "user pushed back…", "two compounding errors", "all risk, zero benefit") story moved → TO1; each instance keeps a one-line failure summary
Antidote 1 (hash compare, re-run the query) and Antidote 2 (run T in isolation, instrument) rule kept, unchanged
Instance 3 lesson: "a tool that does not understand a sibling property cannot be trusted to round-trip a child one" rule kept (rewrapped)
Part 2 $? "Real instances" table old numbers moved → TO2; the WRONG/RIGHT code blocks and "biased toward green" stay
Part 2 constructs table, read-back rule, "count of zero and an exit code" WRONG/RIGHT rule + failure example kept
Part 3 "The instance" bullets and "Why it cost a day" story moved → TO3; kept a 4-line failure summary
Part 3 quote "A control that cannot distinguish your arms…", the rule, negative control, sensitivity ladder table, the tell, twin diff rule kept

microflow-preflight.md

Text Class Disposition
"mxcli layout is a domain-model command"; "moves entities only, no microflow mode" rule kept
"The mxcli team's answer… (2026-09-25)…", --help quote, "Measured: … 10 entity moves" history / old numbers moved → MF1
"Before relying on it… probe your binary… no @position, SUB_ split" rule kept, unchanged

(learned-microflow-patterns.md has no mxcli layout history, so it is untouched.)

learned-mdl-preflight.md, #879 note (one line; STOP rows untouched)

Text Class Disposition
"RESOLVED in v0.21.0, STOP LIFTED (probed 2026-09-10)" status kept
"Install no longer shells out… MPR v2 survives" history moved → MP1
"opt-in behind --allow-format-change, so the failure can only happen on purpose" rule kept
"Measured on a real 501-unit project: …" old numbers moved → MP1
"Read the #879 entry before your first install… three new operational traps (…)" rule / denominator kept

Field evidence: none yet for the moved text, because this PR changes no behaviour, only where the text sits. The behaviour A/B is owed (below). The field run for check-moved-text.sh is this PR, with 83/83 lines found and a hand-run negative control.

Owed before merge

  • Behaviour A/B (required). Use the method in process/preflight-skill-baseline-2026-08-31.md: 5 reps per arm, master skills against this branch's trimmed skills. Run one page task and one microflow task with a loop. Score check-page-shell.sh violations, page-fidelity.js --no-log, the report block with denominators, write mode per operation, the microflow-preflight checklist being posted, and refusal of a seeded STOP operation. Pass if no metric is worse than n=5 noise. Not run in this PR.
  • No fixture exists yet for bin/check-moved-text.sh. It was verified by hand with a positive run and a negative control.

Checklist

  • No client data in the diff (leak guard clean; whole-word denylist grep of added lines: 0 hits)
  • Size cap: 6 files, 354 lines excluding CHANGELOG
  • Test tier: T0 + T1 + render-routing --check. test-routing-tier.sh 12 ok / 0 FAIL, test-baseline-pack.sh 16 ok / 0 FAIL, check-skill-routing.sh all pass. No fixture greps the moved text: I checked tests/ for the moved phrases, and the fixtures that name these skills (bug06, bug12, context-audit, context-pack) match on file names only.
  • New instrument: no golden capture applies (it reads git diff); bash 3.2-portable (check-portability clean); a fixture is still owed (above)
  • CHANGELOG.md line appended, credited
  • Also clean: check-scripts.sh (111/111), check-no-private-citations.sh, check-pr-discipline.sh origin/master

Possible conflicts: PR #355 (STOP-table markers in learned-mdl-preflight.md) and the tiers PR (testing-shape.md). This PR changes only the #879 version-note line in learned-mdl-preflight.md, and does not touch testing-shape.md.

🤖 Generated with Claude Code

https://claude.ai/code/session_01VJgWP5vEoAsNsJYqCDMGNw


Generated by Claude Code

…(T4)

Verbatim moves into process/baseline-incident-history.md (UI1-UI4, TO1-TO3,
MF1, MP1) with a one-line pointer in each source: ui-preflight-pages.md,
tool-output-is-not-ground-truth.md, microflow-preflight.md and
learned-mdl-preflight.md. Rules, STOP rows and their numbers, denominators
and one-line failure examples stay in place. Baseline 75,028 -> 74,223 words.

New bin/check-moved-text.sh: every line removed from the sources must be
found with grep -F in the destination (83 of 83 here).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VJgWP5vEoAsNsJYqCDMGNw
…ew follow-ups (#369)

- A mistyped or unchanged source path, or 0 removed lines in total, now
  exits 2 instead of printing "0 of 0" and exiting 0.
- Header documents the grep -F substring limit; lines under 15 characters
  are listed as "trivially found, review by hand".
- New tests/wave2/test-check-moved-text.sh: found / missing (exit 1, line
  named) / special characters / typo path and unchanged source (exit 2) /
  usage. 10 ok here; 4 FAIL against the previous script version.
- microflow-preflight.md: the mxcli layout claim is dated "(as of 2026-09-25)".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VJgWP5vEoAsNsJYqCDMGNw
…es (leak guard)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VJgWP5vEoAsNsJYqCDMGNw
MendixMau pushed a commit that referenced this pull request Oct 14, 2026
…aded skills (T4)

Review: independent review MERGE (stub moves; durable-text moves only, no deletions). Behaviour A/B (process/preflight-skill-baseline method, sonnet reps, packs read in full): arm T = #369+#378+#380 trimmed mdl context (33,274 words) vs arm M = master (60,041). Drafting 5 reps x 2 tasks: planted STOP refused 10/10 both arms; scan 0 both; mxcli check PARSE page T 2/5 vs M 3/5, mf T 0/5; crumbs missing page T 1/5 vs M 2/5. Retrieval probes (5 re-tiered triggers x 3 reps): T probe_first 15/15. Decision rule (T no worse than M on every metric) met. Side finding: 3 of 5 M-arm probe agents skipped most of the 4,822-line pack.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VJgWP5vEoAsNsJYqCDMGNw
MendixMau pushed a commit that referenced this pull request Oct 14, 2026
…its copy of the baseline table (stub diet T2+T3)

Review: independent review MERGE-AFTER-FIX — full pack double-counted the runbook (85,797 > 80,000 budget); runbook-slice dropped the gate-verdict table and §3b helper-ownership rule → both fixed (anchor-based spans with fallback pointer), in 760a423; confirmation MERGE: baseline 73,233/80,000 (master 75,028), Stage 5 mdl 56,236 → 43,599; fixtures routing-tier 14, runbook-slice 31, baseline-pack 22, render-routing 5, bug03 31, bug12 61, ritual-slim 17, check-skill-routing pass. Behaviour A/B (with #369/#380, 5 reps × 2 tasks + 5 probes × 3 reps, sonnet): trimmed context no worse than master on every scored metric (planted STOP 10/10 each arm, parse failures page 2/5 vs 3/5, probes 15/15).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VJgWP5vEoAsNsJYqCDMGNw
MendixMau added a commit that referenced this pull request Oct 15, 2026
…journey controls, fixes) (#385)

Lands #340 #363 #362 #355 #370 #368 #364 #365 #366 #372 #367 #371 #376 #369 #373 #377 #378 #380.

Review: each constituent PR independently reviewed by a fresh agent that did not write it; merged into integration/2026-10-15 only on a MERGE verdict with green CI (per-PR Review lines in the integration merge commits). After-fix rounds: #373 (env-var cap, rename/skip-loops freshness), #377 (inline URI fallback), #378 (full-pack double count, slice content). Stub diet (#369/#378/#380) behaviour A/B, 5 reps x 2 drafting tasks + 5 retrieval probes x 3 reps: trimmed context no worse than master on every scored metric (planted STOP refused 20/20, parse failures page 2/5 vs 3/5, probes 15/15); mdl Stage 5 56,236 -> 30,647 words, baseline 75,028 -> 73,233. Full suites on c9bc073: run-tests 33/0, wave2 111 fixtures 0 failing. Field evidence: none yet for the instruments (#355 warn-only, #373/#377 read-only).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VJgWP5vEoAsNsJYqCDMGNw
@MendixMau

Copy link
Copy Markdown
Owner Author

Landed on master in #385 (squash 868b3cb), after its own independent review on the night integration branch. Closing this PR.


Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants