Skip to content

chore(context): per-role on-demand token in the routing agents column (stub diet T1) — stacked on #378 - #380

Closed
MendixMau wants to merge 9 commits into
masterfrom
chore/stub-diet-per-role-tier
Closed

MendixMau wants to merge 9 commits into
masterfrom
chore/stub-diet-per-role-tier

Conversation

@MendixMau

Copy link
Copy Markdown
Owner

Stacked on #378 (chore/stub-diet-runbook). This branch contains #378's two commits. Review only the top commit (a406512) until #378 merges; then this diff shrinks to that commit.

Direct lane

What changed and why (one paragraph):
This is item T1 of the stub-diet context-cost research (r-stub-diet). The tier column in bin/lib/skill-routing.tsv applies to the whole row. So a row that one role needs every session, and another role needs once per module, had to be baseline for both. Example: the mdl helper never closes a module, pushes or writes a gate script, yet it read module-review, ui-loop, teamserver-alignment, learned-css-that-never-applied and skills-over-scripts every session.

The fix is a new agents-cell token, <role>? (for example mdl?, or all,mdl?). It makes a baseline row on demand for that one role:

  • Stub: the row stays in the role's stub, among the on-demand rows below the baseline rows, with the trigger only. It stays findable (CLAUDE.md skill rule 8).
  • Word count: the row leaves that role's always-read set and word count.
  • Other roles: every other role the cell names keeps the row at baseline.

Where it is handled:

  • _routing_role_tier is new; _routing_reaches and routing_baseline_pack respect the token.
  • The agent:<x> view places the row by its per-role tier.
  • The Role(s) column renders review (mdl: on demand) and every role except mdl, review (on demand for those). The CLAUDE.local.md and stub preambles each gain one clause saying what "on demand" means.
  • The BAD_AGENT audit checks the name part of a ? token. It refuses ? on a non-baseline row and mdl,mdl? (both forms of one role).

The token is applied as follows:

  • mdl: module-review, ui-loop, teamserver-alignment, learned-css-that-never-applied, skills-over-scripts.
  • review: teamserver-alignment, skills-over-scripts.

Word counts (LC_ALL=C bin/render-routing.sh --check):

master d16f578 after #378 (A) after this PR (A+B)
Baseline total (budget 80,000) 75,028 73,233 73,233 (unchanged: rows stay baseline for other roles)
Stage 5 mdl 56,236 43,352 30,400 (target ~31k)
Stage 5 review 44,818 31,934 30,018
Stage 5 gate / test 34,629 / 33,689 21,745 / 20,805 unchanged
Stage 6 mdl / review 32,983 / 39,723 19,875 / 26,615 9,969 / 24,699
Stage P–3 mdl / review 21,106 7,694–8,603 6,642–7,551

T4 (verbatim narrative moves, about 1.5k) is a separate PR. T5 (detection-gaps register) is untouched while it waits for a maintainer ruling.

Mechanical no-loss check: tests/bench/trim-noloss.sh chore/stub-diet-runbook HEAD (the bench added in #378). Result: 6 lines found elsewhere, 17 routing rows still routed, 0 left. Every removed stub line is a routing row re-rendered in its on-demand form.

Owed before merge: behaviour A/B (maintainer's session, measurement plan from r-stub-diet, process/preflight-skill-baseline-2026-08-31.md method):

  • Fresh Sonnet drafting agents, 5 reps per arm. One arm gets the master stub and skills, the other gets A+B. Two tasks: one page task, and one microflow task with a loop.

  • Score each run on:

    • shell violations (check-page-shell.sh)
    • page-fidelity.js --no-log
    • the report block with its denominators
    • write mode per operation
    • whether the microflow-preflight checklist is posted
    • whether a seeded STOP operation is refused

    Pass if no metric is worse than n=5 noise.

  • Retrieval probe (skill rule 8), specific to this PR: seed each re-tiered trigger, for example "the style change did nothing" (learned-css), "close this module" (module-review), "the page looks wrong" (ui-loop) and "push to Team Server" (teamserver-alignment). The mdl helper must open the right file in 5 of 5 reps.

  • Start tokens per helper from bin/context-audit.sh on the next field build.

Field evidence: none yet.

  • No client data anywhere in the diff
  • Size cap: the top commit changes 13 files and about 196 lines. Excluding tests and CHANGELOG it is 11 files, and 7 of those are generated re-renders (README, ROUTING, agent stubs).
  • Test tier: T0, plus T1 scoped fixtures, plus render-routing --check. All green: test-routing-tier 26/0 (new cases T6–T8; 9 FAIL against chore(context): runbook slice for helpers; runbook drops its copy of the baseline table (stub diet T2+T3) #378's lib as the negative control), test-baseline-pack 16/0, test-render-routing 5/0, test-pipefail-grep-q 3/0, test-runbook-slice 21/0, test-ritual-slim 17/0, test-bug12-sync 61/0. Also clean: bin/check-skill-routing.sh, check-scripts, check-portability, the leak guard, private citations and pr-discipline. T2 field run: none.
  • No new instrument; renderer/table change only, bash 3.2 / POSIX awk
  • CHANGELOG.md line appended, crediting r-stub-diet
  • No bug entries

🤖 Generated with Claude Code

https://claude.ai/code/session_01VJgWP5vEoAsNsJYqCDMGNw


Generated by Claude Code

…the baseline table

T2: skills/conversion-runbook.md is no longer a render-routing surface; its rendered
baseline table (a fourth copy) is replaced by a one-line pointer to CLAUDE.local.md and
ROUTING.md.

T3: bin/runbook-slice.sh <stage> prints §1b plus that stage's section. The span logic
moves from gate-check.sh into bin/lib/runbook-span.sh, shared by both and by the per-role
count. The runbook routing row is split: lead/ba/architect keep the runbook,
mdl/gate/test/review get the slice; routing_baseline_pack counts the slice's printed words.

Adds tests/wave2/test-runbook-slice.sh and tests/bench/trim-noloss.sh; updates
test-routing-tier.sh.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VJgWP5vEoAsNsJYqCDMGNw
… (stub diet T1)

An agents-cell token "<role>?" makes a baseline row on demand for that one role:
it stays in the role stub among the on-demand rows, leaves its always-read set and
word count, and the Role(s) column names it. Handled in _routing_reaches,
routing_baseline_pack, the agent view, the Role(s) column and the BAD_AGENT audit.
Applied for mdl (module-review, ui-loop, teamserver-alignment,
learned-css-that-never-applied, skills-over-scripts) and review
(teamserver-alignment, skills-over-scripts). Cases T6-T8 in test-routing-tier.sh.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VJgWP5vEoAsNsJYqCDMGNw
MendixMau pushed a commit that referenced this pull request Oct 14, 2026
…aded skills (T4)

Review: independent review MERGE (stub moves; durable-text moves only, no deletions). Behaviour A/B (process/preflight-skill-baseline method, sonnet reps, packs read in full): arm T = #369+#378+#380 trimmed mdl context (33,274 words) vs arm M = master (60,041). Drafting 5 reps x 2 tasks: planted STOP refused 10/10 both arms; scan 0 both; mxcli check PARSE page T 2/5 vs M 3/5, mf T 0/5; crumbs missing page T 1/5 vs M 2/5. Retrieval probes (5 re-tiered triggers x 3 reps): T probe_first 15/15. Decision rule (T no worse than M on every metric) met. Side finding: 3 of 5 M-arm probe agents skipped most of the 4,822-line pack.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VJgWP5vEoAsNsJYqCDMGNw
… verdicts and §3b ownership

Review findings on #378:
- routing_baseline_pack counted bin/runbook-slice.sh on top of skills/conversion-runbook.md
  whenever both rows reached the pack (no role). The no-role full pack read 85,797 words, over
  the 80,000 budget, on gate-check's advisory line. The slice now counts 0 when the runbook row
  is already counted; full pack 73,233. test-baseline-pack asserts it on a scratch table and on
  the real one.
- runbook-slice.sh now also prints the gate-verdict table (five verdicts + exit codes) and
  §3b's "Ownership is split" paragraph (a helper reports BRD-touching decisions to the lead),
  located by heading/anchor in lib/runbook-span.sh. test-runbook-slice T6 asserts both for
  every stage. mdl at Stage 5: 43,599 words.
- CHANGELOG: behaviour A/B result replaces "owed before merge".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VJgWP5vEoAsNsJYqCDMGNw
…duplicate the merge left

The merge of #378's review fixes kept both the old and the updated copy of #378's CHANGELOG
line (the adjacent T1 line differs between the branches). Remove the stale copy and replace
"Behaviour A/B owed before merge" on this PR's own line with the result.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VJgWP5vEoAsNsJYqCDMGNw
… slice gained verdicts + §3b

Measured with bin/render-routing.sh --check on this head (master: 56,236; baseline
75,028 → 73,233 unchanged). The line carries no review figure.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VJgWP5vEoAsNsJYqCDMGNw
… review 32,181 → 30,265)

Measured with bin/render-routing.sh --check on #378's head (760a423) and on this head. Also
drops the stale copy of #378's line the merge left beside the updated one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VJgWP5vEoAsNsJYqCDMGNw
MendixMau pushed a commit that referenced this pull request Oct 14, 2026
…its copy of the baseline table (stub diet T2+T3)

Review: independent review MERGE-AFTER-FIX — full pack double-counted the runbook (85,797 > 80,000 budget); runbook-slice dropped the gate-verdict table and §3b helper-ownership rule → both fixed (anchor-based spans with fallback pointer), in 760a423; confirmation MERGE: baseline 73,233/80,000 (master 75,028), Stage 5 mdl 56,236 → 43,599; fixtures routing-tier 14, runbook-slice 31, baseline-pack 22, render-routing 5, bug03 31, bug12 61, ritual-slim 17, check-skill-routing pass. Behaviour A/B (with #369/#380, 5 reps × 2 tasks + 5 probes × 3 reps, sonnet): trimmed context no worse than master on every scored metric (planted STOP 10/10 each arm, parse failures page 2/5 vs 3/5, probes 15/15).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VJgWP5vEoAsNsJYqCDMGNw
MendixMau pushed a commit that referenced this pull request Oct 14, 2026
…gents column (stub diet T1) — stacked on #378

Review: independent review MERGE — `<role>?` on-demand routing token; audit refuses all?, ? on non-baseline rows, and plain+? duplicates; the five re-tiered skills hold no rule mdl needs before drafting. Confirmation MERGE at 966caff (stacked on #378 via merge commits): Stage 5 mdl 43,599 → 30,647, review 32,181 → 30,265, baseline 73,233; fixtures routing-tier 26, runbook-slice 31, baseline-pack 22, render-routing 5, bug03 31, bug12 61, ritual-slim 17. Behaviour A/B as on #378: no worse than master on every metric. Follow-up: completed agent stubs in existing projects are never re-rendered, so they don't get the saving.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VJgWP5vEoAsNsJYqCDMGNw
MendixMau added a commit that referenced this pull request Oct 15, 2026
…journey controls, fixes) (#385)

Lands #340 #363 #362 #355 #370 #368 #364 #365 #366 #372 #367 #371 #376 #369 #373 #377 #378 #380.

Review: each constituent PR independently reviewed by a fresh agent that did not write it; merged into integration/2026-10-15 only on a MERGE verdict with green CI (per-PR Review lines in the integration merge commits). After-fix rounds: #373 (env-var cap, rename/skip-loops freshness), #377 (inline URI fallback), #378 (full-pack double count, slice content). Stub diet (#369/#378/#380) behaviour A/B, 5 reps x 2 drafting tasks + 5 retrieval probes x 3 reps: trimmed context no worse than master on every scored metric (planted STOP refused 20/20, parse failures page 2/5 vs 3/5, probes 15/15); mdl Stage 5 56,236 -> 30,647 words, baseline 75,028 -> 73,233. Full suites on c9bc073: run-tests 33/0, wave2 111 fixtures 0 failing. Field evidence: none yet for the instruments (#355 warn-only, #373/#377 read-only).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VJgWP5vEoAsNsJYqCDMGNw
@MendixMau

Copy link
Copy Markdown
Owner Author

Landed on master in #385 (squash 868b3cb), after its own independent review on the night integration branch. Closing this PR.


Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants