docs(multi-agent,session-flow): fix repo-mode audit pilot findings and schedule a full sweep

Problem

A /multi-agent:audit-defaults repo pilot was run on 2026-10-10 against Claude Code 2.1.296. It covered 12 files in multi-agent and session-flow, used 23 agents, and every finding it confirmed was upheld 3-0 by a skeptic panel. It confirmed 12 findings. The 2 about per-spawn effort were fixed in #6913. The other 10 are model and pointer hygiene outside that PR's scope:

plugins/multi-agent/reference/config.md:31 (stale): best is listed as a role model, but the sub-agents docs list only sonnet, opus, haiku, fable and inherit as subagent aliases. Check whether best is accepted per invocation; if not, drop it from the role values and the frontier default, otherwise point the list at sub-agents#choose-a-model.
plugins/multi-agent/reference/config.md:59 (copied-current): point at sub-agents#choose-a-model and workflows#cost with an as-of date and recheck trigger. The CLAUDE_CODE_SUBAGENT_MODEL caveat applies only when neither the per-invocation model nor the frontmatter sets one.
plugins/multi-agent/skills/route/SKILL.md:81 (copied-current): same pointer and caveat.
plugins/multi-agent/skills/route/SKILL.md:27 (copied-current): point the family list at model-config#model-aliases.
plugins/session-flow/skills/orchestrate/SKILL.md:64 (copied-current): the model-resolution fact needs a sub-agents#choose-a-model pointer; the recorded pointer covers brief contents only.
plugins/session-flow/skills/orchestrate/SKILL.md:114 (unpointed-judgment): cite optimizing-for-cost-and-intelligence#orchestrator-strategy-delegate-bulk-work.
plugins/session-flow/skills/orchestrate/SKILL.md:118 (unpointed-judgment): cite optimizing-for-cost-and-intelligence#tune-effort.
plugins/session-flow/skills/orchestrate/SKILL.md:134 (copied-current): point "withheld from non-fork workers" at sub-agents#available-tools or the sources.md record.
plugins/session-flow/skills/orchestrate/SKILL.md:168 (stale): the anchor and size bands should match workflows#set-a-size-guideline.
plugins/session-flow/README.md:493 (hardcoded-model-pin): use the haiku alias instead of the full ID claude-haiku-4-5.

Proposed next step

Fix the 10 findings above, one PR per plugin.
Schedule a full /multi-agent:audit-defaults repo sweep as its own run. Today's target list holds 110 files in 31 areas that state effort and spawn claims, and at the pilot's rate that is roughly 700 agents. Run it in waves with a pilot, during a fresh plan-usage window.

Done when

The 10 pilot findings are done (#7990, #7991, both closed). What remains is the full sweep, run in waves with a pilot wave first (owner decision 2026-10-12).

/multi-agent:audit-defaults repo ran over every file in its repo-mode target list, in waves after a pilot wave, and the issue has a comment with the count of files covered, findings confirmed and findings refuted.
Each confirmed finding is fixed in the run's PR, or filed as its own issue linked from that comment.
ci-status is green on the PR.

Proposed work class: none

Size: Large
Complexity: Medium
Uncertainty: Knowable
Flags: none
Blocked by: none

Understood: the 10 pilot findings are fixed (#7990, #7991). What remains is the full /multi-agent:audit-defaults repo sweep over the 110-file target list, run in waves after a pilot wave. The run must then be reported in a comment with counts of files covered, findings confirmed and findings refuted. Each confirmed finding is fixed in a PR or filed as its own issue. The sweep needs a claude run, which the unattended issue lane can't do, so it falls to the owner's attended session. No missing information and no likely duplicates found.

_{Automated triage. A maintainer reviews it.}

Decision recorded. The owner approved applying independently verified recommendations on 2026-10-12.

Keep best as a role model value and in the frontier default? → Drop best from role values and the frontier default
Basis: https://code.claude.com/docs/en/sub-agents.md lines 351-357 (fetched 2026-10-12); https://code.claude.com/docs/en/model-config.md line 33; Agent tool schema in this session (model enum sonnet/opus/haiku/fable); plugins/multi-agent/reference/config.md:31,34 on origin/main
How to run the full repo-mode sweep? → Waves with a pilot wave first, in a fresh plan-usage window
Basis: docs(multi-agent,session-flow): fix repo-mode audit pilot findings and schedule a full sweep #6916 body 'Proposed next step'; judgment (spend only, reversible)
Keep best as a role model value and in the frontier default? → Drop best from role values and the frontier default, and remove it from resolve-roles.sh ALIASES in the same change so --session-model best is rejected and the caller omits it (treated as frontier). (added by verification)
Basis: origin/main plugins/multi-agent/scripts/resolve-roles.sh:61,135,298-302,327,335

A 2026-10-10 repo-mode audit pilot confirmed 10 model and pointer hygiene findings in multi-agent and session-flow and proposes a full ~700-agent sweep. Main has moved since (#6998, #7009): the README haiku pin is gone and several pointers now exist, so each finding needs re-checking before fixing.

Verification notes:

Implementation caveat for Q1: resolve-roles.sh:61 keeps best in ALIASES, which also validates --session-model (:135), and SESSION_FRONTIER is computed from the frontier list (:298-302). Dropping best from the frontier default while ALIASES still accepts --session-model best would make a best session count as non-frontier and let inherit fan-out workers bypass the guard. Remove best from ALIASES too (an unknown session already counts as frontier) or keep it in frontier.
Assumption that Workflow opts.model rejects best is unverifiable: the workflows page says agents follow the subagent model order (workflows.md line 453) without listing accepted values.

Verified by an independent fresh-context check (confirmed, risk low): gh issue view 6916 (no owner comment); origin/main plugins/multi-agent/reference/config.md:30,34; origin/main plugins/multi-agent/reference/defaults.yaml:4,14; origin/main plugins/multi-agent/scripts/resolve-roles.sh; origin/main plugins/multi-agent/skills/route/SKILL.md:24-30,74-84; git grep claude-haiku origin/main plugins/session-flow/README.md: no match

Admission to an agent lane stays with the work-class rules and owner ratification.

This was generated by AI during backlog shaping.

This issue is split into 3 pieces, each sized for one lane PR. Re-checked on origin/main as the decision comment requires: the README haiku pin is gone and the orchestrate choose-a-model, available-tools and size-guideline pointers now exist (session-flow orchestrate SKILL.md:189,209; context/sources.md:206,217,295), but best is still a role value and frontier default in config.md:31,34, defaults.yaml:4,23, resolve-roles.sh:30,66, setup SKILL.md:86 and drift-audit.js:200; split into the multi-agent best removal plus remaining pointers, the session-flow pointer recheck, and the paid full sweep.

filed: fix(multi-agent): drop best from the role model values and the frontier default (fix(multi-agent): drop best from the role model values and the frontier default #7990)
filed: docs(session-flow): recheck and point the orchestrate model and cost claims (docs(session-flow): recheck and point the orchestrate model and cost claims #7991)
not filed: chore(multi-agent): run the full audit-defaults repo sweep in waves: paid-run: about 700 agents over 110 files, waves with a pilot in a fresh plan-usage window (owner decision 2026-10-12)

Filed pieces are sub-issues of this one, and the paced queue admits them to the lane. Unfiled pieces wait on the blocker shown. The owner approved this split in an attended backlog-shaping session on 2026-10-18.

Released by the owner's HITL session (2026-10-24, C12): it waited only on a removed gate. Intake triage re-runs on it to set its fields, then admission takes it.

Needs a run of claude (eval, probe or /usage), which the issue lane cannot do (pr-pipeline.yaml:289-314). The owner's HITL session runs it on the operator machine; no owner action needed.

The triage sweep dispatched intake triage for this issue (attempt 1 of 2).

docs(multi-agent): drop the assess cross-checking case from this PR

docs(bugs): keep blocked-alias behavior behind the scan pointer

docs(review): state the opus pin as a choice, not a capability claim

docs: record the Agent-dispatch exception to the frontier binding

docs: keep the bare-audit naming rule a hedge, not a collision claim

docs(discovery): leave env and Explore model behavior to the pointers

docs(docs-hygiene): make the subagent-loading record links-only

docs(knowledge): state the spend bound as the skill's own choice

docs(harness-ops): leave the flagged-message setting to its pointer

Host run (#12067). This is the final wave of the /multi-agent:audit-defaults repo sweep: it covers the 44 held files, plus the files that joined the target list since wave 5. It ends with the final count.

Target list

list-targets.sh at origin/main e09dba5d92 lists 206 files in 47 areas. The four held groups are all on it: plugins/harness-config (8), plugins/plugin-quality (4), plugins/mutation-testing (2) and plugins/playbooks (32, in four entries).
Against wave 5's list at 0ad858ec05, 202 files:
- knowledge/.../vision-json-shapes.md dropped off after its fix.
- playbooks/reference/model-adaptation/fable-5-1.md dropped off.
- Six files joined: three new playbooks/skills/fable-5/context/ chapters, docs/conventions/pr-pipeline/stage-order-baseline.md, and two .changes/ fragments.
That leaves 43 of the 44 held files, plus 6 new files: 49 swept here. 157 swept earlier + 49 = 206 of 206.

What ran

Claude Code 2.1.316, multi-agent 0.14.6, subscription login.
claude -p --model opus --permission-mode auto in a capped scope launched the multi-agent:drift-audit workflow: maxConcurrent: 6, asOf: 2026-11-01, one wave at a time.
Any file with more than about 25 claim lines went in as its own entry.
- Wave K+L: 23 files in 12 entries, 93 agents, 27 minutes.
- Wave M+N: 26 files in 7 entries, 66 agents, 21 minutes.
- No area was skipped and no wave timed out.
Each confirmed finding's quoted upstream lines were searched for in raw pages fetched with lib/fetch-docs.sh on 2026-11-01. No quote was contradicted.
- 49 of 58: every double-quoted span was found verbatim.
- 3 differ from the page only in markup or quoting, and grep found their key sentence: boris/SKILL.md:121, boris/reference/autonomy.md:253 and boris/reference/orchestration.md:95.
- 4 quote in single quotes, which the script does not search. The three in opus-4-8.md were found by hand on the effort and model-config pages. boris/reference/customization.md:63 was not checked.
- 2 rest on models/overview, which is not in the docs index, so it was not read: boris/reference/orchestration.md:33 and :147.

Result for this wave: 59 found, 58 confirmed, 0 refuted, 1 unverified. The unverified one is playbooks/reference/prompt-caching.md:183, where the panel split. Its disposition rewords "strip none yourself" against the thinking page's switching-models exception.

Totals across all waves

"Refuted" here counts panel refutations, findings dropped by the raw re-check, and anchors the rendered page showed still resolve.

wave	files	confirmed	refuted
pilot (after raw re-check)	13	11	2
2	13	9	1
3	23 (3 capped, re-swept in wave 5)	17	2
4	14 (6 capped, re-swept in wave 5)	8	0
5	104 (9 of them the re-sweep)	58	9
6 (this run)	49	58	0
total	206 of 206 at `e09dba5d92`	161	14

The file column covers 158 distinct files through wave 5: its 216 entries, less the 9 re-sweeps and vision-json-shapes.md, which has since left the list. 157 of those are still on it. Add this wave's 49 to reach 206.

Where the confirmed findings went

Pilot through wave 5 (103): fixed in merged PRs docs(implementation): qualify model and effort pin claims #11573, docs(planning): anchor upstream pointers and label judgment #11627, docs(multi-agent): state what the resolver emits, not where agents run #11638, docs(performance): point the older Task tool name at its source #11671, docs(session-flow): point claims at sources and label judgment #11677, docs(testing): correct the opus claim and add model pointers #11678, docs(verification): repoint the usage record and add a model pointer #11679, docs(work-items): point model and userConfig claims at sources #11680, docs: point wave 4 model and workflow claims at upstream sections #11701, docs(discovery): point drifted upstream claims at their sections #11771, docs: point repository docs and lane prompts at current upstream sections #11772, docs(docs-hygiene): correct audit-derivability's subagent-loading claims #11774, docs(review): narrow the opus pin rationale and add upstream pointers #11775, docs(knowledge): narrow drifted upstream claims and drop a model pin #11776, docs(autonomy): point the routines terms and stop gate at upstream sections #11777, docs(harness-ops): date and point three drifted upstream claims #11783, docs(songwriting): qualify object-writing's model and cost claims #11784, docs(code-tidying): label batch-simplify routing fallbacks as local choices #11785, docs(instruction-placement): align MS-13 cells with the re-pinned quote #11786 and docs(session-flow): point depth-limit and task-list claims upstream #11790. The five eval-gated ones are in docs(skills): ship the five eval-gated #6916 drift fixes after a no-loss measurement #11823.
This wave (58):
- 5 have a fix in docs: point plugin-quality and mutation-testing claims at upstream sections #12110, which is open and not yet merged (plugin-quality and mutation-testing pointers).
- 53 are filed as docs(playbooks): fix the 53 drift findings from the held-files sweep #12106 (13 in harness-config, 40 in playbooks). Three of them change skill behavior and go through docs(skills): ship the five eval-gated #6916 drift fixes after a no-loss measurement #11823's eval-gated route.

Not covered

Main moved to 08e5938772 during the run. Its list is still 206 files, with two changes:
- The two .changes/ fragments swept here were released and deleted.
- Two files joined: docs/contracts/seam-phrasing/README.md and .changes/codebase-health/feat-6600-concise-audit-body-5d1e7a20.md. Neither is swept.
list-targets.sh lists .changes/ release fragments, although the skill says changelogs are left out: EXCLUDE matches CHANGELOG* only. Excluding .changes/ would make the list stop changing with every release.
harness-config/.../audit-instructions/reference/criteria.md has 338 claim-matching lines, so its single entry hit CLAIM_CAP (40). Claims after the first 40 were not read.
plugin-quality/agents/auditor.md:17-19 still says the opus pin runs "on a model at least as capable as the one that produced the work". Wave 5 confirmed that claim as stale in the five review agents (fixed in docs(review): narrow the opus pin rationale and add upstream pointers #11775), but this wave's panels did not raise it here.

Target labels: add none; remove none.

Final count for the repo-mode /multi-agent:audit-defaults sweep: 205 of 205 files on the target list are covered at 0e2b3005d6 (current main).

The 206 reported in docs: point plugin-quality and mutation-testing claims at upstream sections #12110 at e09dba5d92 included 2 .changes/ release fragments. fix(multi-agent): leave release fragments out of the repo-mode sweep targets #12178 drops those fragments from the target list, which leaves 204 files at that SHA. At 0e2b3005d6 the list is the same 204 plus docs/contracts/seam-phrasing/README.md, and no file dropped out. Until fix(multi-agent): leave release fragments out of the repo-mode sweep targets #12178 merges, main's unfixed list-targets.sh prints 207: those 205 plus the 2 fragments now on main.
docs/contracts/seam-phrasing/README.md was the last file to sweep, and it has 0 findings. It has one claim, at line 87, which records the Opus 5.5 and Sonnet 5.5 runs of the hand-off eval. That claim is a record of a past eval run, not upstream guidance. Both model IDs appear on the raw models/overview page, fetched with fetch-docs.sh on 2026-11-01.
- The Workflow tool was unavailable in that session, so the workflow's read and find stages ran by hand with the same agent types: multi-agent:drift-reader, then multi-agent:drift-checker.
- With 0 findings, there was no skeptic stage.
fix(multi-agent): leave release fragments out of the repo-mode sweep targets #12178 also changes the workflow's repo-mode source from about-claude/models/overview to models/overview. The old address is not in the platform docs index, so the fetch stage reported it unread. That is why 2 findings in docs: point plugin-quality and mutation-testing claims at upstream sections #12110 were not read against that page. A future run will read it.
Totals across all waves are unchanged by this file: 161 confirmed and 14 refuted.

Closing as completed. I checked each done-when item against origin/main a33d0b315f on 2026-11-01.

1. The sweep covered every file on the repo-mode target list, in waves after a pilot, and a comment gives the counts. Met.

The waves ran in this order: pilot (comment), wave 2 (comment), waves 3 and 4 with the pilot re-check (comment), then wave 5 and the final wave (comment).
The final count comment gives 205 of 205 files covered, 161 confirmed, 14 refuted.
Recount on origin/main a33d0b315f: the target filter of plugins/multi-agent/skills/audit-defaults/scripts/list-targets.sh, as fixed by #12178, selects 205 files. The set matches the one at 0e2b3005d6, the commit the count comment cites.
Disclosed limits. None of these undoes file-level coverage:
- The workflow reads at most 40 claims per entry (CLAIM_CAP). harness-config/skills/audit-instructions/reference/criteria.md has 338 claim lines, so lines after the first 40 were not read. Some earlier capped entries may also have lost later claims (final-wave comment, "Not covered").
- docs/contracts/seam-phrasing/README.md was swept by hand with the drift-reader and drift-checker agents, not through the workflow. It had 0 findings.
- One final-wave finding, playbooks/reference/prompt-caching.md:183, is unverified (its panel split). It is in neither count.

2. Each confirmed finding is fixed in a PR or filed as its own issue, linked from the comment. Met. The links are in the final-wave comment, "Where the confirmed findings went".

wave	confirmed	where they went
pilot	11	3 in #11573, 6 in #11627, and 2 skill-body cuts (`draft-goal-condition/SKILL.md:69`, `:76`) in #11823
2	9	8 in #11638, 1 in #11671
3	17	9 in #11677, 4 in #11678, 2 in #11679, 2 in #11680
4	8	all 8 in #11701, one row each
5	58	#11771, #11772, #11774, #11775, #11776, #11777, #11783, #11784, #11785, #11786, #11790; the 3 behavior changes (`discovery/skills/setup/SKILL.md:61`, the `audit-derivability` spot-test agent, `write-for-agents/SKILL.md:139`) in #11823
final	58	5 in #12110, 53 filed as #12106 (13 `harness-config` rows plus 40 `playbooks` rows); #12106 sends its 3 behavior changes through #11823
total	161	none unaccounted for

Wave 3's testing/skills/cleanup/SKILL.md:280 depended on :174 and needed no edit of its own. The :174 fix in #11678 covers it. #11823 and #12106 stay open as the follow-up issues; #12106 is split into #12107 and #12108.

3. ci-status is green on the PR. Met. The 21 fix PRs in the table and #12178 are merged, and ci-status concluded SUCCESS on each one's final head. #12178 also closed the final-wave slice #12067. Examples: #12110, #12178, #11573, #11772.

This was generated by AI.

Navigation Menu

Repository navigation

docs(multi-agent,session-flow): fix repo-mode audit pilot findings and schedule a full sweep #6916

Problem

Proposed next step

Done when

7 remaining items

Target list

What ran

Totals across all waves

Where the confirmed findings went

Not covered

Metadata

Assignees

Labels

Type

Fields

Projects

Milestone

Relationships

Development

docs(multi-agent,session-flow): fix repo-mode audit pilot findings and schedule a full sweep #6916

Description

Problem

Proposed next step

Done when

Activity

github-actions commented on Oct 11, 2026

kyle-sexton commented on Oct 13, 2026

kyle-sexton commented on Oct 18, 2026

kyle-sexton commented on Oct 24, 2026

kyle-sexton commented on Oct 25, 2026

github-actions commented on Oct 25, 2026

7 remaining items

kyle-sexton commented on Nov 1, 2026

Target list

What ran

Totals across all waves

Where the confirmed findings went

Not covered

kyle-sexton commented on Nov 2, 2026

kyle-sexton commented on Nov 2, 2026

Metadata

Metadata

Assignees

Labels

Type

Fields

Projects

Milestone

Relationships

Development

Issue actions