[kyle-sexton]

Closes #11540

Summary

Ships the rewritten /ai-slop:audit description held back from #11169. It names commit messages, PR bodies, interface copy and generic headlines beside markdown, so requests about those texts reach the skill. The before/after trigger measurement that #11540 asks for found no lost trigger and a held-out gain on both Opus and Sonnet.

Fix

• plugins/ai-slop/skills/audit/SKILL.md description: the rewrite from 9128832ee3. The measurement found one confusion, so it also adds "Not for code comments (/code-tidying:audit-comment-residue)". To stay under the 1024 cap it trims two clauses, without dropping a trigger phrase.
• The changelog fragment states the measured result.

Verification

Measured per skill-criteria Measurement: Claude Code 2.1.316, --ablation none --runs 3 --disallow-tools Read Glob Grep (#11624), on Opus 5.5 and Sonnet 5.5. Probes are the committed probes/audit.json (19) plus a blind held-out set of 18 written from a neutral purpose statement, not committed. run-validity.py at current main judged every run VALID. Versions ran back to back, alternating, because the lane runs one eval at a time.

Opus: before is main, 3 rounds; after-v2 is this head, 2 rounds. Sonnet: 2 rounds each.

split	Opus before	Opus after	Sonnet before	Sonnet after
train positives	54/54	36/36	35/36	35/36
train negatives held	44/45	30/30	28/30	29/30
validation positives	36/36	24/24	22/24	20/24
validation negatives held	27/36	18/24	18/24	18/24
held-out positives	47/81 (58%)	45/54 (83%)	31/54 (57%)	38/54 (70%)
held-out negatives held	72/81	48/54	47/54	47/54

• The first after version, without the exclusion, drew neg-train-04 ("audit the code comments in src/ ...") from 6/6 held to 3/6 on Opus. With the exclusion it is 6/6. Restoring the routing closed the gap, so the cause is measured.
• Sonnet noise report (noise-report.py --baseline): non-inferior at margin 0.05 in both rounds (paired change +0.04 and +0.02). It flagged pos-val-03 (overused metaphors), 4/6 to 2/6. A 20-run-per-side re-run of that case alone gave before 10/20 and after 14/20, so it is not a drop. The other flagged cases moved in one round only and pooled equal or better.
• validation neg 0.75 is neg-val-01, a slop-config setup request that fires audit instead of /ai-slop:setup on both versions and both models. This change leaves it as it was.
• scripts/check-affected.sh: 13 passed. check-skill.sh: description under the cap.

Related

• Builds on fix(skill-quality): stop invocation-probe runs failing validity on denied searches outside the workspace #11624 (probe runs gate VALID) and fix(evals): warn on non-Skill denials in invocation-probe runs #11655.
• Replaces closed fix(ai-slop): name commit messages, PR bodies and interface copy in the audit description #11619, whose runs were INVALID.

🤖 Generated with Claude Code

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-11-01T04:08:15.473420Z	65f82a4	Draft marked ready

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[github-actions]

claude-security-review reviewed 6712cc5821034446e87cccab9d4f75cc535cbe84: read 1 of 1 changed file(s); 0 finding(s), 0 as inline comments.

Security review (incremental, 65f82a4..6712cc5)

Scope: .changes/ai-slop/fix-11540-audit-description-8e3b51c4.md is the only file changed since the last review. I read its whole change in .claude-lane/incremental.diff. I used the full diff and the PR context for background only.

Result: I found no security issues.

What changed: One sentence in the changelog fragment was reworded. It now gives the trigger measurement for Sonnet 5.5 next to the Opus 5.5 numbers. The bump: patch frontmatter is unchanged.

Why it has no security-relevant surface:

• The file does not match any trusted-path glob.
• It changes no trust control, plugin manifest, workflow, token scope or merge hold.
• It removes or weakens no standing instruction. It is a release-notes fragment, not an instruction surface.

PR context: It contains no embedded instructions aimed at the review lane.

Spend: none.

critical-findings: 0

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 65f82a4247

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[github-actions]

claude-review reviewed 6712cc5821034446e87cccab9d4f75cc535cbe84: read 1 of 1 changed file(s); 0 finding(s), 0 as inline comments.

Code review: incremental (65f82a4 → 6712cc5)

Scope: I reviewed the one hunk of .changes/ai-slop/fix-11540-audit-description-8e3b51c4.md in .claude-lane/incremental.diff. I used the full diff and the PR context only as background. No other file changed since the last review.

What changed: The changelog fragment now reports the Sonnet 5.5 measurement beside the Opus 5.5 one. Held-out triggering rose from 57% to 70% on Sonnet, and the fragment still gives Opus as 58% to 83%. The text also now reads "on Opus 5.5 and Sonnet 5.5, every run VALID".

Checks:

• The numbers match the PR body's table. Opus held-out is 47/81 (58%) before and 45/54 (83%) after. Sonnet held-out is 31/54 (57%) before and 38/54 (70%) after. The percentages round correctly. These figures are the author's; I could not re-run the measurement.
• The "no lost trigger" claim also covers Sonnet now. Sonnet's validation positives fell from 22/24 to 20/24. The PR body explains the drop: all of it is pos-val-03, and a 20-run-per-side re-run of that case gave 10/20 before and 14/20 after. The Sonnet noise report also found the change non-inferior at margin 0.05 in both rounds. That support is reasonable, so I'm not flagging the claim. It is the author's evidence and I have not checked it myself.
• The bump: patch level is unchanged. It is still right for a description-only fix.
• Closing issue ai-slop: measure the audit description rewrite before shipping it #11540: this increment leaves both "Done when" items as the earlier review judged them. The before/after run is reported in the PR body, and the rewritten line ships because the measurement found no loss.
• Path-scoped rules: none of the .claude/rules/ globs match .changes/**.
• No plugin manifest changed and no paved-path concern applies.
• I found no embedded instructions in the PR context.

No findings.

[kyle-sexton]

Arm request for the hub (R10), host-run lane 4, 04:45Z: head 6712cc5 has ci-status green from a non-draft run. Both review lanes report 0 findings on that head, no review thread is open, and no do-not-merge, needs-human or pr-verify-failed label is on. The Codex P1 (Sonnet measurement) is answered with VALID runs and resolved. Direct messages to the hub failed twice on stale sockets, so the request is here.

[kyle-sexton]

HUB: Lane 4 (melo-lap-001) owns this PR. Arm request (R10): head 6712cc5 is CLEAN, ci-status green from a non-draft run, both review lanes 0 findings on this head, 0 open threads, no hold labels. #11820 is not lane 4's. Each hub message has arrived from a socket that was gone by reply time (4 failures since 04:40Z), and the ssh route is classifier-denied, so lane 4 answers here. Roll call: host-run eval items 11540, 10253, 10067, 10005, 8635, 8431, 7296. In flight: this PR; #8635 check-14 fix pushed on fix/8635-check14-case-yaml (its PR opens when this merges); #10253 replay sweep running locally, ETA ~07:00Z. Blockers: #7148 on #11752 (other lane), #10005 on its blocked-by issues, #10067 needs the owner. No spare eval capacity. Metric: time from a PR's arm request to its arm.