feat\(corrections\): absorb retro codify and /review:audit-enforceability into /corrections:codify and remove them - part 1 of 2

Part 1 of 2 of #11121, split at intake (owner Q93). The done-when below is copied from #11121 as written.

Done when

Codify has eval cases for a findings-file input and a PR-review-comment source, and each removed skill's or mode's eval cases are ported to codify with before/after results in the PR, showing no loss.

Proposed work class: work-class: scoped
Basis: automated intake triage; the lanes App applied this class.

Size: Medium
Complexity: Medium
Uncertainty: Knowable
Flags: none
Blocked by: none

Understood: add codify eval cases for a findings-file input and a PR-review-comment source, and port each removed skill's or mode's eval cases, with before/after results in the PR.

Decision recorded.

These answer the lane's five questions (#11128 (comment)). The split into part 1/2 did not carry these owner answers over.

Rung selection: strongest-first for every input. This is owner Q117, accepted in poteto-review round 23 ("Fold /review:audit-enforceability into codify, with strongest-first as the only selection rule"). Audit-enforceability evals 1, 3, 8 and 9 and retro evals 7, 9, 18 and 20 test cheapest-rung or encode_policy behavior. Each is ported with its expected value changed to the strongest-first answer, and the PR names each one as an intended change, not a loss.

Findings-file output: each finding row becomes a described mistake that codify counts, proves and ends in a proposal. emit-stubs.sh and the stub shape go with the removed skill.
[quote truncated; the full decision is at the link]

#11128 (comment)

This decision settles the open questions. The before/after results come from a host run, which a person does after the lane-checkable work.

_{Automated triage, applied by the lanes App.}

The work-resolve-issue lane did not start this issue: it needs a person to make it clearer first.

Reason: intent

Why, in the run's own words:

Codify can only take the removed skills' eval cases "with no loss" if someone first decides how it handles each new input, and neither ＃11128 nor ＃11121 says how. Please record a decision on each of these: (1) Rung selection. The ladder gives three readers three different rules (plugins/corrections/skills/codify/reference/enforcement-ladder.md, Selection rules): codify takes the strongest rung, audit-enforceability the cheapest rung that can assert the finding, and retro codify follows encode_policy, whose default promote-when-must-hold starts a lesson as a CLAUDE.md or rules line. Does codify keep strongest-first for every input, take the cheapest rung for a findings file, or gain encode_policy? audit-enforceability evals 1, 3, 8 and 9 and retro evals 7, 9, 18 and 20 test the rules codify does not use. (2) Findings-file output. Does codify keep audit-enforceability's one stub per finding (emit-stubs.sh, the stub shape, earliest-stage, error text, the fence against the reviews home), or does each finding row become a described mistake that is counted and proved and ends in a proposal? (3) Settings ownership. Do ratchet_offer (now review's userConfig plus docs/conventions/review.yaml, audit-enforceability evals 10 to 15), encode_policy and review_mining_prs (now session-flow's, retro eval 9) move into corrections' schema and userConfig, or are those evals dropped as intended loss? (4) PR review comments. ＃11121 asks codify to count "the author's own [comments] after merge". Retro's fetch-review-comments.sh drops the PR author's comments, and retro eval 11 asserts it does. Which behavior wins? And may codify ship its own GH -calling script, given that codify SKILL.md section 4 says it names no forge CLI command and goes through /source-control:pull-request? (5) Before/after results. Who runs the eval suites to produce the before/after results the done-when wants in the PR? This lane has no eval runner.

What the run says it did not do:

No implementation or eval cases were written: the run pushed back before any edit, on open design decisions
tidy skipped: the change is empty
review, verify and class audits skipped: the change is empty

Run: https://github.com/melodic-software/claude-code-plugins/actions/runs/40999514263

To retry, answer the question in a comment whose first line is **Decision recorded.**, then remove needs-human and add agent-ready again.

Not admitted to the issue lane: it carries host-run. Admission takes the issue once each of those is met, when intake triages it again (.github/actions/resolve-config/admission-rules.json).

The latest triage verdict holds nothing: admission moved the issue to status: confirmed; its comment says why.

No description provided.

Host run. Built codify's findings-file and review-comment inputs, the settings move and a 25-case deterministic eval suite on the PR below; the before and after claude plugin eval runs did not run here and are pending on the laptop lane.

What ran (Claude Code 2.1.316, base c7e50d15a057fa5cbff38c4a5b23021d9f861ccb):

plugins/corrections/scripts/codify-settings.test.sh: 37 checks pass, including the precedence audit-enforceability evals 12, 14 and 15 tested (repository file over user option; an invalid repository value falls to the default, never the user option).
read-review-comments.test.sh: 38 checks pass, including the after-merge rule (Decision 4).
validate-cases.py over plugins/corrections/evals: no FAIL; every regex and tool grader sorted its samples. Its WARNs are the expected --allow-tools notes.
check-skill.sh --require-evals plugins/corrections/skills: PASS.
scripts/check-affected.sh --tests on the committed head: 14 passed, 2 skipped.

Against the acceptance criterion in the owner's decision comment:

Item	State
Codify has eval cases for a findings-file input	Written: `codify-findings-file-input` and 14 more ported cases
Codify has eval cases for a PR-review-comment source	Written: `codify-reviews-routes-recurring-lessons` and 3 more
Each removed skill's or mode's eval cases are ported	Written: audit-enforceability 1-15, retro 2, 7-12, 18, 20, and `re-codify-routes-check-asks-first`
The PR shows before/after results with no loss	Waiting. Needs `claude plugin eval`; no eval ran

The eval step that remains: run the suite on the PR head from plugins/corrections, and the before arm on the pinned base. make-before-arm.py (one-off, gitignored, on this host only) lays out the before arm (23 cases; the two ratchet cases that assert the moved corrections.yaml layer have no shared grader) with each old skill's own prompt, and its docstring gives the command. Both Bash-granting runs use the brief's HOME and capped-scope form.

Three behavior changes go beyond the owner's named list and are flagged in the PR for the owner to see:

A person's findings invocation lifts the threshold for every row, since a one-row finding counts 1 and would otherwise never reach a proposal. reviews keeps the threshold. Basis: judgment.
A person's invocation writes the proposal without a further approval, section 6's existing consent rule; retro asked first. CLAUDE.md is still not touched.
Decision 2 also retires what audit-enforceability evals 1, 6, 7 and 9-15 asserted about stub counts, stub homes, earliest-stage and writer flags.

Frees once the eval results land and the PR merges: #11129 and #11121 directly; #7573, #7575 and #7590 behind them; #7555 also waits on the #7711 host-run root. Nothing is freed yet.

Admission refuses the issue lane while host-run is on, and the eval run needs this PR's code, so the label stays until that run is done.

PR: #11629 (draft; /source-control:pull-request ready has not run). The before and after evals are pending on the laptop host-run lane, which runs them against this head; the PR body gives the commands.

Target labels: add none; remove none.

fix(corrections): name the guardrail criterion and the rung id in cod…

fix(corrections): lead codify's report with any no-guardrail finding

fix(corrections): answer #11629 review round 1

test(corrections): keep prove-check's fixture tags unsigned

docs(catalog): regenerate for the corrections description

fix(pr-pipeline): narrow the eval and tracker-output admission rows

Navigation Menu

Repository navigation

feat\(corrections\): absorb retro codify and /review:audit-enforceability into /corrections:codify and remove them - part 1 of 2 #11128

Done when

6 remaining items

Metadata

Assignees

Labels

Type

Fields

Projects

Milestone

Relationships

Development

feat\(corrections\): absorb retro codify and /review:audit-enforceability into /corrections:codify and remove them - part 1 of 2 #11128

Description

Done when

Activity

github-actions commented on Oct 30, 2026

melodic-automation-lanes commented on Oct 30, 2026

6 remaining items

melodic-automation-lanes commented on Oct 30, 2026

melodic-automation-lanes commented on Oct 30, 2026

kyle-sexton commented on Oct 31, 2026

kyle-sexton commented on Oct 31, 2026

Metadata

Metadata

Assignees

Labels

Type

Fields

Projects

Milestone

Relationships

Development

Issue actions