Concise skills rollout: codebase-health (2 skills, 1 model-invoked) #6600

Activity

1. kyle-sexton

   added a parent issue on Oct 7, 2026

   • Concise skills: one skill-creation standard, evals, and a fleet-wide rollout #6476
2. github-actions

   added

   needs-triageNot yet classified. Floor until a type and one priority tier are set.Not yet classified. Floor until a type and one priority tier are set.

   on Oct 7, 2026
3. github-actions commented on Oct 7, 2026

   

   github-actionsBot

   on Oct 7, 2026 – with GitHub Actions

   Last edited by melodic-automation-lanes

   Contributor

   More actions

   Proposed work class: work-class: scoped
   Basis: automated intake triage; the lanes App applied this class.

   Size: Medium
   Complexity: Medium
   Uncertainty: Known
   Flags: none
   Blocked by: none

   Understood: rewrite the two codebase-health skills (audit, setup) to the skill criteria in one draft PR, following the per-plugin runbook. The PR should include a probe file for audit, a passing eval suite, a per-skill decision table, a list of anything left unmeasured, and the version or changelog entry. The issue gives the eval spend approval and the review rules, so nothing is missing. I found no likely duplicates. It is part of #6476.

   _{Automated triage, applied by the lanes App.}
4. github-actions

   added

   needs-humanHuman-in-the-loop required; autonomous sessions must not resolve items carrying this.Human-in-the-loop required; autonomous sessions must not resolve items carrying this.

   on Oct 7, 2026
5. kyle-sexton

   added the

   Task

   issue type on Oct 12, 2026
6. kyle-sexton

   removed

   needs-triageNot yet classified. Floor until a type and one priority tier are set.Not yet classified. Floor until a type and one priority tier are set.

   on Oct 12, 2026
7. kyle-sexton

   removed

   needs-humanHuman-in-the-loop required; autonomous sessions must not resolve items carrying this.Human-in-the-loop required; autonomous sessions must not resolve items carrying this.

   on Oct 24, 2026
8. kyle-sexton commented on Oct 24, 2026

   

   kyle-sexton

   on Oct 24, 2026

   MemberAuthor

   More actions

   Released by the owner's HITL session (2026-10-24, C12): it waited only on a removed gate. Intake triage re-runs on it to set its fields, then admission takes it.
9. github-actions commented on Oct 25, 2026

   

   github-actionsBot

   on Oct 25, 2026 – with GitHub Actions

   Contributor

   More actions

   The triage sweep dispatched intake triage for this issue (attempt 1 of 2).
10. github-actions

    added

    priority: mediumReal value, no hard deadline; normal backlog flow.Real value, no hard deadline; normal backlog flow.

    on Oct 25, 2026
11. melodic-automation-lanes

    added

    status: readyTriaged, unblocked, and fully specified; eligible to pick up.Triaged, unblocked, and fully specified; eligible to pick up.

    work-class: scopedA briefed fix or small feature; blast radius bounded by the brief, tests exist.A briefed fix or small feature; blast radius bounded by the brief, tests exist.

    admission-deferred

    agent-readyFully specified and briefed; eligible for autonomous pickup from the frontier.Fully specified and briefed; eligible for autonomous pickup from the frontier.

    needs-humanHuman-in-the-loop required; autonomous sessions must not resolve items carrying this.Human-in-the-loop required; autonomous sessions must not resolve items carrying this.

    and removed

    admission-deferred

    agent-readyFully specified and briefed; eligible for autonomous pickup from the frontier.Fully specified and briefed; eligible for autonomous pickup from the frontier.

    on Oct 25, 2026
12. melodic-automation-lanes commented on Oct 25, 2026

    

    melodic-automation-lanesBot

    on Oct 25, 2026

    Contributor

    More actions

    The work-resolve-issue lane stopped partway through this issue and needs a person.

    Why, in the run's own words:

    This lane cannot run the rewrite protocol the issue requires, and the protocol forbids shipping an unmeasured change. The issue follows plugins/playbooks/skills/skill-authoring/reference/rewrite-a-plugin.md, and its Done-when needs /skill-quality:check plugins/codebase-health on the PR head, /evals:validate on the suite, smoke and before/after /evals:plugin-eval passes, and /docs-hygiene:compress compare. Its PR body also needs a per-skill decision table with measured before and after numbers. skill-criteria.md 'Measurement' step 6 says to 'ship only parts that were measured'. This lane loads only the work-items plugin, so none of the evals, skill-quality or docs-hygiene skills resolve. Bash is limited to the named wrappers, so neither the claude CLI nor a paid eval pass can run. Rewriting audit or setup here would ship unmeasured text. Writing the cases and probes alone would leave them without the smoke pass, grader calibration and frozen digest that runbook step 2 requires before they count. Run the issue in a session that has the evals, skill-quality and docs-hygiene plugins and eval API credit (pre-approved on ＃6476 at --max-cost-usd 5 per pass).

    What the run says it did not do:

    • Rewrite of codebase-health's audit and setup skills: it ships only after measured before/after eval passes, and this lane has no evals plugin, no claude CLI and no eval spend
    • Outcome cases and the probes/audit.json probe file: runbook step 2 needs a smoke pass, grader calibration and a frozen digest, which this lane cannot run
    • /skill-quality:check and /evals:validate on plugins/codebase-health: neither skill is loaded in this lane
    • PR-prep review and build/test steps: skipped because there is no diff to review

    Run: https://github.com/melodic-software/claude-code-plugins/actions/runs/40195185003

    To retry, remove needs-human and add agent-ready again.
13. kyle-sexton

    marked this as blocked on Oct 27, 2026

    • feat(ci): validate, gate and summarize dispatched plugin eval runs #9704
14. kyle-sexton commented on Oct 27, 2026

    

    kyle-sexton

    on Oct 27, 2026

    MemberAuthor

    More actions

    This was generated by AI.

    Parked for capacity, not for the owner. This rollout's remaining work is the measured eval passes of the batch-1 protocol, which the issue lane cannot run: it cannot start claude or claude plugin eval (see the lane's escalation on #6585, https://github.com/melodic-software/claude-code-plugins/actions/runs/40186335064). It waits on the plugin eval dispatch work in #9704, now a blocked-by edge, so admission holds it until #9704 closes.

    Under the owner's 2026-10-26 stop rule, a person is needed only for a change in intent or vision, an implementation finding that changes the intent or primary design, or a high or critical security issue; host, attended and eval runs are agent work. So needs-human comes off, and the issue moves to host-run with status: confirmed, which keeps the issue lane from taking it when #9704 closes: #9704 adds an eval dispatch workflow, not eval capability for the issue lane.
15. kyle-sexton

    added

    host-runNeeds an unattended run on a host; the host runner takes it, not a person or the issue lane.Needs an unattended run on a host; the host runner takes it, not a person or the issue lane.

    status: confirmedA maintainer confirmed the item is real and in scope; ready to be worked.A maintainer confirmed the item is real and in scope; ready to be worked.

    and removed

    status: readyTriaged, unblocked, and fully specified; eligible to pick up.Triaged, unblocked, and fully specified; eligible to pick up.

    needs-humanHuman-in-the-loop required; autonomous sessions must not resolve items carrying this.Human-in-the-loop required; autonomous sessions must not resolve items carrying this.

    on Oct 27, 2026
16. kyle-sexton

    self-assigned this

    on Oct 31, 2026
17. kyle-sexton commented on Oct 31, 2026

    

    kyle-sexton

    on Oct 31, 2026

    Last edited by kyle-sexton

    MemberAuthor

    More actions

    No description provided.
18. kyle-sexton commented on Nov 1, 2026

    

    kyle-sexton

    on Nov 1, 2026

    MemberAuthor

    More actions

    This was generated by AI.

    Progress: #11814 merged (f414218). It adds five outcome eval cases (24 deterministic graders) and 20 frozen trigger probes for the codebase-health audit skill, and widens one grader the smoke pass caught rejecting a correct reply. No skill text changed.

    This issue stays open. The concise rewrite and its measured passes wait on the eval validity fixes in #11618 and #11628. The frozen suite digest for those runs is f9c3125e4d0d496cfc662adb16490dd82e885ac8 (plugins/codebase-health/evals at 4bce0ae). The claim on this issue is released.
19. kyle-sexton commented on Nov 1, 2026

    

    kyle-sexton

    on Nov 1, 2026

    Last edited by kyle-sexton

    MemberAuthor

    More actions

    No description provided.
20. kyle-sexton

    added a commit that references this issue on Nov 1, 2026

    test(codebase-health): widen the scope-ask and section-heading graders

    ...

    6cef243
21. kyle-sexton

    closed this as completedin #12050on Nov 1, 2026