Concise skills rollout: tdd (1 skills, 1 model-invoked)

Part of #6476. Rewrite tdd's skills to the skill criteria in one draft PR, following the per-plugin runbook (/playbooks:skill-authoring, reference/rewrite-a-plugin.md). The runbook and the rewrite protocol it cites decide what ships; this issue holds only what is specific to the plugin.

The plugin today

Skills: principles.
Outcome eval cases under evals/: 0. Probe files under probes/: 0.

Path (owner decision 2026-10-18)

The 1-skill tail takes the description-only path, measured by trigger probes (decision). Only the principles description changes; the body stays as on main. Eval spend for the probe runs falls under the prepaid-credit pre-approval, with the batch-1 ceiling of --max-cost-usd 5 per pass.

Done when

git diff --name-only origin/main -- plugins/tdd/skills/ lists only plugins/tdd/skills/principles/SKILL.md, and its diff touches only the description frontmatter line.
bash plugins/skill-quality/scripts/check-skill.sh plugins/tdd/skills exits 0.
bash plugins/skill-quality/scripts/measure-invocation.sh validate plugins/tdd/probes exits 0 on the committed frozen probe file plugins/tdd/probes/principles.json.
The PR body carries the before and after trigger rates from held-out probes (measure-invocation.sh emit-plugin-eval, run with --ablation none on each target model) and the measure-invocation.sh compare output; the after description ships only when its rate is not below the before rate outside noise, otherwise the before description stays.
ci-status is green on the PR.

Generated by Claude Code

Proposed work class: work-class: scoped
Basis: automated intake triage; the lanes App applied this class.

Size: Small
Complexity: Medium
Uncertainty: Knowable
Flags: none

Understood: change only the description line of the tdd principles skill, and keep the new description only if held-out trigger probes show its rate is not below the old one outside noise. The body path and spend look settled by the #6476 decision. The probe runs (claude plugin eval) can't be done by an unattended lane run, so they need an eval-capable session. The other done-when items (diff scope, check-skill.sh, validate on the probe file) can be checked in the lane. No likely duplicates found among the open issues.

_{Automated triage, applied by the lanes App.}

This was generated by AI during triage.

Held for a person: The #6476 owner decision (2026-10-18) moves this 1-skill plugin to a description-only path measured by trigger probes, but this body still asks for a full rewrite and a person's eval-spend approval; the body and spend need the owner's update first.

Released by the owner's HITL session (2026-10-24, C12): it waited only on a removed gate. Intake triage re-runs on it to set its fields, then admission takes it.

Held again by the owner's HITL session (2026-10-24): my release earlier today was wrong, because it needs trigger-probe runs (claude plugin eval) to measure the description; the issue lane cannot run it: a lane run executes only the verify-scripts listed in docs/conventions/pr-pipeline.yaml under dontAsk, so it cannot start claude, claude plugin eval or a workflow dispatch. Run it from an eval-capable session (the batch-1 protocol).

The work-resolve-issue lane did not start this issue: it needs a person to make it clearer first.

Why, in the run's own words:

Issue ＃6615 needs before and after trigger rates from claude plugin eval runs (measure-invocation.sh emit-plugin-eval, --ablation none on each target model, with a --max-cost-usd 5 ceiling per pass), and the after description may ship only if its rate is not below the before rate. This unattended lane cannot start claude or claude plugin eval, so it can't produce those rates or decide whether the new description ships. The owner's latest comment (2026-10-24) held the issue again for this reason and said to run it from an eval-capable session (the batch-1 protocol). Question for a person: should the issue stay held for an eval-capable session, or should it be re-scoped to something this lane can verify, such as shipping the description with the rates left to a later run?

What the run says it did not do:

Whole issue: no edit to plugins/tdd/skills/principles/SKILL.md, no probe runs, no PR-body rates.

Run: https://github.com/melodic-software/claude-code-plugins/actions/runs/40128858690

To retry, answer the question here, remove needs-human and add agent-ready again.

This was generated by AI.

Parked for capacity, not for the owner. This rollout's remaining work is the measured eval passes of the batch-1 protocol, which the issue lane cannot run: it cannot start claude or claude plugin eval (see the lane's escalation on #6585, https://github.com/melodic-software/claude-code-plugins/actions/runs/40186335064). It waits on the plugin eval dispatch work in #9704, now a blocked-by edge, so admission holds it until #9704 closes.

Under the owner's 2026-10-26 stop rule, a person is needed only for a change in intent or vision, an implementation finding that changes the intent or primary design, or a high or critical security issue; host, attended and eval runs are agent work. So needs-human comes off, and the issue moves to host-run with status: confirmed, which keeps the issue lane from taking it when #9704 closes: #9704 adds an eval dispatch workflow, not eval capability for the issue lane.

Host run. Description-only trigger screen for tdd:principles: before and after copies launched together, frozen probes.

What ran

Claude Code 2.1.316, base a781adf89a, models claude-opus-5-5 and claude-sonnet-5-5, --ablation none, -j 2, cases from measure-invocation.sh emit-plugin-eval, 20 probes (10 positive, 10 negative), digest e6552aaae0a9.
Description length: 621 to 553 characters (11.0%). Probes were written by a fresh agent from the before description. The rewriter was barred from reading probes and evals, and kept every quoted trigger phrase.
Rule written before any after-version result: --runs 1 screen, then --runs 3, where after must score at least before on every model (the done-when), or the before description ships.
Meaning diff: two labelers, union of losses. PASS from both.
Command in short form: claude plugin eval <copy> --eval-dir trigger --model <model> --ablation none --runs N -j 2 --keep-temp --max-cost-usd 5, each in a capped scope.
Result files: /home/kyle/worktrees/host-run-tail-eval/out/<label>-before.json and -after.json, labels s1-tdd-opus, s1-tdd-sonnet. Rule: PREREG.md beside them. Frozen probes: /home/kyle/worktrees/host-run-tail-eval/probes/tdd/principles.json.

Done-when

Description-only diff: not covered, no PR. The run copies differed only in the description.
Probe file in the PR: not met: frozen outside the repository, no PR.
Before and after scores in the PR body: not covered, no PR. They are in this record.
After at least before, or the PR ships the before description: not covered: no --runs 3 gate ran.
/skill-quality:check, no new FAIL: not covered: no PR head.
Changelog entry: not met, no PR.

Scores (hits on both polarities, over probes x runs)

Pass	Model	Before	After
screen 1 run	opus	20/20	20/20
screen 1 run	sonnet	17/20	18/20

Result

Passed the screen. No --runs 3 gate ran inside the time box, so the description is unchanged on main and nothing ships yet. Remaining: the gate, then the PR with the probe file.

Caveats

run-validity.py has no trigger mode: a missed positive prints INVALID on skill fired, and an ungranted-tool No such tool available error prints a denial. Both were read per an addendum written before any --runs 3 pass, the same for both versions. That addendum fails a pass on a denial of a granted tool, and every other row had to pass.
The eval loads one plugin, so these numbers show no rival skills from other plugins. A score at the ceiling cannot show an improvement.
No separate held-out set ran: the rewriter never saw these probes and cut no quoted phrase.

Target labels: add none; remove none.

No description provided.

This was generated by AI.

Progress: #12129 merged (1870f1b). It adds outcome eval cases and a frozen trigger-probe file for the tdd plugin's principles skill. No skill text changed.

Three review rounds hardened the graders before merge:

the precondition graders judge each item's verdict per clause, so verdict-first, comma-, "and"- and "but"-joined replies grade correctly;
db-real no longer passes a reply that mocks the database;
asserts-output no longer matches the method name Total(cart), and it accepts "verifies".

One negative probe that the skill actually answers was replaced before the file froze. Replaying the stored smoke replies changed no result in any round.

This issue stays open for its measured pass.

Navigation Menu

Repository navigation

Concise skills rollout: tdd (1 skills, 1 model-invoked) #6615

The plugin today

Path (owner decision 2026-10-18)

Done when

1 remaining item

What ran

Done-when

Scores (hits on both polarities, over probes x runs)

Result

Caveats

Metadata

Assignees

Labels

Type

Fields

Projects

Milestone

Relationships

Development

Concise skills rollout: tdd (1 skills, 1 model-invoked) #6615

Description

The plugin today

Path (owner decision 2026-10-18)

Done when

Activity

github-actions commented on Oct 7, 2026

kyle-sexton commented on Oct 19, 2026

kyle-sexton commented on Oct 24, 2026

kyle-sexton commented on Oct 24, 2026

1 remaining item

melodic-automation-lanes commented on Oct 24, 2026

kyle-sexton commented on Oct 27, 2026

kyle-sexton commented on Oct 31, 2026

What ran

Done-when

Scores (hits on both polarities, over probes x runs)

Result

Caveats

kyle-sexton commented on Nov 1, 2026

kyle-sexton commented on Nov 1, 2026

Metadata

Metadata

Assignees

Labels

Type

Fields

Projects

Milestone

Relationships

Development

Issue actions