Repository navigation
skill-quality: skill authoring, quality and catalog #7582
Description
Parent
Refs #7555. Source: PLAN Phase 14, topic poteto-review. Thread: Skill authoring, quality and
catalog.
Depends on
- config-cascade: settings resolution and convention home #7571 Settings resolution and convention home (description target and warn settings)
Outcome
Skills follow Anthropic's authoring guidance as measured rules: a SKILL.md router with actions,
evals on all skills, a description standard with probe-proven triggers, consolidated *:check
skills, and catalog files generated in the release PR.
Decisions
- Skills follow Anthropic's authoring guidance: SKILL.md a router under 5k tokens,
actions/per
action, evals on all skills, split only by exception, rollout across every skill (D107, Q203).
Origin: ours (owner reworked the size options). - Description standard: one what-sentence, "Use when" with one trigger per branch, at most three
probe-proven hook terms, target 250, warn above 419, fail at 1,024; check 12 drops the single-quote
rule (D82, Q183, Q184). Settings: description target 250; warn 419 (1,024 fixed). Check 3's
unquoted-trigger tracking lands first, in the same PR, with a red-then-green test; two attempts
ranked on the session-flow trim-pilot diff (B2). - Consolidate the
*:checkskills (recount first: 29), thenarchitecture:map-*and
code-metrics:audit-*; fix the Cursor/checkcollision by consolidation plus distinct first
clauses (D82, Q179, Q186; owner observed the collision). - Stack-specific tools: detect, research live, interview, write setup into the project; use an
existing tool without an interview (D66, Q96). - Bare invocation: infer the action; read-only proceeds and names what it inferred (setting
proceed); state changes propose and wait (fixed floor); unattended state changes stop and report
(D79, Q93). - Invocation convention: any automation starter means model-invocable, safety in body gates and
scripts; user-invoked only for starved skills no automation starts (D53, Q138). - Skill criteria gain "choose leading words on purpose" with
--baseline; write-for-agents gains a
leading-words section, a sentence-level no-op test, the context-boundary line (D4, Q525). - Don't wrap skill-creator; add a tested route for cutting text (D7).
- Files, not registry entries; catalog, cheat-sheet and marketplace manifest generated in the release
PR; problem-first section above the generated cheat-sheet block (Q143, Q144, Q146, Q546;
Lauren's files-not-registry rule). - Shared copies stay copies; one shared-copies record (D83, Q180). Plugin READMEs use one install form
or point to the root README (Q526). Re-audit shipped skills on each model release (D6; measured gap:
median 1,477 words versus Matt 506 and pstack 364). - Probes on this thread's slices: four invocation probes, routine first (D53); picker label
re-observation before leaf-name registry removal (Q179); skill-visibility measurement gates
Q183-Q187 (Q137, Q182); trim pilot on session-flow, non-inferiority with a 5-point margin (Q185);
each consolidation merge probed and the recount (Q186); hook terms probe-proven (Q183); hand-off
phrasing eval, 12 cases, 3 arms, 3 runs (D85, Q125, Q188); steering words with--baseline(D4);
each cut tested with and without removed text (D6, D107);skillListingBudgetFractionstays 0.07
until the pilot ships (Q187). - Sub-topic promotion: this thread gets its own plan at phase start (B7).
Key interfaces
From the design's module layout: "skill-quality: check 12 drops the single-quote rule; description
standard (target 250, warn 419, fail 1,024). *:check skills consolidate after a recount, then
architecture:map-* and code-metrics:audit-*." Settings: "Description target 250, warn 419 (1,024
fixed). ... Bare run read-only proceed."
Done when
- The check-skill test suite exits 0 and includes a case asserting check 3 warns on a dropped
unquoted trigger (no such case on main). -
rg -c "not single-quoted" plugins/skill-quality/scripts/check-skill.shreturns 0 (nonzero on
main), andbash plugins/skill-quality/scripts/check-skill.sh plugins/session-flow/skillsexits 0. - Every slice sub-issue is closed by a merged PR, each with its probe or eval result.
Out of scope
- Setting the fleet listing budget (housekeeping thread, after the pilot).
Related
- Concise skills rollout: the 15 check-and-setup plugins #6619 concise skills rollout, check-and-setup plugins (stays under Concise skills: one skill-creation standard, evals, and a fleet-wide rollout #6476; linked, not moved).
- Concise skills rollout: playwright (3 skills, 2 model-invoked) #6587 concise skills rollout, playwright (stays under Concise skills: one skill-creation standard, evals, and a fleet-wide rollout #6476; linked, not moved).
Activity
github-actions commentedon Oct 16, 2026
Proposed work class: work-class: structural
Basis: automated intake triage; the lanes App applied this class.
Size: Large
Complexity: Medium
Uncertainty: Knowable
Flags: none
Blocked by: none
Understood: this is the parent tracker for skill authoring, quality and catalog work. The slices are mostly delivered. #11957 and #11960 are still open, and the
*:check,architecture:map-*andcode-metrics:audit-*consolidation slices are not filed yet.#7582 (comment)
That decision settles the open questions. The two exit-code checks and the host-run probes are for a host run, not a person decision.
Automated triage, applied by the lanes App.
kyle-sexton commentedon Oct 24, 2026
Released for re-triage under the owner's 2026-10-24 rule: an issue stops for a person only for an intent or vision question, a security call, a host or attended run, or spend outside prepaid credit. The hold label came off and intake triage runs again now; it re-applies a hold, with a numbered question, when one of those applies.
To hold this issue again, add
needs-human(orstatus: needs-decision) and say why in a comment.28 remaining items
kyle-sexton commentedon Nov 2, 2026
Decision recorded.
Most of two of this issue's three "Done when" lines is on origin/main; the two exit codes they require were not run in this session. Check 3's unquoted-trigger case is at
plugins/skill-quality/scripts/check-skill.test.sh:442-472(case 9a, "dropped unquoted Use-when item warns"), delivered by #11965.not single-quotedno longer appears inplugins/skill-quality/scripts/check-skill.sh. Seven of the nine slices listed in the 2026-11-01 split comment are closed completed: #11954, #11955, #11956, #11958, #11959, #11965, #11971. #8645 is also closed completed. That comment describes itself as a session reading; it applies owner decision Q-F ("go w/ recommendation"). Two slices are still open: #11957 (catalog generation in the release PR) and #11960 (the four invocation probes). Both are labeled host-run. That comment also said the*:checkconsolidation and the Cursor/checkcollision fix would be filed after #11956 closed, as host-run waves, because they need the Q179 picker re-observation and the Q186 merge probes. #11956 closed on 2026-11-01 with a three-wave plan of 32 prerequisite-check skills (report). No consolidation slice is filed yet. Thearchitecture:map-*andcode-metrics:audit-*consolidations, which the body orders after it, are not filed either.Acceptance criteria:
check-skill.test.sh:442-472, covers check 3 warning on a dropped unquoted trigger).bash plugins/skill-quality/scripts/check-skill.sh plugins/session-flow/skillsexits 0.*:checkskills, thenarchitecture:map-*andcode-metrics:audit-*; fix the Cursor/checkcollision by consolidation plus distinct first clauses (D82, Q179, Q186), each consolidation merge probed.Route: host-run
Posted by the issue-triage session. This was generated by AI.
kyle-sexton commentedon Nov 2, 2026
Lane 1 (hub assignment 2026-11-02 17:2xZ) took #7582. Criteria 1 and 2 of the acceptance record (issuecomment-6582916179) are met on main b000bb5. No file under plugins/skill-quality, plugins/session-flow or plugins/playbooks/skills/skill-authoring differs from origin/main 57477a9.
bash plugins/skill-quality/scripts/check-skill.test.sh: rc=0, 302 ok, 0 not ok. Case 9a (check-skill.test.sh:442-472) passes.bash plugins/skill-quality/scripts/check-skill.sh plugins/session-flow/skills: rc=0, 19 passed, 0 failed.Criteria 3 and 4 are next. Before the consolidation waves can be filed, the target they consolidate into (D82/Q179/Q186) has to be found: no durable copy is in this repo, in architecture owner-decisions.md, or in Epic: absorb Lauren Tan's and Matt Pocock's agent workflows (poteto-review) #7555 or docs(invocation-mode): record the invocation probes and narrow the scheduled-task rule (#11960) #12056. Lane 1 has asked the owner for it.
kyle-sexton commentedon Nov 2, 2026
Slices filed for the
*:checkconsolidation, the Cursor/checkcollision fix, and thearchitecture:map-*andcode-metrics:audit-*consolidations. All are sub-issues of this issue, labeledhost-run, with native blocked-by links.lib/prerequisites.mjs, not a/<plugin>:checkskill*:checkwave 1: 14 hook-runtime checks*:checkwave 2: 8 formatter and linter checks*:checkwave 3: 6 external-tool checks and the 4check-prerequisitesskillsinstruction-placement:check,skill-quality:checkandtoolchain:check, so nocheckleaf remains (Q179 picker re-observation)architecture:map-*consolidation; first item is a sized wave plan, and the target is an open questioncode-metrics:audit-*consolidation; first item is a sized wave plan, and the target is an open questionWhy the contract is its own slice:
docs/contracts/prerequisites/prerequisites.schema.json:36,63requires each entry'scheckand restricts it to a slash command.lib/prerequisites.mjs:207-208rejects any other value, and:520and:719print it in the notice.All 35 plugins with a
checkorcheck-prerequisitesskill already ship aprerequisites.jsonandlib/prerequisites.mjs(origin/main b000bb5). Each wave still adds one, per the schema, for any plugin that lacks one when the wave starts.Posted by a lane 1 worker session. This was generated by AI.