Repository navigation
harness-config: concise rewrite of audit-permission-grants, audit-permission-state, draft-auto-mode-rules, setup (rollout slice B) #11565
Description
Parent
Depends on
Outcome
audit-permission-grants, audit-permission-state, draft-auto-mode-rules, setup meet the skill criteria, with every shipped description and body change backed by a valid measured pass against the frozen cases from #11561.
Demo: the PR body's per-skill decision table, with the before/after numbers behind each row.
Decisions
- Q93/Q94: the harness-config rollout (Concise skills rollout: harness-config (10 skills, 9 model-invoked) #6555) is split into sub-issues with blocked-by edges before any run: outcome eval cases first, then rewrite slices of a few skills each.
- Eval passes run on the Claude subscription; that usage is a capacity limit, not owner cost, so no spend approval gates them.
Key interfaces
Steps 4-8 of the per-plugin runbook; the skill criteria and its measurement protocol decide what ships. The rewriter's brief forbids reading any probe file.
Done when
- Before and after versions are measured together on each target model, with and without the plugin, same
--runs, judge and threshold,--keep-temp; held-out triggering probes run viameasure-invocation emit-plugin-evalwith--ablation none. -
/evals:plugin-evaljudges each result VALID, and anycase dropis restored or the before text ships. -
/docs-hygiene:compress compareran with two independent labelers; the union of losses is resolved. -
setup(user-invoked) is measured by a semantic diff of old against new body only; each flagged loss is labelled intended or real, any real loss blocks the PR, and the PR names it as unmeasured by eval with the reason. - The PR body carries a per-skill decision table (description and body decided separately) with its numbers; any skill left unmeasured is named with the reason.
-
/skill-quality:checkon the plugin root is no worse than the before-state from harness-config: outcome eval cases and probes for audit-permission-grants, audit-permission-state, draft-auto-mode-rules (rollout slice B) #11561, and the plugin's changelog fragment is added.
Out of scope
- Skills outside this slice; changes to the frozen cases from harness-config: outcome eval cases and probes for audit-permission-grants, audit-permission-state, draft-auto-mode-rules (rollout slice B) #11561.
Metadata
Metadata
Assignees
Labels
Type
Fields
No fields configured for Task.
Activity
github-actions commentedon Oct 31, 2026
Proposed work class: work-class: scoped
Basis: automated intake triage; the lanes App applied this class.
Size: Medium
Complexity: Medium
Uncertainty: Known
Flags: none
Blocked by: #11561
Understood: rewrite four harness-config skills (audit-permission-grants, audit-permission-state, draft-auto-mode-rules, setup) more concisely. Each change ships only if it is backed by a valid measured pass against the frozen eval cases from #11561, and the PR carries a per-skill decision table. #11561 must land first, so this issue is blocked by it. No duplicates found. Sibling slices #11564 and #11566 cover different skills.
Automated triage, applied by the lanes App.
melodic-automation-lanes commentedon Oct 31, 2026
Not admitted to the issue lane: it carries host-run. Admission takes the issue once each of those is met, when intake triages it again (
.github/actions/resolve-config/admission-rules.json).melodic-automation-lanes commentedon Oct 31, 2026
The latest triage verdict holds nothing: admission moved the issue to
status: confirmed; its comment says why.kyle-sexton commentedon Nov 1, 2026