Repository navigation
harness-config: concise rewrite of audit-automation-gaps, audit-prompting-postures, unhobble (rollout slice C) #11566
Description
Parent
Depends on
Outcome
audit-automation-gaps, audit-prompting-postures, unhobble meet the skill criteria, with every shipped description and body change backed by a valid measured pass against the frozen cases from #11563.
Demo: the PR body's per-skill decision table, with the before/after numbers behind each row.
Decisions
- Q93/Q94: the harness-config rollout (Concise skills rollout: harness-config (10 skills, 9 model-invoked) #6555) is split into sub-issues with blocked-by edges before any run: outcome eval cases first, then rewrite slices of a few skills each.
- Eval passes run on the Claude subscription; that usage is a capacity limit, not owner cost, so no spend approval gates them.
Key interfaces
Steps 4-8 of the per-plugin runbook; the skill criteria and its measurement protocol decide what ships. The rewriter's brief forbids reading any probe file.
Done when
- Before and after versions are measured together on each target model, with and without the plugin, same
--runs, judge and threshold,--keep-temp; held-out triggering probes run viameasure-invocation emit-plugin-evalwith--ablation none. -
/evals:plugin-evaljudges each result VALID, and anycase dropis restored or the before text ships. -
/docs-hygiene:compress compareran with two independent labelers; the union of losses is resolved. - The PR body carries a per-skill decision table (description and body decided separately) with its numbers; any skill left unmeasured is named with the reason.
-
/skill-quality:checkon the plugin root is no worse than the before-state from harness-config: outcome eval cases and probes for audit-automation-gaps, audit-prompting-postures, unhobble (rollout slice C) #11563, and the plugin's changelog fragment is added.
Out of scope
- Skills outside this slice; changes to the frozen cases from harness-config: outcome eval cases and probes for audit-automation-gaps, audit-prompting-postures, unhobble (rollout slice C) #11563.
Metadata
Metadata
Assignees
Labels
Type
Fields
No fields configured for Task.
Activity
github-actions commentedon Oct 31, 2026
Proposed work class: work-class: scoped
Basis: automated intake triage; the lanes App applied this class.
Size: Medium
Complexity: Medium
Uncertainty: Knowable
Flags: none
Blocked by: #11563
Understood: rewrite the three harness-config skills (audit-automation-gaps, audit-prompting-postures, unhobble) concisely. Each shipped change needs a valid measured pass against the frozen cases from #11563, and the PR carries a per-skill decision table. The scope and done-when items are clear. The eval and measurement steps may need a host run, which admission can route. This slice depends on #11563 being done first. The other open rollout-slice issues (A and B) cover different skills, so I found no duplicate.
Automated triage, applied by the lanes App.
melodic-automation-lanes commentedon Oct 31, 2026
Not admitted to the issue lane: it carries host-run. Admission takes the issue once each of those is met, when intake triages it again (
.github/actions/resolve-config/admission-rules.json).melodic-automation-lanes commentedon Oct 31, 2026
The latest triage verdict holds nothing: admission moved the issue to
status: confirmed; its comment says why.kyle-sexton commentedon Nov 1, 2026
fix(harness-config): ship main's unhobble text after the measured pass