feat(plugin): optimize-skill's execution track, measured end to end — plus the skill_triggered and --split fixes it surfaced - #109

Open

uipreliga wants to merge 202 commits into

mainfrom

feat/plugin-optimize-skill

Commits on Aug 20, 2026

feat(dataset): 1/3 — add Dataset.split_field and the --split row filter

uipreliga
and
claude
committed
7376062
feat(plugin): 2/3 — add the optimize-skill skill and split-label the activation template

uipreliga
and
claude
committed
240d66c
docs: 3/3 — add the skill-optimization tutorial, and fix the reachability guidance it disproved

uipreliga
and
claude
committed
8920410
style: apply ruff format to the reachability lint assertion

uipreliga
and
claude
committed
2c30397
docs(harness): record the all-skipped-run-exits-0 gap found while adding --split

uipreliga
and
claude
committed
ae3c39b
fix: code review fixes for the split-field / optimize-skill plan

uipreliga
and
claude
committed
b53c7d4
feat(plugin): promote a measured `analyze` description, and close the two open findings

uipreliga
and
claude
committed
d7d56f1
docs(tutorial): Stage C completed — the analyze promotion is confirmed on holdout

uipreliga
and
claude
committed
844348d
docs: record the bare-name collision hazard, and mark the plan complete

uipreliga
and
claude
committed
e340b58
feat(plugin): add an execution track to optimize-skill — optimize the body, not just the description

uipreliga
and
claude
committed
35e7714
fix(plugin): repairs from live-testing optimize-skill with cold agents

uipreliga
and
claude
committed
9526d54
docs(tutorial): conform 08 to the tutorial front-matter and title convention

uipreliga
and
claude
committed
2a0d72d
refactor(dataset): rename the split values tune/holdout to train/test, and label them for the user

uipreliga
and
claude
committed
6e9c1b4
docs(tutorial): use blockquotes, not mkdocs admonitions, in tutorial 08

uipreliga
and
claude
committed
6ebe2d9
fix(plugin): 1/8 — correct optimize-skill's execution track and cost table

uipreliga
and
claude
committed
a50a356
feat(plugin): 2/8 — ship the outcome-suite template for the execution track

uipreliga
and
claude
committed
a90f849
fix(docs): 3/8 — repair the stale skill counts, and the sensor that missed them

uipreliga
and
claude
committed
17aa999
feat(tasks): 4/8 — add the ci-outcome sample, the execution track's worked example

uipreliga
and
claude
committed
b910adc
feat(lint): 8/8 — CE035 catches the partly-labelled split dataset

uipreliga
and
claude
committed
f252e6e
fix(plugin): 5/8 — execution-track A/B returns a NULL result, and corrects the

uipreliga
and
claude
committed
b375fb8
docs(tutorial): 6/8 — add tutorial 09, the content track, reported as a null

uipreliga
and
claude
committed
c0d6959
docs(tutorial): 7/8 — make tutorial 08's A/B reproducible, and untangle it

uipreliga
and
claude
committed
eb758e4
fix: code review fixes for the optimize-skill plan

uipreliga
and
claude
committed
d5b4fb0
test(lint): CE036 — a row's prompt must not contain what its criteria grade

uipreliga
and
claude
committed
c2315ed
docs(harness): defer three gaps found reviewing the optimize-skill plan

uipreliga
and
claude
committed
95553ec
fix(criteria): an errored Skill call is not engagement

uipreliga
and
claude
committed
3b2802b
docs: correct tutorial 09 — the round's null had a different cause than reported

uipreliga
and
claude
committed
2ddb737
docs(tutorial): 09 gets the real baseline — a verified ceiling, 1.000 on 6/6

uipreliga
and
claude
committed
5f14f73
fix(tasks): gate engagement, in the artifacts that call it a gate

uipreliga
and
claude
committed
c40cdd4
docs(tasks): make the ci-outcome sample runnable in two lines, and record the dead end

uipreliga
and
claude
committed
bc9125b
chore: close the last loose ends from the optimize-skill plan

uipreliga
and
claude
committed
3ce116b
docs: the task guide documented the old skill_triggered detection rule

uipreliga
and
claude
committed
b1893ac
fix(criteria): 1/8 — skill_triggered's live and final verdicts agree again

uipreliga
and
claude
committed
d3d0f5b
fix(dataset): 2/8 — a --split that matches nothing aborts instead of exiting 0

uipreliga
and
claude
committed
2483f5a
feat(plugin): 3/8 — the activation template caps and isolates, as its own skill advises

uipreliga
and
claude
committed
1087cee
test(lint): 4/8 — CE036 gets unit fixtures, and exempts skill_name

uipreliga
and
claude
committed
9098751
fix(dataset): 5/8 — validate every row id, not just the selected ones

uipreliga
and
claude
committed
38efbce
feat(cli): 6/8 — plan expands datasets and takes --split

uipreliga
and
claude
committed
b23b6a1
docs: 7/8 — state the method's caveats, and make tutorial 08's numbers true

uipreliga
and
claude
committed
775f8fc
docs(plugin): 8/8 — extract the optimize method, and fix three seams

uipreliga
and
claude
committed
514d8cd
docs(plugin): the cost table's Stage B row priced only the activation gate

uipreliga
and
claude
committed
fec1ff0
docs(tutorials): audit all nine, and measure what tutorial 08 could only assert

uipreliga
and
claude
committed
2774bf4
docs(tutorials): name 08 for what it optimizes — a skill's description

uipreliga
and
claude
committed
edbfc63
docs(tutorials): re-run Part 2's Stage A, and let it stop at Stage A

uipreliga
and
claude
committed
94a9347
feat(stats): 1/9 — one F1 implementation, and the gate's two primitives

uipreliga
and
claude
committed
857dd5c
feat(optimize): 2/9 — the activation gate, as a library the skill drives

uipreliga
and
claude
committed
5ae26be
feat(optimize): 3/9 — price the noise floor, and guard cost and latency

uipreliga
and
claude
committed
2d09fca
docs(plugin): 4/9 — the skill's gate is the one the tool computes

uipreliga
and
claude
committed
e46d745
docs(plugin): 5/9 — the proposal is a shape, not an improvisation

uipreliga
and
claude
committed
ec8ee39
feat(optimize): 6/9 — measurements get a schema, the ledger keeps its prose

uipreliga
and
claude
committed
330505f
docs(plugin): 7/9 — establish the body does anything before optimizing its wording

uipreliga
and
claude
committed
565db6e
feat(optimize): 8/9 — the shape of a round, not just its average

uipreliga
and
claude
committed
b015f78
docs(plugin): 9/9 — halving, priced honestly

uipreliga
and
claude
committed
8e63a7b
fix: code review fixes for the optimize-skill gate corrections

uipreliga
and
claude
committed
c74e6f0
test(lint): CE038 — extra="forbid" must reach the models it appears to protect

uipreliga
and
claude
committed
92f206b
fix(stats): 1/6 — the bootstrap p is (b+1)/(m+1), not a clamped count

uipreliga
and
claude
committed
89e983a
feat(optimize): 2/6 — a gate that refuses rather than overclaiming

uipreliga
and
claude
committed
b004518
test(lint): 3/6 — CE039, prose claims checked by computing them

uipreliga
and
claude
committed
d148abf
feat(optimize): 4/6 — the execution track gets a floor, on its own metric

uipreliga
and
claude
committed
ac4ca95
feat(optimize): 5/6 — the merge shortlist is a different set from the discard one

uipreliga
and
claude
committed
ee3d719
feat(optimize): 6/6 — cost as a second axis of the shortlist, never a second gate

uipreliga
and
claude
committed
d5ab9bb
test(lint): CE040 — the bootstrap's p-floor is derived in one place

uipreliga
and
claude
committed
9e5a07f
fix(plugin): the control-arm command needs a file Step 9 authors

uipreliga
and
claude
committed
f9407ed
fix: cross-phase review findings for the optimize-gate plan

uipreliga
and
claude
committed
ffdd441
fix(optimize): cost-front coverage is a set test, not a count

uipreliga
and
claude
committed
1c0e07c
docs(optimize): 1/6 — the sentence CE039 cannot see

uipreliga
and
claude
committed
100254e
feat(optimize): 2/6 — a refusal whose remedy is true

uipreliga
and
claude
committed
197e6ba
docs(optimize): 3/6 — the sizing rule, before the money is spent

uipreliga
and
claude
committed
23a0fcf
fix(optimize): review findings on 2/6 — two false claims about the lever

uipreliga
and
claude
committed
5735206
feat(optimize): 4/6 — the regression corpus gets a reader

uipreliga
and
claude
committed
8f32fc1
feat(optimize): 5/6 — the execution track gets a computed, corrected verdict

uipreliga
and
claude
committed
4800769
feat(optimize): 6/6 — the sibling guardrail stops being opt-in

uipreliga
and
claude
committed
2e667f2
docs(harness): two deferred gaps from the optimize-gate review

uipreliga
and
claude
committed
7b0c87a
fix: code review fixes for the optimize-gate v8/v2/v3/v5/v1/v4/v6 plan

uipreliga
and
claude
committed
4868951
docs(harness): two more deferred gaps from the optimize-gate final review

uipreliga
and
claude
committed
eaee08b
feat(optimize): 1/3 — a search loop that bounds nothing, and two pointers

uipreliga
and
claude
committed
48f09bc
feat(optimize): 2/3 — an anti-memorization check that does not cry wolf

uipreliga
and
claude
committed
edeba1f
feat(optimize): 3/3 — the gold solution, and failures with names

uipreliga
and
claude
committed
1413d8b
fix: code review fixes for the ReAPO optimize-skill plan

uipreliga
and
claude
committed
9f46ebd
docs(harness): two deferred gaps from the ReAPO optimize-skill review

uipreliga
and
claude
committed
0700d34
docs(harness): the search loop's arithmetic belongs in a tested function

uipreliga
and
claude
committed
c5009f0
feat(optimize): the search loop's decision becomes tested code

uipreliga
and
claude
committed
c7dcad9
docs: bring the optimize surfaces back in line with the gate

uipreliga
and
claude
committed
c163267
feat(optimize): 1/4 — a zero-variance execution verdict is not a result

uipreliga
and
claude
committed
49ae82f
feat(optimize): 2/4 — the execution gate fails closed on a degenerate arm

uipreliga
and
claude
committed
6301a96
feat(optimize): 3/4 — a mistyped verdict field raises instead of vanishing

uipreliga
and
claude
committed
736fe9e
feat(optimize): 4/4 — the coverage front learns the guard its siblings had

uipreliga
and
claude
committed
47e6cd6
fix: code review fixes for the optimize-gate promotion-correctness plan

uipreliga
and
claude
committed
058295d
test(optimize): a fixture that builds twice under one tmp_path now fails loudly

uipreliga
and
claude
committed
7d69a35
fix(optimize): close the three gaps this plan deferred

uipreliga
and
claude
committed
1416c3a
test(optimize): 1/6 — pin the six rendered blocks before anything moves

uipreliga
and
claude
committed
39c1b7a
feat(optimize): 2/6 — the replicate-padding seam, one glob, and CE042

uipreliga
and
claude
committed
e8f69d9
feat(optimize): 3/6 — one declaration for the trim and the shared Holm notes

uipreliga
and
claude
committed
76f7018
feat(optimize): 4/6 — extract the worst complexity blocks

uipreliga
and
claude
committed
c73ac84
feat(optimize): 5/6 — SearchComparison.accepted becomes a derived property

uipreliga
and
claude
committed
42d2361
feat(optimize): 6/6 — split the presentation and sidecar halves out of the gate

uipreliga
and
claude
committed
eaf6cdd
docs(harness): two deferred guardrails from the optimize-gate module split

uipreliga
and
claude
committed
414c2e5
test(cli): 1/6 — the row selectors get a test that goes through the CLI

uipreliga
and
claude
committed
3fea4d0
feat(models): 2/6 — one RowSelection, recorded in run.json and rendered

uipreliga
and
claude
committed
e93c14e
feat(cli): 3/6 — plan previews what run actually executes

uipreliga
and
claude
committed
6f56e25
feat(optimize): 4/6 — the gate stops pairing a train run against a test run

uipreliga
and
claude
committed
5b5da35
feat(criteria): 5/6 — one label constant, and both activation suites re-measured

uipreliga
and
claude
committed
d85c2ae
docs(ci): 6/6 — the ci skill learns about splits, and the docs catch up

uipreliga
and
claude
committed
927da36
test(lint): CE038 reaches BatchRunConfig, the container that motivated it

uipreliga
and
claude
committed
d5fbe81
docs(harness): the fingerprint-diff blind spot a moved config key opens

uipreliga
and
claude
committed
56fe4ea
fix: cross-phase review findings for the row-selection plan

uipreliga
and
claude
committed
0b2b826
docs(harness): the sampler half of the cross-split refusal

uipreliga
and
claude
committed
40c45ce
test(lint): 1/6 — CE044 and CE045, the two sensors that could not see

uipreliga
and
claude
committed
2e41340
test(lint): 2/6 — CE046 and CE047, the CLI's two undocumented surfaces

uipreliga
and
claude
committed
3541a70
refactor(models): 3/6 — copy_with closes the update-key hole, CE048 holds it

uipreliga
and
claude
committed
6fa1fe2
test(optimize): 4/6 — three prose claims that were not true

uipreliga
and
claude
committed
091f2dc
test(lint): 5/6 — the method file's sign sentence is bound to the gate

uipreliga
and
claude
committed
2be1327
ci(stats): 6/6 — a rendered statistic that steps must say why

uipreliga
and
claude
committed
5712c3f
fix: cross-phase review findings for the harness-sensors plan

uipreliga
and
claude
committed
a0076d9
test(lint): guard CE id collisions the runner's assert cannot see

uipreliga
and
claude
committed
d29ef22
fix(early-stop): 1/9 — the decide_within budget stops expiring mid-call

uipreliga
and
claude
committed
da2ca2b
feat(optimize): 2/9 + 3/9 — the index gets a lower bound, the guardrail gets a veto

uipreliga
and
claude
committed
5b39088
feat(optimize): 4/9 — the gate stops trusting a run.json about a tree it does not describe

uipreliga
and
claude
committed
d25ae34
feat(cli): 5/9 — untrusted text stops being read as Rich markup, and plan prints one banner

uipreliga
and
claude
committed
7846c18
test(lint): 6/9 + 7/9 — four import rules stop failing open, and C90 gets a ratchet

uipreliga
and
claude
committed
08911e3
test(lint): 8/9 — the ci-outcome fixture loads, and CE052 makes sure the next one does

uipreliga
and
claude
committed
d3cfa7f
feat(orchestration): 9/9 — BatchRunConfig accepts the flat row selectors for one more release

uipreliga
and
claude
committed
5b5c17b
docs(harness): two deferred guards the top-10 review-fixes run surfaced

uipreliga
and
claude
committed
d8e537c
fix: cross-phase review findings for the top-10 review-fixes plan

uipreliga
and
claude
committed
30ab379
docs(stats): the ledger row also accounts for the pins' re-encoding

uipreliga
and
claude
committed
c5a91f2
fix: adversarial review findings — two guards that did not guard

uipreliga
and
claude
committed
06a9604
docs(optimize-skill): findings from the xlsx execution-track dogfood run

uipreliga
and
claude
committed
26b481b
fix(criteria): 1/3 — allowlist parity for skill_triggered's file-read gate

uipreliga
and
claude
committed
f8a1f4c
fix(criteria): 2/3 — reject an off-kind agent_judge judge config at load

uipreliga
and
claude
committed
b219bee
feat(criteria): 3/3 — name the non-engagement reason in skill_triggered details

uipreliga
and
claude
committed
80711a9
fix: code review fixes for the Plan A scoring-correctness fixes

uipreliga
and
claude
committed
884f83a
feat(lint): CE054 — a criterion compares result_status in exactly one place

uipreliga
and
claude
committed
d24cce6
docs(optimize-skill): four more guardrails the xlsx dogfood round earned

uipreliga
and
claude
committed
4103588
feat(optimize): 1/7 — every run-tree reader reconciles, and CE053 keeps it that way

uipreliga
and
claude
committed
2071799
feat(optimize): 2/7 — one meaning for `promoted` across both gates

uipreliga
and
claude
committed
a2fd568
feat(optimize): 3/7 — a refusal suppresses negative-result prose, and the MDE names its cause

uipreliga
and
claude
committed
1d75318
refactor(optimize): 4/7 — three declarations where there were six, and no E-grade left

uipreliga
and
claude
committed
7e4d1ed
refactor(optimize): 5/7 — the two row-selection preflights become one named function

uipreliga
and
claude
committed
ec38350
test(lint): 6/7 — a broken snippet import fails the build, before Phase 7 needs it

uipreliga
and
claude
committed
25a3dd5
refactor(optimize): 7/7 — one 3,521-line module becomes six, split by track

uipreliga
and
claude
committed
66f12f6
fix(optimize): the BLOCKED rung reads BOTH veto lists, on both tracks

uipreliga
and
claude
committed
8542f32
test(lint): a retired prose claim must stay retired

uipreliga
and
claude
committed
e0d65a2
fix: final cross-phase review findings

uipreliga
and
claude
committed
0a6c76a
docs(harness): a dotted module reference in prose has no sensor

uipreliga
and
claude
committed
89e623c
feat(cli): 1/5 — `plan` validates that a mounted template source can be mounted

uipreliga
and
claude
committed
64d6205
feat(plugin): 2/5 — an outcome-suite mode for `/coder-eval:task`

uipreliga
and
claude
committed
dafaf08
feat(plugin): 3/5 — ship the outcome-suite grader scaffold

uipreliga
and
claude
committed
aa37c52
feat(plugin): 4/5 — prove the grader discriminates before paying for a stage

uipreliga
and
claude
committed
6b5a1a4
test(plugin): 5/5 — the answer key must not ship inside the exam

uipreliga
and
claude
committed
065e089
fix: final cross-phase review findings for the outcome-suite plan

uipreliga
and
claude
committed
03f8b92
docs(plugin): 1/6 — an outcome suite states what it can resolve before it is paid for

uipreliga
and
claude
committed
fbc3b97
feat(plugin): 2/6 — the grader says which rule each check enforces

uipreliga
and
claude
committed
676cffe
feat(optimize): 3/6 + 4/6 — say what the suite can resolve, and what one draw is worth

uipreliga
and
claude
committed
895af7f
feat(optimize): 5/6 — record which instrument produced a round's scores

uipreliga
and
claude
committed
32c53aa
feat(lint): 6/6 — CE057, an outcome prompt may not supply what its expectations grade

uipreliga
and
claude
committed
f714580
docs(harness): a computed claim that cannot fail is a claim nobody is checking

uipreliga
and
claude
committed
45798ed
fix: final cross-phase review — the advisory layer must be honest about what it does not know

uipreliga
and
claude
committed
ee57112
test(plugin): dead code in the file users copy must not be invisible

uipreliga
and
claude
committed
d52fc68
feat(models): 2/8 — mirror `weight` onto CriterionResult, and CE058 to keep it stamped

uipreliga
and
claude
committed
0a79c86
feat(optimize): 1/8 — report the dead weight, so the gate's magnitude is auditable

uipreliga
and
claude
committed
e6b4917
feat(optimize): 3/8 — a suite fingerprint, the activation track's first instrument provenance

uipreliga
and
claude
committed
0deb166
feat(optimize): 4/8 — say what resolution the gate actually achieved

uipreliga
and
claude
committed
020cf5b
feat(optimize): 5/8 — the leak preflight reads the skill, not one file

uipreliga
and
claude
committed
26026c8
feat(optimize): 6/8 + 7/8 — a computed verdict for Stage C, and a wider candidate space

uipreliga
and
claude
committed
b64020a
feat(optimize): 8/8 — seed stability, and the rest of the proposer's surfaces

uipreliga
and
claude
committed
499fe11
test(harness): a persisted result field documented nowhere fails nothing

uipreliga
and
claude
committed
0269764
fix: final cross-phase review — three magnitudes, three samples, and four blind digests

uipreliga
and
claude
committed
1df1491
refactor(optimize): 1/8 + 2/8 — the family becomes a package, and its sensors derive their scope

uipreliga
and
claude
committed
bafa453
refactor(optimize): 3/8 — a name siblings share is public, and CE059 keeps it that way

uipreliga
and
claude
committed
66e5fad
refactor(optimize): 4/8 — one declaration of what vetoes a promotion

uipreliga
and
claude
committed
c4c4261
refactor(optimize): 5/8 — one headline ladder for both Stage B tracks

uipreliga
and
claude
committed
5470ecf
test(optimize): 6/8 — shared fixtures, and the lint layer stops reaching into a test file

uipreliga
and
claude
committed
3b7fa9e
test(optimize): 7/8 — split the 9,367-line test monolith to mirror the package

uipreliga
and
claude
committed
bdc43f3
docs: 8/8 — move the optimize rationale to the code it describes

uipreliga
and
claude
committed
8b54e96
fix: final cross-phase review — a veto with nothing on the page saying why

uipreliga
and
claude
committed
b7cb6a2
fix: reconcile the optimize-skill branch with main's CE035 and run-limit work

uipreliga
and
claude
committed
38401be
refactor(models): 1/10 — the two verdicts' shared fields, declared once

uipreliga
and
claude
committed
e1bb26e
refactor(optimize): 2/10 — one Holm loop, and therefore one `promoted`

uipreliga
and
claude
committed
07dbc59
refactor(optimize): 3/10 — one floor preflight, one first-cause sink

uipreliga
and
claude
committed
ea8c3d3
refactor(optimize): 4/10 — the execution gate becomes an orchestration function

uipreliga
and
claude
committed
7242c7b
refactor(optimize): 5/10 — load_and_pair splits along its stages

uipreliga
and
claude
committed
f9a79a3
feat(lint): 6/10 — resolving an import becomes the default path

uipreliga
and
claude
committed
c1d064c
refactor(lint): 7/10 — the evaluator's whitelist becomes its dispatch, and CE044 retires

uipreliga
and
claude
committed
3ea15f5
test(lint): 8/10 — split the lint monolith, and make the marker the contract

uipreliga
and
claude
committed
901d6dc
docs(optimize): 9/10 — three registers for three kinds of sentence

uipreliga
and
claude
committed
bbb4327
docs: 10/10 — CLAUDE.md stops restating what the code already says

uipreliga
and
claude
committed
1c04718
fix: final cross-phase review — a trim that deleted three specifications

uipreliga
and
claude
committed
3cbb9c0
test(harness): close the two pointer classes a test move breaks silently

uipreliga
and
claude
committed
820db68
feat(optimize): 1/11 — api.py, the rank-4 seam, and the two noise-floor composites

uipreliga
and
claude
committed
2a6df63
feat(optimize): 2/11 — the three fences with no branches

uipreliga
and
claude
committed
b0bea56
feat(optimize): 3/11 — headroom and corpus, and the tree they never reconciled

uipreliga
and
claude
committed
f0eb372
feat(optimize): 4/11 — the per-replicate reduction the skill hand-wrote

uipreliga
and
claude
committed
b5046dc
feat(optimize): 5/11 — the leak preflight, and the last function in a fence

uipreliga
and
claude
committed
a029b86
feat(optimize): 6/11 — search_report, and the lineage head gets a caller

uipreliga
and
claude
committed
3e0b97e
feat(optimize): 7/11 — Stage B's ordering becomes structural

uipreliga
and
claude
committed
204ceab
feat(optimize): 8/11 — execution Stage B, and the correction that shrank

uipreliga
and
claude
committed
c4d834a
feat(optimize): 9/11 — Stage C recomputes, and stops overruling rank 1

uipreliga
and
claude
committed
6b70fd7
feat(optimize): 10/11 — the ledger, and two ways a round lied about itself

uipreliga
and
claude
committed
db665f2
feat(lint): 11/11 — CE066, and the surface stops being derived

uipreliga
and
claude
committed
c970d76
test(optimize): derive the boundary registry instead of remembering it

uipreliga
and
claude
committed
6e30030
fix(optimize): final cross-phase review — two fail-open paths and a fence that could not run

uipreliga
and
claude
committed
c6cb8dd

Commits on Aug 21, 2026

fix: reconcile with main — five CE ids this branch and main both claimed

uipreliga
and
claude
committed
68fa740

Uh oh!

Commits

Commits on Aug 20, 2026