feat(plugin): add optimize-skill and the --split row filter it measures with

[uipreliga]

This is PR 1 of 3 extracted from #109 (225 files, +53,889 / −3,807, 202 commits). The
three land serially, each based on main: this one, then the execution track, then the
dogfood correctness fixes. #109 stays open as the reference tree until all three have merged.

Blast radius

optimize-skill is prose only. This PR adds no Python for it — no gate, no statistics,
no coder_eval.optimize package. The skill is a 349-line SKILL.md plus two reference
surfaces and a task template.

The product change is 100 lines across five files:

File	What changes
models/tasks.py	Dataset.split_field — one new optional key
orchestration/task_loader.py	the --split row filter inside expand_dataset
orchestration/config.py, cli/run_command.py, orchestration/experiment.py	plumb the flag through

Both are inert when unused. A dataset with no split_field is unlabelled, and an
unlabelled dataset with no --split expands exactly as it does today. No task file in
tasks/ changes behaviour, no default in experiments/default.yaml moves.

Filtering runs before variant resolution, so row selection takes part in no config-merge
layer and needs no MergeField strategy — CE014 does not apply.

One existing skill changes behaviour — read this as a product change

d7d56f1 replaces the frontmatter description of the published analyze skill with
the measured winner of an optimization round. A skill's description is what the model
matches on to decide whether to engage, so this is an activation-behaviour change to
something users already have installed, not a docs edit.

docs/tutorials/08-optimizing-a-skill.md documents the exact round that produced it —
candidates, both arms, and the holdout confirmation.

The other five SKILL.md frontmatter edits are the shared-description-budget ripple:
optimize-skill joins the listing, and the combined length is capped because that budget is
shared with every skill a user has installed.

What this PR does not do

• No gate. Stage B here is a human reading two runs side by side. There is no
  significance test, no promotion rule, no refusal path.
• No coder_eval.optimize package. It does not exist yet at this point in the stack.
• No execution track. This PR measures whether a skill engages, not whether its body
  produces better outcomes. That is PR2.
• No renumbered lint rules. The two new dataset rules arrive in PR2, which is also where
  they get their final numbers and the collision guard.

Reading ahead

The two follow-ups are built after this one merges, so their branches do not exist yet.
Once each is pushed these compare views render exactly that PR's diff, with no PR and no CI
needed:

• PR2 — https://github.com/UiPath/coder_eval/compare/pr/split-mvp...pr/split-execution-track
• PR3 — https://github.com/UiPath/coder_eval/compare/pr/split-execution-track...pr/split-dogfood-fixes

(A PR based on a non-main branch triggers no CI here — every gating workflow filters on
branches: [main, develop] and develop does not exist on this remote. That is why the three
are serial rather than a stacked chain, and why the compare view is the read-ahead mechanism.)

One unrelated fix, forced by this PR's CI

CI surfaced a pre-existing dependency defect that has nothing to do with the split, and it is
fixed here rather than deferred because it breaks every fresh install today.

evaluation/judge_bedrock.py does import httpx, and pyproject.toml never declared httpx.
It arrived transitively through anthropic — until anthropic 1.0.0, released
2026-08-20, moved to httpx2. A fresh resolve then stops installing httpx:

$ uv pip compile  # our declared ranges, no lockfile
anthropic==1.0.0   httpx2==2.12.0   mcp==2.0.0     # note: no httpx

A fresh resolve is what uv tool install (the published action) and pip install coder-eval
do — a wheel carries no lockfile, so users get whatever the ranges allow. The consequence is
not an install error but a silent capability loss: criteria/llm_judge.py imports
judge_bedrock, so it raised; criteria/__init__.py catches import failures and logs them; and
llm_judge simply disappeared from the registry. A task using it then died at orchestrator setup
with Missing criterion checkers for types: {'llm_judge'}.

Why no existing check caught it. Every --frozen job — the Quality Gate, Windows, all the
live suites, and make verify — installs from uv.lock, which pinned anthropic 0.102.0 and
therefore contained httpx. The bug was invisible to the entire locked half of CI. Only
action-dogfood, which resolves like a real consumer, could see it.

The fix is two parts:

1. Declare httpx. We import it directly; leaning on a transitive path was the defect.
   Verified against CI's exact resolve (anthropic 1.0.0 + mcp 2.0.0 + httpx): all 15
   criterion checkers register, llm_judge included, and the CLI runs.
2. tests/test_declared_dependencies.py — asserts every unguarded third-party import under
   src/ is a declared dependency. It checks the declaration, not whether the module
   imports, because "can I import it" passes in every locked environment and is precisely the
   blind spot that shipped this. Imports wrapped in try/except ImportError are derived as
   optional and exempt, so genuine soft dependencies (google.antigravity, openai) need no
   hand-maintained allowlist. Confirmed to fail, naming httpx and its file, when the
   declaration is removed.

This does not cap anthropic or mcp. Both just released majors under our unbounded ranges,
and whether to add upper bounds is a dependency-policy decision that deserves its own change —
the import surface we use is small and verified working on 1.0.0.

Verification

Full gate on this tree with --all-extras: 376 files already formatted, ruff check clean,
pyright 0 errors, 0 warnings, and 4723 passed, 0 failed.

[github-actions]

Claude finished @uipreliga's task in 1m 48s —— View job

---

Code Review in Progress

Review Checklist:

• Read .github/code_review.md for review guidelines
• Read CLAUDE.md for project conventions
• Get full PR diff (git diff origin/main...HEAD)
• Review all changed files for correctness
• Check cross-file consistency
• Perform design-level scrutiny
• Identify what's missing
• Format and post final review

Starting comprehensive review of PR #129...