Repository navigation
Closed
Copy link
Labels
agent-readyFully specified and briefed; eligible for autonomous pickup from the frontier.Fully specified and briefed; eligible for autonomous pickup from the frontier.priority: mediumReal value, no hard deadline; normal backlog flow.Real value, no hard deadline; normal backlog flow.status: readyTriaged, unblocked, and fully specified; eligible to pick up.Triaged, unblocked, and fully specified; eligible to pick up.work-class: scopedA briefed fix or small feature; blast radius bounded by the brief, tests exist.A briefed fix or small feature; blast radius bounded by the brief, tests exist.
Description
Parent #7584's reviewer-diversity eval (Q134, Q160, D101; approved as recommended by the owner on 2026-10-15, poteto-review decision row 99) runs 12 fixtures under four arms. #12139 adds the arm wiring, the graders and four reused-scaffold fixtures, and describes the case layout under plugins/review/evals/. This slice adds the four new defect fixtures from the approved Q160 design, in that layout.
Done when
-
plugins/review/evals/gains fourfanout-diversity-*fixtures, each with a scaffold that plants one defect: a race on shared state, a resource leak on an error path, a caller left on an old contract, and a test that cannot fail. - Each fixture has a ground-truth file that names its planted defect and that defect's axis (
specwhen the change fails its stated purpose, elsestandards), in the shape test(review): add the fanout reviewer-diversity eval wiring and its four reused-scaffold cases #12139 uses. - Each fixture has one case directory per arm and the three graders, each grader file following its template in
plugins/review/evals/fanout-diversity.mdand differing from test(review): add the fanout reviewer-diversity eval wiring and its four reused-scaffold cases #12139's only in the planted-defect text, and each grader has asamples/<grader>.jsonwith at least one pass and one fail entry. - Each case uses
schema_version: "1.1"incase.yamland setsruns: 3, as test(review): add the fanout reviewer-diversity eval wiring and its four reused-scaffold cases #12139's cases do. - The diff edits no file under
plugins/review/skills/and no file underplugins/review/agents/. - A
.changes/review/*.mdfragment withbump: nonedescribes the cases. - The PR's CI is green.
Out of scope
- The decoy fixtures (a sibling slice), and running the eval.
Parent: #7584
Metadata
Metadata
Assignees
Labels
agent-readyFully specified and briefed; eligible for autonomous pickup from the frontier.Fully specified and briefed; eligible for autonomous pickup from the frontier.priority: mediumReal value, no hard deadline; normal backlog flow.Real value, no hard deadline; normal backlog flow.status: readyTriaged, unblocked, and fully specified; eligible to pick up.Triaged, unblocked, and fully specified; eligible to pick up.work-class: scopedA briefed fix or small feature; blast radius bounded by the brief, tests exist.A briefed fix or small feature; blast radius bounded by the brief, tests exist.
Type
Fields
No fields configured for Task.
Activity
github-actions commentedon Nov 2, 2026
melodic-automation-lanes commentedon Nov 2, 2026
The work-resolve-issue run that
agent-readystarted waited in its slot past a change to its lane workflow files on main, so it stopped before any model turn. The run's report-outcome removed and addedagent-readyback to start a fresh run on main's workflow files. If this run ends the same way, it re-fires the issue again, waiting longer each time.github-actions commentedon Nov 2, 2026
github-actions commentedon Nov 2, 2026