feat(reviewer-eval): swap the codex reviewer to gpt-6.1-sol after two pre-registered campaigns

.claude/commands/ai-review-local.md

@@ -16,7 +16,7 @@ Two backends are supported:
 
 | Backend | Latency | Cost | Quality |
 |---|---|---|---|
-| `api` (`gpt-5.6-sol`) | 30-60s | metered at the `PRICING` rates in `openai_review.py` via `OPENAI_API_KEY`; `--backend api --dry-run` prints the estimate for your diff | Single-shot — won't grep, can't load files on its own initiative |
+| `api` (`gpt-6.1-sol`) | 30-60s (measured on gpt-5.6-sol; untested on gpt-6.1-sol, the codex backend is the validated path) | metered at the `PRICING` rates in `openai_review.py` via `OPENAI_API_KEY`; `--backend api --dry-run` prints the estimate for your diff | Single-shot — won't grep, can't load files on its own initiative |
 | `codex` (any auth) | 3-15 min | depends on your `codex login` mode (subscription vs API key) — see codex docs | Agentic — matches CI Codex reviewer, can grep / load files / multi-turn |
 
 Choose with `--backend {auto,codex,api}` (default `auto`):
@@ -66,7 +66,7 @@ Notes:
   *Api backend only.*
 - `--force-fresh`: Skip delta-diff mode, run a full fresh review even if previous state exists
 - `--full-registry`: Include the entire REGISTRY.md instead of selective sections
-- `--model <name>`: Override the model (default: `gpt-5.6-sol`). Applies to both backends.
+- `--model <name>`: Override the model (default: `gpt-6.1-sol`). Applies to both backends.
 - `--timeout <seconds>`: applies to both backends: on the api backend an HTTP request timeout (900 s default for reasoning models: gpt-5.4/gpt-5.5/gpt-5.6*/gpt-6*, *-pro, o1/o3/o4; 300 otherwise), on the codex backend a hard wall-clock cap on the codex process (uncapped when omitted). `0` means no timeout on either backend.
 - `--dry-run`: Print the compiled prompt without invoking the chosen backend
   (no API call, no codex subprocess)
@@ -108,7 +108,7 @@ Step 5 invokes the chosen backend:
 
 Parse `$ARGUMENTS` for the optional flags listed above. All flags are optional —
 the default behavior (auto-detect backend, standard context for api or
-agentic loading for codex, selective registry, gpt-5.6-sol)
+agentic loading for codex, selective registry, gpt-6.1-sol)
 requires no arguments.
 
 ### Step 2: Validate Prerequisites
@@ -419,7 +419,7 @@ Note: `--force-fresh` is a skill-only flag — it controls whether delta diffs a
 generated in Step 4 and is NOT passed to the script.
 
 **Reasoning model handling:** If the model starts with `gpt-5.4`/`gpt-5.5`/`gpt-5.6`/`gpt-6`, contains `-pro`, or starts with
-`o1`/`o3`/`o4` (e.g., `gpt-5.6-sol`, `gpt-6-astra`, `gpt-5.4-pro`, `o3`, `o4-mini`):
+`o1`/`o3`/`o4` (e.g., `gpt-6.1-sol`, `gpt-6-astra`, `gpt-5.4-pro`, `o3`, `o4-mini`):
 - The script auto-resolves `--timeout` to 900s for reasoning models when omitted, so
   no extra flag is required unless overriding
 - Run the Bash command with `run_in_background: true` (bypasses the 600s Bash tool timeout cap)

.claude/scripts/openai_review.py

@@ -878,6 +878,7 @@ def apply_token_budget(
     "gpt-5.6-luna": (0.20, 1.20),
     "gpt-6-astra": (10.00, 50.00),
     "gpt-6-sol": (2.00, 10.00),
+    "gpt-6.1-sol": (2.00, 10.00),
     "gpt-6-luna": (0.10, 0.50),
     "gpt-4.1": (2.00, 8.00),
     "gpt-4.1-mini": (0.40, 1.60),
@@ -1142,7 +1143,7 @@ def compile_prompt(
 # ---------------------------------------------------------------------------
 
 ENDPOINT = "https://api.openai.com/v1/responses"
-DEFAULT_MODEL = "gpt-5.6-sol"
+DEFAULT_MODEL = "gpt-6.1-sol"
 DEFAULT_TIMEOUT = 300  # seconds
 REASONING_TIMEOUT = 900  # seconds
 DEFAULT_MAX_TOKENS = 16384
@@ -1347,7 +1348,7 @@ def _build_codex_cmd(
 ) -> "list[str]":
     """Construct the argv for `codex exec`.
 
-    Pinned to match the CI Codex action's invocation (gpt-5.6-sol + xhigh effort +
+    Pinned to match the CI Codex action's invocation (gpt-6.1-sol + xhigh effort +
     read-only sandbox) so local reviews give CI-equivalent quality. ``effort``
     defaults to the production/CI value (xhigh) so this argv stays byte-identical
     for production callers; the reviewer-eval harness passes other levels

.claude/skills/plan-review/SKILL.md

@@ -1,14 +1,17 @@
 ---
 name: plan-review
-description: Review a Claude Code plan file with the campaign-selected dual engine (Opus reviewer + codex-sol, merge/verify), or revise a plan from its review. Produces the .review.md the ExitPlanMode content-hash gate requires.
+description: Review a Claude Code plan file with the dual engine (Opus reviewer + codex gpt-6.1-sol, merge/verify; the Opus+Sol pair was Campaign-1 selected, run with gpt-5.6-sol, and 6.1-sol is carried over from the 2026-10 code-review campaign), or revise a plan from its review. Produces the .review.md the ExitPlanMode content-hash gate requires.
 ---
 
 # Plan Review (dual engine)
 
 Owns the plan-review → revise cycle. The review ENGINE is **dual** — two blind
-reviewers (Claude @ Opus 4.8 + codex `gpt-5.6-sol`) then a merge/verify pass —
-the configuration Campaign 1 selected: its exploratory run had dual reliably
-catch **7/9** must-catch plan defects vs **1/9** for any single reviewer. That
+reviewers (Claude @ Opus 4.8 + codex `gpt-6.1-sol`) then a merge/verify pass —
+the configuration Campaign 1 selected (measured with gpt-5.6-sol; the codex pin
+now tracks the production code-review model, carried over from the 2026-10
+code-review campaign 2 and not re-validated by plan-review-eval): its
+exploratory run had dual reliably catch **7/9** must-catch plan defects vs
+**1/9** for any single reviewer (measured with gpt-5.6-sol). That
 campaign was **NON-GATING** (contaminated negatives + a criteria regression);
 the sensitivity signal is what selects dual, and a clean re-validation is
 tracked — see `tools/plan-review-eval/verdicts/campaign-1.md`. The content-hash
@@ -27,8 +30,9 @@ Bundled in this skill dir (`.claude/skills/plan-review/`):
 - `render.py` — strict `__TOKEN__` renderer. **Always render prompts with it;
   never free-text-substitute the templates** (that is a different, unmeasured
   engine).
-- `codex_review.py` — the codex (reviewer 2) half, with the campaign
-  model/effort pins and loud-fallback exit codes.
+- `codex_review.py` — the codex (reviewer 2) half, with the production-tracked
+  model/effort pins (see its header comment for validation status) and
+  loud-fallback exit codes.
 
 ## Modes
 
@@ -137,7 +141,7 @@ read-only intent — whose prompt is the exact contents of
 and returns the findings list + summary table. Write its output to
 `<work_dir>/review_a.md`.
 
-### 4. Reviewer 2 — codex-sol (blind) — DUAL mode only
+### 4. Reviewer 2 — codex gpt-6.1-sol (blind) — DUAL mode only
 
 **Single mode**: skip steps 4-5 and take the **Single-reviewer mode
 (deliberate)** path below — the review body is reviewer 1's output with the
@@ -167,7 +171,7 @@ python3 ".claude/skills/plan-review/codex_review.py" \
 - **Exit 0** → codex review is in `<work_dir>/review_b.md`; go to step 5.
 - **Non-zero (2 = codex absent, 3 = timeout/error)** → codex is unavailable;
   take the **Loud fallback** below and skip steps 5-6. A hung codex cannot
-  wedge the gate — `codex_review.py` caps at 2400s and exits 3.
+  wedge the gate — `codex_review.py` caps at 3600s and exits 3.
 
 ### 5. Merge + verify — Claude @ Opus
 
@@ -200,7 +204,7 @@ verbatim, then the contents of `<work_dir>/review_a.md`:
 
 ```
 > ⚠ **codex unavailable — SINGLE-Claude fallback.** This is the un-validated
-> weak arm (Campaign 1: caught 1/9 hard plan defects vs 7/9 for the dual
+> weak arm (Campaign 1, measured with gpt-5.6-sol: caught 1/9 hard plan defects vs 7/9 for the dual
 > engine). Run `codex login` and re-review for full coverage.
 ```
 
@@ -216,7 +220,7 @@ failure), then the contents of `<work_dir>/review_a.md`:
 
 ```
 > Single-Opus review (deliberate one-reviewer choice, not the dual engine).
-> The dual engine caught 7/9 hard plan defects vs 1/9 single in Campaign 1 —
+> The dual engine caught 7/9 hard plan defects vs 1/9 single in Campaign 1 (measured with gpt-5.6-sol) —
 > re-review in dual mode if this plan turns out higher-risk than expected.
 ```
 

.claude/skills/plan-review/codex_review.py

@@ -2,7 +2,7 @@
 
 Called by SKILL.md with an already-rendered reviewer prompt. Detects codex,
 invokes `openai_review.call_codex` with the campaign-graded model/effort
-pins (gpt-5.6-sol @ xhigh, arm C as-run) under a read-only sandbox, writes the codex
+pins (gpt-6.1-sol @ xhigh; the production code-review pin) under a read-only sandbox, writes the codex
 review. Before invoking, it runs openai_review's own sensitive-file scan +
 stderr notice (the codex `--cd` read surface is the whole repo — the same
 notice the `/ai-review-local` codex path prints). Exit codes let SKILL.md fall
@@ -21,15 +21,24 @@
 import shutil
 import sys
 
-# Campaign-graded codex pins (Campaign 1: arm C, as-run — the campaign was
-# exploratory / non-gating). Do not change without re-validation —
-# tests/test_plan_review_skill.py asserts these.
-CODEX_MODEL = "gpt-5.6-sol"
+# Codex pins follow the PRODUCTION code-review model (lockstep-tested against the
+# CI workflow and openai_review.DEFAULT_MODEL). gpt-5.6-sol was the engine
+# Campaign 1 graded (arm C, as-run; exploratory / non-gating); gpt-6.1-sol was
+# carried over by the 2026-10 code-review campaign 2
+# (tools/reviewer-eval/campaigns/2026-10-gpt61sol-VERDICT.md), NOT re-run through
+# plan-review-eval. To re-validate: first re-pin
+# tools/plan-review-eval/config/configs.json cli_version (the composite
+# "<claude-code version> | codex-cli <version>" string on all five arms) to the
+# live Claude Code + codex CLI pair, add a gpt-6.1-sol arm, run that harness
+# against the sol baseline (TODO.md row). tests/test_plan_review_skill.py asserts
+# these pins.
+CODEX_MODEL = "gpt-6.1-sol"
 CODEX_EFFORT = "xhigh"
-# The timeout is NOT the campaign value (the campaign ran unattended at
-# CODEX_TIMEOUT_S=3600). It is an interactive-gate ceiling: high enough to clear
-# real plan-review runtimes with room to spare, low enough to bound the
-# interactive wait. A timeout is treated as codex being unavailable (exit 3 →
+# The timeout equals the code-review harness's CALL_TIMEOUT_S
+# (tools/reviewer-eval/adapters/codex_reviewer.py; tests/test_plan_review_skill.py
+# pins the equality). It is still an interactive-gate ceiling, not a campaign
+# setting: high enough to clear real plan-review runtimes with room to spare,
+# low enough to bound the interactive wait. A timeout is treated as codex being unavailable (exit 3 →
 # LOUD single-Claude fallback), so err generous.
 #
 # Raised 1200 → 2400 on 2026-08-08. The old value was calibrated to the campaign's
@@ -45,7 +54,13 @@
 # Changing this does NOT require engine re-validation: the "do not change without
 # re-validation" note above binds CODEX_MODEL and CODEX_EFFORT (what Campaign 1
 # graded). The ceiling is an operational knob and is graded by nothing.
-CODEX_TIMEOUT_S = 2400.0
+#
+# Raised 2400 -> 3600 on 2026-10-17 with the gpt-6.1-sol swap, to equal the
+# code-review harness ceiling (the 2026-10 campaigns measured 6.1-sol at
+# ~180 s mean / 277 s p95 per code review, faster than 5.6-sol, but plan review
+# is the slower surface and the two pins are kept in lockstep); still a hard
+# cap, exit 3 = loud single-Claude fallback.
+CODEX_TIMEOUT_S = 3600.0
 
 
 def _load_openai_review(repo_root: str):

.github/workflows/ai_pr_review.yml

@@ -530,7 +530,7 @@ jobs:
           # configs.json pin + a harness smoke (the lockstep test enforces equality).
           codex-version: 0.162.1
           # Recommended by OpenAI for review quality/consistency:
-          model: gpt-5.6-sol
+          model: gpt-6.1-sol
           effort: xhigh
 
       - name: Post PR comment (new on every event except initial open)

CLAUDE.md

@@ -250,9 +250,11 @@ Before calling `ExitPlanMode`, ALWAYS offer all three review options via
 `AskUserQuestion`, with the `(Recommended)` tag chosen ADAPTIVELY from the
 plan's complexity and risk (not a fixed default) — set exactly one:
 - **Dual review** — the campaign-selected engine (two blind reviewers, Claude @
-  Opus + codex `gpt-5.6-sol`, then merge/verify; Campaign 1 was exploratory /
-  NON-GATING but showed dual catching 7/9 must-catch plan defects vs 1/9
-  single). Recommend for SUBSTANTIVE / high-risk plans:
+  Opus + codex `gpt-6.1-sol`, then merge/verify; Campaign 1, run with gpt-5.6-sol,
+  was exploratory / NON-GATING but showed dual catching 7/9 must-catch plan
+  defects vs 1/9 single; gpt-6.1-sol is carried over from the 2026-10 code-review
+  campaign and not re-validated by plan-review-eval). Recommend for SUBSTANTIVE /
+  high-risk plans:
   estimator, methodology, variance/inference, or `docs/methodology/REGISTRY.md`
   changes; multi-file, architectural, or public-API changes.
 - **Single review** (Opus only, no codex) — recommend for LOCALIZED, mechanical
@@ -264,7 +266,7 @@ plan's complexity and risk (not a fixed default) — set exactly one:
 (`.claude/skills/plan-review/`) in the chosen mode. Its Review phase snapshots
 the plan via the tested helper (confirm the printed `plan_path` — the ingress
 file is shared per-worktree), runs the reviewers (dual = Opus reviewer + codex
-`gpt-5.6-sol` + merge/verify; single = the Opus reviewer alone), writes the
+`gpt-6.1-sol` + merge/verify; single = the Opus reviewer alone), writes the
 review to the helper-derived `review_path`, and runs `plan_snapshot.py persist`
 (which re-verifies the live plan against the recorded snapshot — exit 3 = plan
 changed mid-review, not persisted, re-review — stamps `plan:`/`plan_sha256:`

DEFERRED.md

@@ -123,7 +123,6 @@ For survey-specific limitations (`NotImplementedError` paths), see the
 | `HeterogeneousAdoptionDiD` `covariates=` (Theorem 6 multivariate-covariate extension) not implemented — `fit(covariates=...)` raises `NotImplementedError` via the shipped future-work trap (locked by the `test_had.py` / `test_methodology_had.py` L73 tests); the deferred work is the Theorem 6 extension itself. | `had.py` | Phase 2a | Low |
 | MultiPeriodDiD wild bootstrap not supported (falls back to analytical, n_bootstrap-independent) — user-facing edge-case limitation; the 4.0 removal replaces the fallback with a raise (v4-design §4.1), and the merged TWFE event-study mode already RAISES since 3.9 (Phase 3(a)) — this row now governs only the deprecated class itself. | `estimators.py` (MultiPeriodDiD.fit wild-fallback block) | — | Low |
 | `predict()` raises `NotImplementedError` — rarely needed; user-facing limitation. | `estimators.py:890-911` | — | Low |
-| `gpt-6-sol` cost-fallback probe for the Codex reviewer: a ChatGPT-account `codex login` rejects `gpt-6-sol` (400, "not supported when using Codex with a ChatGPT account"), so the local harness cannot run a cheaper-tier arm; needs an API-key codex login (metered) before an arm can be added. | `tools/reviewer-eval/config/configs.json` | #849 | Low |
 
 ## Version-gated (v4)