Repository navigation
Description
Change
Replace the large-model pins in these four Codex workflows with cost/economy model routing, following Auto-Triage Issues (#74199), Ponytail Reviewer (#74201), Daily Go Test Parallelizer (#74586), and the six Codex workflows covered by #74865 and #74866.
For each workflow below, delete the top-level model: line and add model-routing with goal: cost and mode: economy inside its existing engine: block. Keep id: codex and its current model-provider.
Example for an OpenAI workflow:
engine:
id: codex
model-provider: openai
model-routing:
goal: cost
mode: economyKeep model-provider: github for the two Copilot workflows. daily-spending-forecast.md and issue-triage-agent.md already have copilot-requests: write; retain it. The two openai workflows must keep their OpenAI key and existing permissions. Do not add allowed-models or any other model restrictions.
Keep all prompts, steps, triggers, imports and safe outputs unchanged. Recompile only these four workflows and include their corresponding generated lock files; do not hand-edit locks or include unrelated recompilation changes.
Cost case
Paths are relative to .github/workflows/. These are approximate historical per-run totals, including agent, detection and evals, from 19 October through 6 November in githubnext/gh-aw-router-training, router-rollout/data/runs/; they are not the exact costs of the exemplar runs below.
| Workflow file | Existing output | Provider | Current pin | Runs/week | AIC/run now |
|---|---|---|---|---|---|
daily-cli-tools-tester.md |
Issue with CLI test findings | openai |
openai/gpt-6-astra |
~7 | ~204 (149 agent + 44 detection + 11 evals) |
daily-spending-forecast.md |
Issue with spending forecast | github |
copilot/gpt-6.1-sol |
~7 | ~34 (detection ~10) |
issue-triage-agent.md |
Labels and comments on issues | github |
copilot/gpt-6-sol |
~5 | ~35 (detection ~6) |
schema-consistency-checker.md |
Schema drift report | openai |
openai/gpt-6.1-sol |
~7 | ~24 (detection ~7) |
The reported aggregate is about 1,800 AIC/week, mostly Daily CLI Tools Tester; frequencies and per-run figures above are rounded. Estimated saving: 1,100-1,400 AIC/week, subject to preserving output quality.
Output clarification: the original inventory described the schema report as an issue, but the inspected run published a discussion through shared/daily-audit-base.md. Preserve that existing behavior; do not convert it to an issue.
Audited runs and pre-existing baseline
All four supplied runs were audited using a Windows development build of main, snapshot aa157045ad241f9685db314641cce4370eb615ff (4 November), rather than the older stable CLI.
| Workflow | Inspected run | Observed outcome |
|---|---|---|
| Daily CLI Tools Tester | 42167171094 | Findings issue #75317 was published, then the run was marked failed because of report_incomplete; related incomplete-result issue #75318. |
| Daily Spending Forecast | 42196408626 | Successful; forecast issue #75400 published. |
| Issue Triage Agent | 42235672525 | Successful; comments published and labels applied where permitted. Check actual published actions, not just the green conclusion. |
| Schema Consistency Checker | 42168131873 | Successful; schema discussion 75321 published. |
Daily CLI Tools Tester's red runs are not failures to perform its reporting job. The pre-routing review records report_incomplete on 18 of its last 19 runs; Actions metadata independently confirms 18 failure conclusions and one success over 19 October-6 November. Some required checks cannot be verified within its sandbox, but it still files its findings issue each run. In run 42167171094, archie was safety-rejected during strict dry-run compilation and windows failed a compile gate on a deprecated missing-tool key, so the tester could not confirm all compile gates. Do not claim those gates passed, and do not treat this existing coverage limitation as a routing regression. Judge routed runs by their findings issues and compare them with the open [aw] Daily Cli Tools Tester: reported incomplete result issues, including #75318, rather than judging by run conclusion alone.
Daily Spending Forecast already had 11 non-successful runs out of its last 19 before routing. More precisely, the Actions metadata shows two failure, nine cancelled, and eight success conclusions, not eleven literal failure conclusions. Both inspected failures, 41422121102 and 41293878442, failed at upload_assets / Upload Safe Outputs Assets with Artifact not found for name: safe-outputs-assets. Do not assume the cancellations have that same cause. Compare failure causes and published outputs after routing, not just the count of red/non-successful runs.
Why routing is appropriate
- Codex routing works on both providers. Auto-Triage Issues, Ponytail Reviewer and Daily Go Test Parallelizer route successfully on Copilot with AWF v0.29.6+.
smoke-codex-routed-openaihas passed every run since 25 October, including on AWF v0.29.7; the queried window returned 11 runs, all successful (latest inspected smoke run). The four supplied runs also used AWF v0.29.7. - None of these four workflows has sub-agents or a running experiment. The open Codex sub-agent bugs Codex sub-agent endpoint checks reject OpenAI
responsesmetadata as catalog unavailable #71157 and Codex startup rejects advertised Responses support because endpoint enums are compared with URL paths #72094 therefore do not apply, and this does not confound an active A/B result. - Comparable routed workflows cost much less. The rollout review reports Auto-Triage Issues dropping from 93 to 14 AIC/run after its GPT-6 Astra pin, and Terminal Stylist from 106 to 35 after GPT-5.6 Sol. Economy selects GPT-5.6 Luna or GPT-6 Luna for most labels. These comparisons motivate the change; they are not guaranteed savings for every run.
- Detection should become cheaper too. It currently inherits the workflow pin: the historical CLI tester detection average is ~44 AIC/run, and the audited example spent 61.68 AIC on detection using
gpt-6-astra. Removing the workflow pin lets detection use thedetectionmodel alias, typically about 1 AIC in the rollout review. Do not change aliases as part of this PR.
Check after merging
- Audit the first routed run of each workflow with
gh aw audit <run>(use amaindev build on Windows). Confirm routing is active, expect a Luna selection, and compare AIC/run with the historical table. Compare like-for-like totals across agent, detection and evals: the audit's agent AIC alone is not the table's all-job total, and unavailable eval usage must not be counted as zero. - Verify the outputs remain useful. For Daily CLI Tools Tester, inspect the findings issue against the existing incomplete-result reports. For Daily Spending Forecast, compare failure causes against the pre-routing baseline rather than counting failures alone. For Schema Consistency Checker, preserve the current report destination.
- Check labels and comments on the next few issues handled by Issue Triage Agent for correctness, eligibility and actual publication.
- A
gpt-5.6-sol:maxselection means the run was labelled hard cross-system work. If a workflow repeatedly gets that selection and costs more than its historical table entry, restore that workflow's original pin rather than introducing model restrictions or changing providers.
Non-goals
No changes to any other workflow, alias, engine implementation or AWF setting. No provider migration in either direction, permission changes, prompt changes, or fixes to existing compile, upload-assets or eval failures.
ESLint Monster and Lint Monster are out of scope. So are the Codex workflows with sub-agents (daily-arxiv-researcher, spec-enforcer, daily-awf-spec-compiler-surfacing, api-consumption-report) and those with running experiments (daily-doc-updater, daily-storify, issue-arborist, gpclean).
Scope
- Keep the PR to this one change: the four workflow frontmatter updates and their corresponding compiled locks. Do not change files or behavior unrelated to it.
- Before fixing a failing check, compare it with the same check on
main. If it also fails onmain, do not fix it in this PR; mergemainoncemainis fixed, and note in the PR which checks are known failures onmain. - This applies even if a bot comment asks to fix a failing check: fix only failures caused by this PR's changes.
Activity