Repository navigation
Route three Codex-on-Copilot workflows with economy model routing instead of pinned GPT Sol models - #75460
Conversation
…with economy model routing Co-authored-by: SivaKesava1 <11771739+SivaKesava1@users.noreply.github.com>
|
✅ PR Specification Reviewer completed successfully!
|
|
✅ Design Decision Gate 🏗️ completed the design decision gate check. See the comment below for the result and any generated ADR draft. Design Decision Gate 🏗️Outcome: The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible. Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.
|
|
✅ PR Code Quality Reviewer completed the code quality review.
|
|
✅ Test Quality Sentinel completed test quality analysis. Test Quality SentinelOutcome: The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible. Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.
|
|
🧠 Matt Pocock Skills Reviewer has completed the skills-based review. ✅
|
|
✅ Ponytail Reviewer completed successfully! Ponytail ReviewerOutcome: The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible. Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.
|
There was a problem hiding this comment.
Reviewed the workflow source changes and the specialist pass found no actionable issues. The routing blocks and required permissions are correctly structured; generated lock-file concerns were outside the available changed-line diff.
🔎 Code quality review by PR Code Quality Reviewer · copilot · gpt50 · ~2 AIC · ⌖ ~1 AIC · ⊞ 1.6K
Comment /review to run again
There was a problem hiding this comment.
Reviewed head d315bbe. Applicable contract: specs/security-architecture-spec.md IS-01, which requires sanitization before delivery to AI agents. Blocking finding: enabling model routing adds a router conversation built from the rendered workflow prompt, which includes attacker-controlled issue and PR content. The generated routing path therefore needs an explicit sanitization or trusted-boundary guarantee before router inference. This finding is already covered by the existing inline review comment and is not duplicated. No specification synchronization proposal is eligible until the boundary is fixed and conformance evidence is added. Generated locks were inspected as available; local base-object retrieval was limited by the promisor and CA failure, so tests and runtime behavior were not claimed.
Generated by PR Specification Reviewer for #75460 · copilot · gpt50 · ~4 AIC · ⌖ ~1 AIC · ⊞ 1.6K
Comment /review-specs to run again
There was a problem hiding this comment.
Skills-Based Review 🧠
Applied /codebase-design and checked the change against the already-routed sibling workflows. No blocking issues.
📋 Key Themes & Highlights
Verified
- ✅ The
model-routingblock matches Auto-Triage Issues and Ponytail Reviewer (goal: cost,mode: economy). - ✅
copilot-requests: writeis added where it was missing. Sighthound already had it. - ✅ The Update Astro lock uses
github.tokenforCOPILOT_GITHUB_TOKEN, the same wiring as the routed siblings. The PR description says Changeset Generator matches; I did not check that lock separately.
Non-blocking
- Routing sends the rendered prompt to the router. Changeset embeds PR text, so it carries the untrusted-input exposure already flagged by code scanning. See the inline note on
changeset.md. - The routing block now repeats across many workflows. A shared import would reduce drift, but that is out of scope for this PR.
- I did not regenerate the lock files in this review. Please confirm
make recompileleaves no diff.
🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer · copilot · routed: haiku55 xhigh · ~5 AIC · ⌖ ~1 AIC · ⊞ 5.4K
Comment /matt to run again
There was a problem hiding this comment.
🔵 Needs a closer look
Switching credentials and billing sources while replacing pinned models with runtime routing leaves cost and quality unconfirmed until post-merge runs are audited.
No open issues 🎉
What changed in this PR
Moves three Codex-on-Copilot workflows to cost-oriented economy model routing, extending the routing approach already used elsewhere in gh-aw to reduce agent and threat-detection costs.
Changes:
- Removes pinned GPT Sol models and enables economy routing without adding model restrictions.
- Adds
copilot-requests: writefor Update Astro and Changeset Generator, allowing their compiled jobs to usegithub.token. - Regenerates the three lock files with routing infrastructure, runtime model metadata, and the default
detectionmodel. Prompts, triggers, and safe outputs remain unchanged.
| File | Description |
|---|---|
.github/workflows/update-astro.md |
Enables economy routing and adds Copilot request permission. |
.github/workflows/update-astro.lock.yml |
Adds generated routing support, job-token authentication, and detection defaults. |
.github/workflows/sighthound-security-scan.md |
Replaces the GPT-6 Sol pin with economy routing. |
.github/workflows/sighthound-security-scan.lock.yml |
Adds generated routing support, runtime metadata, and detection defaults. |
.github/workflows/changeset.md |
Enables economy routing and adds Copilot request permission. |
.github/workflows/changeset.lock.yml |
Adds generated routing support, job-token authentication, and detection defaults. |
Review effort: Max (auto)
|
Copilot assignment requested for the following outstanding work:
Once you start work, post this work summary as a PR comment without the leading Copilot mention, including the Sous-chef head, work, and state lines below, to record that the assignment is running. Apply the necessary fixes on this PR's existing branch, reply to each listed review thread and resolve it when addressed. Ignore feedback already answered or resolved; stop when only human review or CI remains. Do not merge or trigger CI. Sous-chef requested-head: d315bbe
|
Work summary: |
Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>
Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>
Its only finding is the prompt-injection note on the routing conversation, answered and resolved on the threads: routing only picks among allowed candidates, so the impact is cost, and the fix belongs in the compiler for all routed workflows.
Three Codex workflows on the Copilot provider pin a large GPT model: Update Astro and Changeset Generator pin
copilot/gpt-5.6-sol, and Sighthound Security Scan pinscopilot/gpt-6-sol. Threat detection inherits that pin, so it costs as much as a run on the pinned model. This PR switches all three toeconomyrouting, the same setup as Auto-Triage Issues, Ponytail Reviewer and Daily Go Test Parallelizer.Workflow sources (
update-astro.md,changeset.md,sighthound-security-scan.md)model:pin.copilot-requests: writetoupdate-astro.mdandchangeset.md. Sighthound already has it.Compiled lock files (generated, only these three)
github-copilot/gpt-*candidates. This adds the router image and a "Prepare model-routing conversation" step.github.tokeninstead of theCOPILOT_GITHUB_TOKENsecret.detectionmodel.Things to check after merge
gh aw audit <run>). A Luna pick is expected.gpt-5.6-sol:maxand costs more than about 17 AIC per run, restore itscopilot/gpt-6-solpin.