Skip to content

Route three Codex-on-Copilot workflows with economy model routing instead of pinned GPT Sol models - #75460

Merged
SivaKesava1 merged 4 commits into
mainfrom
copilot/model-routing-update-astro-changeset-scan
Nov 6, 2026
Merged

SivaKesava1 merged 4 commits into
mainfrom
copilot/model-routing-update-astro-changeset-scan

Conversation

Copilot AI commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

Three Codex workflows on the Copilot provider pin a large GPT model: Update Astro and Changeset Generator pin copilot/gpt-5.6-sol, and Sighthound Security Scan pins copilot/gpt-6-sol. Threat detection inherits that pin, so it costs as much as a run on the pinned model. This PR switches all three to economy routing, the same setup as Auto-Triage Issues, Ponytail Reviewer and Daily Go Test Parallelizer.

Workflow sources (update-astro.md, changeset.md, sighthound-security-scan.md)

  • Removed the top-level model: pin.
  • Added routing to the existing engine block:
    engine:
      id: codex
      model-provider: github
      model-routing:
        goal: cost
        mode: economy
  • Added copilot-requests: write to update-astro.md and changeset.md. Sighthound already has it.
  • Prompts, triggers, steps and safe outputs are unchanged. No model restrictions are added.

Compiled lock files (generated, only these three)

  • Routing is enabled with github-copilot/gpt-* candidates. This adds the router image and a "Prepare model-routing conversation" step.
  • Update Astro and Changeset Generator now use github.token instead of the COPILOT_GITHUB_TOKEN secret.
  • Threat detection drops the pin and uses the detection model.

Things to check after merge

  • Audit the first routed runs (gh aw audit <run>). A Luna pick is expected.
  • If Sighthound keeps picking gpt-5.6-sol:max and costs more than about 17 AIC per run, restore its copilot/gpt-6-sol pin.
  • Confirm Update Astro's next PRs still pass the docs build.

…with economy model routing

Co-authored-by: SivaKesava1 <11771739+SivaKesava1@users.noreply.github.com>
Copilot AI changed the title [WIP] Route Update Astro, Changeset Generator and Sighthound Security Scan to economy Route three Codex-on-Copilot workflows with economy model routing instead of pinned GPT Sol models Nov 6, 2026
Copilot AI requested a review from SivaKesava1 November 6, 2026 18:57
Comment thread .github/workflows/update-astro.lock.yml Resolved
@SivaKesava1
SivaKesava1 marked this pull request as ready for review November 6, 2026 19:41
Copilot AI auto review requested due to automatic review settings November 6, 2026 19:41
@github-actions

github-actions Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

✅ PR Specification Reviewer completed successfully!

Generated by PR Specification Reviewer for #75460

@github-actions

github-actions Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

✅ Design Decision Gate 🏗️ completed the design decision gate check. See the comment below for the result and any generated ADR draft.

Design Decision Gate 🏗️

Outcome: reported_no_action

The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible.

Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.

Generated from Design Decision Gate 🏗️ · routed: gpt56 max · ~1 AIC · ⌖ ~2 AIC · ⊞ 7.7K · ◷

🏗️ ADR gate enforced by Design Decision Gate 🏗️

@github-actions

github-actions Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

✅ PR Code Quality Reviewer completed the code quality review.

🔎 Code quality review by PR Code Quality Reviewer

@github-actions

github-actions Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

✅ Test Quality Sentinel completed test quality analysis.

Test Quality Sentinel

Outcome: reported_no_action

The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible.

Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.

Generated from Test Quality Sentinel · routed: gpt56 max · ~1 AIC · ⌖ ~1 AIC · ⊞ 6.3K · ◷

🧪 Test quality analysis by Test Quality Sentinel

@github-actions

github-actions Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

🧠 Matt Pocock Skills Reviewer has completed the skills-based review. ✅

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer

@github-actions

github-actions Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

✅ Ponytail Reviewer completed successfully!

Ponytail Reviewer

Outcome: reported_no_action

The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible.

Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.

Generated from Ponytail Reviewer · routed: gpt56 max · ~2 AIC · ⌖ ~1 AIC · ⊞ 118 · ◷

Generated by Ponytail Reviewer for #75460

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the workflow source changes and the specialist pass found no actionable issues. The routing blocks and required permissions are correctly structured; generated lock-file concerns were outside the available changed-line diff.

🔎 Code quality review by PR Code Quality Reviewer · copilot · gpt50 · ~2 AIC · ⌖ ~1 AIC · ⊞ 1.6K
Comment /review to run again

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed head d315bbe. Applicable contract: specs/security-architecture-spec.md IS-01, which requires sanitization before delivery to AI agents. Blocking finding: enabling model routing adds a router conversation built from the rendered workflow prompt, which includes attacker-controlled issue and PR content. The generated routing path therefore needs an explicit sanitization or trusted-boundary guarantee before router inference. This finding is already covered by the existing inline review comment and is not duplicated. No specification synchronization proposal is eligible until the boundary is fixed and conformance evidence is added. Generated locks were inspected as available; local base-object retrieval was limited by the promisor and CA failure, so tests and runtime behavior were not claimed.

Generated by PR Specification Reviewer for #75460 · copilot · gpt50 · ~4 AIC · ⌖ ~1 AIC · ⊞ 1.6K
Comment /review-specs to run again

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Skills-Based Review 🧠

Applied /codebase-design and checked the change against the already-routed sibling workflows. No blocking issues.

📋 Key Themes & Highlights

Verified

  • ✅ The model-routing block matches Auto-Triage Issues and Ponytail Reviewer (goal: cost, mode: economy).
  • ✅ copilot-requests: write is added where it was missing. Sighthound already had it.
  • ✅ The Update Astro lock uses github.token for COPILOT_GITHUB_TOKEN, the same wiring as the routed siblings. The PR description says Changeset Generator matches; I did not check that lock separately.

Non-blocking

  • Routing sends the rendered prompt to the router. Changeset embeds PR text, so it carries the untrusted-input exposure already flagged by code scanning. See the inline note on changeset.md.
  • The routing block now repeats across many workflows. A shared import would reduce drift, but that is out of scope for this PR.
  • I did not regenerate the lock files in this review. Please confirm make recompile leaves no diff.

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer · copilot · routed: haiku55 xhigh · ~5 AIC · ⌖ ~1 AIC · ⊞ 5.4K
Comment /matt to run again

Comment thread .github/workflows/changeset.md Resolved

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Switching credentials and billing sources while replacing pinned models with runtime routing leaves cost and quality unconfirmed until post-merge runs are audited.

No open issues 🎉

What changed in this PR

Moves three Codex-on-Copilot workflows to cost-oriented economy model routing, extending the routing approach already used elsewhere in gh-aw to reduce agent and threat-detection costs.

Changes:

  • Removes pinned GPT Sol models and enables economy routing without adding model restrictions.
  • Adds copilot-requests: write for Update Astro and Changeset Generator, allowing their compiled jobs to use github.token.
  • Regenerates the three lock files with routing infrastructure, runtime model metadata, and the default detection model. Prompts, triggers, and safe outputs remain unchanged.
File Description
.github/​workflows/​update-astro.md Enables economy routing and adds Copilot request permission.
.github/​workflows/​update-astro.lock.yml Adds generated routing support, job-token authentication, and detection defaults.
.github/​workflows/​sighthound-security-scan.md Replaces the GPT-6 Sol pin with economy routing.
.github/​workflows/​sighthound-security-scan.lock.yml Adds generated routing support, runtime metadata, and detection defaults.
.github/​workflows/​changeset.md Enables economy routing and adds Copilot request permission.
.github/​workflows/​changeset.lock.yml Adds generated routing support, job-token authentication, and detection defaults.

Review effort: Max (auto)

@gh-aw-bot

Copy link
Copy Markdown
Collaborator

@copilot

Copilot assignment requested for the following outstanding work:

  1. Update this branch with the latest main using make merge-main, resolving any conflicts and preserving the intended changes.
  2. Review (.github/workflows/changeset.md:21): Non-blocking security note: this workflow embeds sanitized PR text (steps.sanitized.outputs.text) in its prompt, and the model-routing step sends the rendered prompt to the router. This is the same exposure that the code-scanning alert on update-astro.lock.yml describes. Routing only selects among candidate models and does not change tools or permissions, so the impact is cost steering rather than privilege escalation. The fix belongs upstream in the compiler, so I would track it there rather than block this PR. - Route three Codex-on-Copilot workflows with economy model routing instead of pinned GPT Sol models #75460 (comment)

Once you start work, post this work summary as a PR comment without the leading Copilot mention, including the Sous-chef head, work, and state lines below, to record that the assignment is running. Apply the necessary fixes on this PR's existing branch, reply to each listed review thread and resolve it when addressed. Ignore feedback already answered or resolved; stop when only human review or CI remains. Do not merge or trigger CI.

Sous-chef requested-head: d315bbe
Sous-chef requested-work: 33df97069ae87fb860b43b6659a1b95b3a362e78038df91e4eb05fbf7f8eeadb
Sous-chef requested-state: 424c179a00576efb22db97eaba952db1b48dd934715e62bbc1b626d97d56af07

Generated by 👨🍳 PR Sous Chef · pi · haiku45 · ~11 AIC · ⌖ ~11 AIC · ⊞ 1.2K · ◷
Comment /souschef to run again

Copilot AI commented Nov 6, 2026

Copy link
Copy Markdown
Contributor Author

@copilot

Copilot assignment requested for the following outstanding work:...

Work summary:
Sous-chef requested-head: d315bbe
Sous-chef requested-work: 33df97069ae87fb860b43b6659a1b95b3a362e78038df91e4eb05fbf7f8eeadb
Sous-chef requested-state: 424c179a00576efb22db97eaba952db1b48dd934715e62bbc1b626d97d56af07

Copilot AI and others added 2 commits November 6, 2026 21:23
Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>
Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>
Copilot AI requested a review from gh-aw-bot November 6, 2026 21:36
Comment thread .github/workflows/changeset.lock.yml Resolved
@SivaKesava1
SivaKesava1 dismissed github-actions[bot]’s stale review November 6, 2026 21:57

Its only finding is the prompt-injection note on the routing conversation, answered and resolved on the threads: routing only picks among allowed candidates, so the impact is cost, and the fix belongs in the compiler for all routed workflows.

@SivaKesava1
SivaKesava1 merged commit a71f85c into main Nov 6, 2026
41 checks passed
@SivaKesava1
SivaKesava1 deleted the copilot/model-routing-update-astro-changeset-scan branch November 6, 2026 21:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Model routing: route Update Astro, Changeset Generator and Sighthound Security Scan (Codex on Copilot) instead of pinning GPT-5.6 Sol and GPT-6 Sol

5 participants