Skip to content

Python: Sum streamed GitHubCopilotAgent usage across model calls - #10088

Merged
Eduard van Valkenburg (eavanvalkenburg) merged 1 commit into
microsoft:mainfrom
droideronline:fix/github-copilot-streaming-usage-accumulation
Nov 6, 2026
Merged

Eduard van Valkenburg (eavanvalkenburg) merged 1 commit into
microsoft:mainfrom
droideronline:fix/github-copilot-streaming-usage-accumulation

Conversation

@droideronline

Copy link
Copy Markdown
Contributor

Motivation & Context

A streamed GitHubCopilotAgent run that makes several model calls (for example model call, tool call, model call) reports only the usage of the last call. Earlier calls are dropped, so the final usage_details undercounts the run. Anything that records or limits usage from streamed Copilot runs sees only a fraction of the real tokens. The non-streaming path already sums correctly.

Description & Review Guide

Each ASSISTANT_USAGE event carries the usage of one model call, not a cumulative total. The streaming handler emitted it with merge_mode="replace", which core treats as "overwrite the running value" (Content._add_usage_content). Each event therefore overwrote the previous one.

  • What are the major changes? The streaming ASSISTANT_USAGE handler now emits usage with merge_mode="append", so core sums the per-call usage with add_usage_details. A regression test streams three usage events (100/10, 200/20, 300/30) and asserts the final response reports 600/60 (660 total). The test fails on main with 300/30.
  • What is the impact of these changes? Streamed runs now report the same total usage as non-streamed runs. No API change.
  • What do you want reviewers to focus on? Whether append is the right choice versus accumulating a running total in the adapter and emitting it with replace. I chose append because the events are per-call deltas and it is the smaller change.

Related Issue

Fixes #10087

AI Assistance

  • No material AI assistance was used.
  • This is an AI-assisted contribution. I reviewed, understood, and verified all submitted content and accept responsibility for it.

AI assistance details: implementation, tests, and this PR description.

Contribution Checklist

  • The code builds clean without any errors or warnings
  • All unit tests pass, and I have added new tests where possible
  • The PR follows the Contribution Guidelines
  • This PR links to an agreed issue with no competing open PR, or the Related Issue section documents a trivial-change or repository-automation exception.
  • This is not a breaking change. If it is a breaking change, add the breaking change label (or add "[BREAKING]" to the title prefix, before or after any language prefix) — a workflow keeps the label and title prefix in sync automatically.

Each ASSISTANT_USAGE event carries the usage of a single model call, but
the streaming handler emitted it with merge_mode="replace", so core kept
only the last call. Emit it with merge_mode="append" so the calls are
summed, matching the non-streaming path.
Copilot AI balanced review requested due to automatic review settings November 4, 2026 08:42

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

@eavanvalkenburg

Copy link
Copy Markdown
Member

/review

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MAF Automated Review — Iteration 1

Result: No findings
Scope: full PR (1 commit(s)): 1d421d17a1d4
Model: gpt-5.6-sol

Overview

The PR changes streamed ASSISTANT_USAGE updates from replacement to additive merging so core aggregation sums per-model-call deltas, matching the existing non-streaming path. Core's usage merge contract handles missing and invalid counters, while the added regression and existing streaming and non-streaming tests exercise the intended behavior. No Critical, High, or Medium PR-introduced defect was established.

Reviewed the supplied pull-request change set across correctness, security/reliability, architecture, and failure behavior.
No publishable findings remained after source verification for this scope.

Merged via the queue into microsoft:main with commit 2fa7b9a Nov 6, 2026
58 checks passed

This branch was previously deployed

1 inactive deployment
github-app-auth — 1d421d17 Deployed Nov 4, 2026 by droideronline via team_check #8684
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

python Usage: [Issues, PRs], Target: Python

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Python: [Bug]: GitHubCopilotAgent streaming undercounts usage across multiple model calls

3 participants