Repository navigation
Python: Vertex AI chat completion usage now counts thinking tokens - #14722
Closed
alexcastgonz (pcbeingused333) wants to merge 1 commit into
Closed
alexcastgonz (pcbeingused333) wants to merge 1 commit into
alexcastgonz (pcbeingused333) wants to merge 1 commit into
Conversation
alexcastgonz (pcbeingused333)
temporarily deployed
to
github-app-auth
November 4, 2026 23:08 — with
GitHub Actions
Inactive
Contributor
There was a problem hiding this comment.
🟢 Approval recommended
No active review issues remain for the change to include thinking tokens in Vertex AI completion-token usage.
No open issues 🎉
What changed in this PR
Updates Vertex AI chat completion usage to include thinking tokens as output tokens, aligning it with the Google AI connector change in #14671.
Changes:
- Calculates completion tokens as candidate tokens plus thinking tokens.
- Adds a regression test verifying that 10 candidate tokens and 40 thinking tokens produce 50 completion tokens.
| File | Description |
|---|---|
python/tests/unit/connectors/ai/google/vertex_ai/services/test_vertex_ai_chat_completion.py |
Adds coverage for usage that includes thinking tokens. |
python/semantic_kernel/connectors/ai/google/vertex_ai/services/vertex_ai_chat_completion.py |
Includes thinking tokens in completion-token metadata. |
Review effort: Balanced
Contributor
Author
|
Closing in favor of #14716, which carries the same Vertex AI thinking-token usage fix for chat completion and also covers the text completion path. Sorry for the duplicate. |
This branch was previously deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation and Context
Follow-up to #14671, which did the same for the Google AI connector.
VertexAIChatCompletionreportedcompletion_tokensfromcandidates_token_countalone. Vertex AI reports thinking tokens separately inthoughts_token_countbut bills them as output tokens, so for thinking models the usage metadata under-counted the completion tokens.Description
_get_metadata_from_responsenow reportscandidates_token_count + thoughts_token_countascompletion_tokens. The field defaults to 0 when the model does not think, so non-thinking responses are unchanged.Tests
Added
test_vertex_ai_chat_completion_usage_includes_thinking_tokens(10 candidate + 40 thinking tokens -> 50). It fails before the change (10 != 50) and passes with it; the wholevertex_aiunit-test folder passes (57) and ruff format/check are clean. The response is built by hand; no live Vertex AI call was made.Contribution Checklist
This change was made with Claude Code.