Repository navigation
Python: count Vertex AI thinking tokens in completion_tokens usage - #14736
Open
alexcastgonz (pcbeingused333) wants to merge 1 commit into
Open
alexcastgonz (pcbeingused333) wants to merge 1 commit into
alexcastgonz (pcbeingused333) wants to merge 1 commit into
Conversation
alexcastgonz (pcbeingused333)
temporarily deployed
to
github-app-auth
November 7, 2026 04:05 — with
GitHub Actions
Inactive
Contributor
There was a problem hiding this comment.
🟢 Approval recommended
There are no active review issues for the update to include thinking tokens in Vertex AI completion-token usage.
No open issues 🎉
What changed in this PR
Updates the Python Vertex AI connectors to include thinking tokens in completion-token usage, aligning their accounting with the Google AI connector change in #14671.
Changes:
- Both chat and text services now report candidate tokens plus thinking tokens as
completion_tokens. - Adds a regression test for each service verifying that 10 candidate tokens and 40 thinking tokens produce 50 completion tokens.
| File | Description |
|---|---|
python/tests/unit/connectors/ai/google/vertex_ai/services/test_vertex_ai_text_completion.py |
Tests text-completion usage with thinking tokens. |
python/tests/unit/connectors/ai/google/vertex_ai/services/test_vertex_ai_chat_completion.py |
Tests chat-completion usage with thinking tokens. |
python/semantic_kernel/connectors/ai/google/vertex_ai/services/vertex_ai_text_completion.py |
Includes thinking tokens in text-completion usage. |
python/semantic_kernel/connectors/ai/google/vertex_ai/services/vertex_ai_chat_completion.py |
Includes thinking tokens in chat-completion usage. |
Review effort: Balanced
This branch was previously deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation and Context
Follow-up to #14671, which did the same for the Google AI connector.
VertexAIChatCompletionandVertexAITextCompletionreportedcompletion_tokensfromcandidates_token_countalone. Vertex AI reports thinking tokens separately inthoughts_token_countbut bills them as output tokens, so for thinking models the usage metadata under-counted the completion tokens.Description
_get_metadata_from_responsein both services now reportscandidates_token_count + thoughts_token_countascompletion_tokens. The field is 0 when the model does not think, so non-thinking responses are unchanged.Tests
Added one usage test per service (10 candidate + 40 thinking tokens -> 50). Both fail before the change and pass with it; the whole
vertex_aiunit-test folder passes (58) and ruff format/check are clean. Responses are built by hand; no live Vertex AI call was made.Contribution Checklist
This change was made with Claude Code.