Skip to content

Python: count Vertex AI thinking tokens in completion_tokens usage - #14736

Open
alexcastgonz (pcbeingused333) wants to merge 1 commit into
microsoft:mainfrom
pcbeingused333:fix/vertex-thoughts-tokens
Open

alexcastgonz (pcbeingused333) wants to merge 1 commit into
microsoft:mainfrom
pcbeingused333:fix/vertex-thoughts-tokens

Conversation

@pcbeingused333

Copy link
Copy Markdown
Contributor

Motivation and Context

Follow-up to #14671, which did the same for the Google AI connector. VertexAIChatCompletion and VertexAITextCompletion reported completion_tokens from candidates_token_count alone. Vertex AI reports thinking tokens separately in thoughts_token_count but bills them as output tokens, so for thinking models the usage metadata under-counted the completion tokens.

Description

_get_metadata_from_response in both services now reports candidates_token_count + thoughts_token_count as completion_tokens. The field is 0 when the model does not think, so non-thinking responses are unchanged.

Tests

Added one usage test per service (10 candidate + 40 thinking tokens -> 50). Both fail before the change and pass with it; the whole vertex_ai unit-test folder passes (58) and ruff format/check are clean. Responses are built by hand; no live Vertex AI call was made.

Contribution Checklist

  • The code builds clean without any errors or warnings
  • The PR follows the SK Contribution Guidelines
  • All unit tests pass, and I have added new tests where possible
  • I didn't break anyone 😄

This change was made with Claude Code.

Copilot AI balanced review requested due to automatic review settings November 7, 2026 04:05
@semantic-kernel-automation semantic-kernel-automation Bot added the python Pull requests for the Python Semantic Kernel label Nov 7, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

There are no active review issues for the update to include thinking tokens in Vertex AI completion-token usage.

No open issues 🎉

What changed in this PR

Updates the Python Vertex AI connectors to include thinking tokens in completion-token usage, aligning their accounting with the Google AI connector change in #14671.

Changes:

  • Both chat and text services now report candidate tokens plus thinking tokens as completion_tokens.
  • Adds a regression test for each service verifying that 10 candidate tokens and 40 thinking tokens produce 50 completion tokens.
File Description
python/​tests/​unit/​connectors/​ai/​google/​vertex_ai/​services/​test_vertex_ai_text_completion.py Tests text-completion usage with thinking tokens.
python/​tests/​unit/​connectors/​ai/​google/​vertex_ai/​services/​test_vertex_ai_chat_completion.py Tests chat-completion usage with thinking tokens.
python/​semantic_kernel/​connectors/​ai/​google/​vertex_ai/​services/​vertex_ai_text_completion.py Includes thinking tokens in text-completion usage.
python/​semantic_kernel/​connectors/​ai/​google/​vertex_ai/​services/​vertex_ai_chat_completion.py Includes thinking tokens in chat-completion usage.

Review effort: Balanced

This branch was previously deployed

1 inactive deployment
github-app-auth — 5d51777b Deployed Nov 7, 2026 by pcbeingused333 via add_label #29616
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

python Pull requests for the Python Semantic Kernel

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants