Skip to content

Python: Vertex AI chat completion usage now counts thinking tokens - #14722

Closed
alexcastgonz (pcbeingused333) wants to merge 1 commit into
microsoft:mainfrom
pcbeingused333:fix/vertex-reasoning-tokens
Closed

alexcastgonz (pcbeingused333) wants to merge 1 commit into
microsoft:mainfrom
pcbeingused333:fix/vertex-reasoning-tokens

Conversation

@pcbeingused333

Copy link
Copy Markdown
Contributor

Motivation and Context

Follow-up to #14671, which did the same for the Google AI connector. VertexAIChatCompletion reported completion_tokens from candidates_token_count alone. Vertex AI reports thinking tokens separately in thoughts_token_count but bills them as output tokens, so for thinking models the usage metadata under-counted the completion tokens.

Description

_get_metadata_from_response now reports candidates_token_count + thoughts_token_count as completion_tokens. The field defaults to 0 when the model does not think, so non-thinking responses are unchanged.

Tests

Added test_vertex_ai_chat_completion_usage_includes_thinking_tokens (10 candidate + 40 thinking tokens -> 50). It fails before the change (10 != 50) and passes with it; the whole vertex_ai unit-test folder passes (57) and ruff format/check are clean. The response is built by hand; no live Vertex AI call was made.

Contribution Checklist

  • The code builds clean without any errors or warnings
  • The PR follows the SK Contribution Guidelines
  • All unit tests pass, and I have added new tests where possible
  • I didn't break anyone 😄

This change was made with Claude Code.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues remain for the change to include thinking tokens in Vertex AI completion-token usage.

No open issues 🎉

What changed in this PR

Updates Vertex AI chat completion usage to include thinking tokens as output tokens, aligning it with the Google AI connector change in #14671.

Changes:

  • Calculates completion tokens as candidate tokens plus thinking tokens.
  • Adds a regression test verifying that 10 candidate tokens and 40 thinking tokens produce 50 completion tokens.
File Description
python/​tests/​unit/​connectors/​ai/​google/​vertex_ai/​services/​test_vertex_ai_chat_completion.py Adds coverage for usage that includes thinking tokens.
python/​semantic_kernel/​connectors/​ai/​google/​vertex_ai/​services/​vertex_ai_chat_completion.py Includes thinking tokens in completion-token metadata.

Review effort: Balanced

@semantic-kernel-automation semantic-kernel-automation Bot added the python Pull requests for the Python Semantic Kernel label Nov 4, 2026
@pcbeingused333

Copy link
Copy Markdown
Contributor Author

Closing in favor of #14716, which carries the same Vertex AI thinking-token usage fix for chat completion and also covers the text completion path. Sorry for the duplicate.

This branch was previously deployed

1 inactive deployment
github-app-auth — 0d81678b Deployed Nov 4, 2026 by pcbeingused333 via add_label #29594
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

python Pull requests for the Python Semantic Kernel

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants