Python: Preserve Mistral prompt-cache usage details - #7597
Python: Preserve Mistral prompt-cache usage details#7597Ruiming Zhao (uuzzrm) wants to merge 5 commits into
Conversation
Map prompt cache hits from Mistral chat usage into the standard usage details. Add regression coverage for regular and streaming responses.
There was a problem hiding this comment.
Pull request overview
Preserves Mistral prompt-cache usage in standard framework usage details.
Changes:
- Maps cached prompt tokens to provider-specific and standard fields.
- Adds non-streaming and streaming regression coverage.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
python/packages/mistral/agent_framework_mistral/_chat_client.py |
Parses cached prompt-token usage. |
python/packages/mistral/tests/mistral/test_mistral_chat_client.py |
Tests cache usage propagation. |
💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.
|
@microsoft-github-policy-service agree |
|
Following up on the strict integer validation feedback: the current head |
|
Synced the existing branch with the latest upstream main in merge commit 17da48c. The review fix itself is unchanged. The current head retains strict integer validation for cached_tokens (excluding bool) and regression coverage for string, float, and boolean payloads. Local verification after the sync is environment-limited: the bundled Python has pytest but not the repository dependencies or uv, so collection stops before imports with ModuleNotFoundError for agent_framework. The prior focused CI checks for the fix were green; the new push has triggered the repository CI for the synced head. The PR remains open and unmerged, awaiting the repository checks and maintainer review. |
Giles Odigwe (giles17)
left a comment
There was a problem hiding this comment.
Automated Code Review
Reviewers: 5 | Confidence: 93% | Result: All clear
Reviewed: Correctness, Security Reliability, Test Coverage, Failure Modes, Design Approach
Automated review by giles17's agents
|
Hey Ruiming Zhao (@uuzzrm) please address failing checks on the PR |
Narrow prompt token details before reading cached_tokens so the Mistral package passes strict Pyright without changing runtime validation.\n\nAddresses microsoft#7597 (comment)
|
AI says: I addressed the failing checks in commit Verified locally with:
Prysai |
|
AI says: The fix is pushed in Prysai |
…ached-token-usage
|
Updated the branch with the current upstream main (27d82b1) in merge commit 3c705e0. Re-ran the focused checks locally after the sync:
The fork workflow runs for this head are currently action_required, so the repository test workflows have not started and require maintainer approval. The PR remains open and ready for review. AI assistance was used to prepare this update. |
Motivation & Context
Mistral chat-completion responses can include prompt cache hits under
usage.prompt_tokens_details.cached_tokens. The Mistral adapter currently drops that field, so callers cannot observe cache reads through standardUsageDetails. This is especially visible in cost and telemetry reporting for cached prompts.Description & Review Guide
prompt_tokens_details.cached_tokensto bothprompt/cached_tokensandcache_read_input_token_countin_parse_usage.Related Issue
Fixes #7589
Contribution Checklist
breaking changelabel (or add "[BREAKING]" to the title prefix) — a workflow keeps the label and title prefix in sync automatically.