Description
Summary
agent_framework_mistral's RawMistralChatClient._parse_usage reads only the three top-level counts from a chat-completion usage payload — prompt_tokens, completion_tokens, total_tokens — and discards prompt_tokens_details. Mistral La Plateforme reports prompt-caching hits there (usage.prompt_tokens_details.cached_tokens, the same field name OpenAI uses), so any application using the Mistral client cannot see how much of a prompt was served from cache: UsageDetails["cache_read_input_token_count"] — the framework's standard key for exactly this figure — is never populated on Mistral responses.
Why it matters
Mistral bills cached prompt tokens at 10% of the standard input rate (see Mistral's prompt caching documentation). Without the mapping, cost accounting and GenAI telemetry built on UsageDetails read cached prompt reads at face value — for a long agentic conversation with a stable prefix that can overstate real input cost by up to 10×. The asymmetry is the surprising part: the framework already maps the equivalent field on every comparable first-party client, so cross-provider usage dashboards silently lose one provider.
agent_framework_openai (chat completions): maps usage.prompt_tokens_details.cached_tokens → cache_read_input_token_count.agent_framework_openai (responses): maps usage.input_tokens_details.cached_tokens → cache_read_input_token_count.agent_framework_gemini: maps cached_content_token_count → cache_read_input_token_count.agent_framework_mistral: drops the field.
Notably, the Mistral client already declares prompt_cache_key as a supported chat option — the request half of the caching feature is wired, so a caller can opt in to caching but cannot observe what it got back.
Reproduction
- Send two identical chat-completion requests through
MistralChatClient with a shared prefix ≥ 64 tokens and the same prompt_cache_key chat option.
- The second response's raw payload carries e.g.
"usage": {"prompt_tokens": 1000, "completion_tokens": 12, "total_tokens": 1012, "prompt_tokens_details": {"cached_tokens": 896}}.
- The parsed
UsageDetails on the response contains only input_token_count=1000, output_token_count=12, total_token_count=1012 — cache_read_input_token_count is absent.
Proposed behavior
In RawMistralChatClient._parse_usage, when the payload carries prompt_tokens_details as a mapping with an integer cached_tokens, set details["cache_read_input_token_count"] to it — mirroring the OpenAI chat-completions client's handling of the identically-named field. The value is documented by Mistral as a subset of prompt_tokens (billed at the discounted rate), matching the semantics cache_read_input_token_count already has for the other providers. Both the non-streaming path and the final streaming chunk's usage go through the same _parse_usage, so one change covers both.
Code Sample
Language/SDK
Python
Description
Summary
agent_framework_mistral'sRawMistralChatClient._parse_usagereads only the three top-level counts from a chat-completionusagepayload —prompt_tokens,completion_tokens,total_tokens— and discardsprompt_tokens_details. Mistral La Plateforme reports prompt-caching hits there (usage.prompt_tokens_details.cached_tokens, the same field name OpenAI uses), so any application using the Mistral client cannot see how much of a prompt was served from cache:UsageDetails["cache_read_input_token_count"]— the framework's standard key for exactly this figure — is never populated on Mistral responses.Why it matters
Mistral bills cached prompt tokens at 10% of the standard input rate (see Mistral's prompt caching documentation). Without the mapping, cost accounting and GenAI telemetry built on
UsageDetailsread cached prompt reads at face value — for a long agentic conversation with a stable prefix that can overstate real input cost by up to 10×. The asymmetry is the surprising part: the framework already maps the equivalent field on every comparable first-party client, so cross-provider usage dashboards silently lose one provider.agent_framework_openai(chat completions): mapsusage.prompt_tokens_details.cached_tokens→cache_read_input_token_count.agent_framework_openai(responses): mapsusage.input_tokens_details.cached_tokens→cache_read_input_token_count.agent_framework_gemini: mapscached_content_token_count→cache_read_input_token_count.agent_framework_mistral: drops the field.Notably, the Mistral client already declares
prompt_cache_keyas a supported chat option — the request half of the caching feature is wired, so a caller can opt in to caching but cannot observe what it got back.Reproduction
MistralChatClientwith a shared prefix ≥ 64 tokens and the sameprompt_cache_keychat option."usage": {"prompt_tokens": 1000, "completion_tokens": 12, "total_tokens": 1012, "prompt_tokens_details": {"cached_tokens": 896}}.UsageDetailson the response contains onlyinput_token_count=1000,output_token_count=12,total_token_count=1012—cache_read_input_token_countis absent.Proposed behavior
In
RawMistralChatClient._parse_usage, when the payload carriesprompt_tokens_detailsas a mapping with an integercached_tokens, setdetails["cache_read_input_token_count"]to it — mirroring the OpenAI chat-completions client's handling of the identically-named field. The value is documented by Mistral as a subset ofprompt_tokens(billed at the discounted rate), matching the semanticscache_read_input_token_countalready has for the other providers. Both the non-streaming path and the final streaming chunk's usage go through the same_parse_usage, so one change covers both.Code Sample
Language/SDK
Python