Skip to content

docs(providers): prompt cache batch 03 - #6814

Open
rekram1-node wants to merge 2 commits into
devfrom
cache-audit-03
Open

docs(providers): prompt cache batch 03#6814
rekram1-node wants to merge 2 commits into
devfrom
cache-audit-03

Conversation

@rekram1-node

Copy link
Copy Markdown
Collaborator

Documents prompt-cache (chat) strategy for batch 03 providers. Header-only changes (leading comment block above first key); no schema fields touched. bun validate passes.

Provider Verdict Mechanism Docs
arcee TOLERATES Request schema omits prompt_cache_key with no additionalProperties:false, so unknown key is accepted but ignored. https://docs.arcee.ai/api-reference/chat-completion
atomic-chat UNKNOWN Local OpenAI-compatible server documents no prompt_cache_key or caching behavior; backend-dependent. https://github.com/AtomicBot-ai/Atomic-Chat
auriko SUPPORTS Explicit prompt_cache_key field with OpenAI-compatible passthrough. https://docs.auriko.ai/api-reference/chat-completions
azure SUPPORTS Automatic prefix caching plus prompt_cache_key routing hint (retention/breakpoints on newer models). https://learn.microsoft.com/en-us/azure/foundry/openai/how-to/prompt-caching
azure-cognitive-services SUPPORTS Same Azure OpenAI prompt-caching surface: automatic prefix caching plus prompt_cache_key. https://learn.microsoft.com/en-us/azure/foundry/openai/how-to/prompt-caching
bailing UNKNOWN No prompt_cache_key or prompt-caching behavior documented. https://alipaytbox.yuque.com/sxs0ba/ling/intro
baseten REJECTS Strict ChatCompletionRequest (additionalProperties:false) with no prompt_cache_key; unknown key fails validation. https://docs.baseten.co/reference/inference-api/chat-completions
berget TOLERATES Schema omits prompt_cache_key but permits extras (additionalProperties:{}), so unknown key is accepted without effect. https://api.berget.ai/openapi.json
blueclaw UNKNOWN OpenAI-compatible gateway docs list chat/streaming/tools with no cache param or schema. https://blueclaw.network/how-it-works.html
bothub UNKNOWN No prompt_cache_key or prompt-caching behavior documented for the OpenAI-compatible endpoint. https://bothub.ru/models
cerebras SUPPORTS Automatic prefix caching plus optional prompt_cache_key routing hint (max 1024 chars, longer values rejected with 400). https://inference-docs.cerebras.ai/capabilities/prompt-caching

Verification notes:

  • Cerebras prompt_cache_key limit (1024 chars, 400 if longer) confirmed in docs.
  • Baseten REJECTS confirmed via OpenAPI (ChatCompletionRequest additionalProperties:false, no prompt_cache_key).
  • Berget TOLERATES confirmed via openapi.json (no prompt_cache_key anywhere; ChatCompletionRequest additionalProperties:{}).
  • Auriko SUPPORTS confirmed via OpenAPI (explicit prompt_cache_key: "Prompt caching identifier. Supported by OpenAI.").
  • Azure SUPPORTS confirmed via MS prompt-caching guide (automatic caching + prompt_cache_key on GPT-5.6+).

@github-actions

Copy link
Copy Markdown
Contributor

No actionable findings.

@github-actions github-actions Bot added the reviewer: ready Automated review found no actionable items label Sep 11, 2026
@github-actions github-actions Bot removed the reviewer: ready Automated review found no actionable items label Sep 11, 2026
@github-actions

Copy link
Copy Markdown
Contributor

No actionable findings.

@github-actions github-actions Bot added the reviewer: ready Automated review found no actionable items label Sep 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

reviewer: ready Automated review found no actionable items

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant