Skip to content

docs(cache): audit prompt-cache strategy batch 08 - #6816

Open
rekram1-node wants to merge 2 commits into
devfrom
cache-audit-08
Open

docs(cache): audit prompt-cache strategy batch 08#6816
rekram1-node wants to merge 2 commits into
devfrom
cache-audit-08

Conversation

@rekram1-node

Copy link
Copy Markdown
Collaborator

Batch 08 (11 providers): impossibl, inception, inceptron, inference, inferx, infomaniak, io-net, iteracompute, jalapeno, jiekou, kenari.

Leading header comment only (# Prompt cache (chat) + # Docs) per provider.

  • SUPPORTS (2): inception (automatic prefix cache, discounted cached-input rates, usage.prompt_tokens_details.cached_tokens), jalapeno (automatic prefix cache + optional prompt_cache_key, dedicated caching doc).
  • NATIVE_OTHER (3): impossibl (automatic on most routes; Anthropic needs cache_control breakpoint — prior verdict verified), jiekou (explicit Anthropic cache_control + implicit OpenAI/Gemini — prior verdict verified), io-net (automatic prefix cache, no request fields; per-model supports_prompt_cache + cache read/write rates).
  • TOLERATES (1): infomaniak (OpenAI-compatible chat API, no cache params/usage/rates published).
  • UNKNOWN (5): inceptron (setup-only docs), inference (closed param table, no cache fields), inferx (no public API reference), iteracompute (no cache params; usage is prompt/completion/total only), kenari (site/docs unreachable).

Note: bun validate fails identically on the clean base (pre-existing AuthoredModelShape.deepPartial environment error); all 11 TOML files verified parseable.

@github-actions

Copy link
Copy Markdown
Contributor

No actionable findings.

@github-actions github-actions Bot added the reviewer: ready Automated review found no actionable items label Sep 11, 2026
@github-actions github-actions Bot removed the reviewer: ready Automated review found no actionable items label Sep 11, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Action items

  • [high] [possible mistake] providers/jiekou/provider.toml:1 - Check: Cache-key strategy must match documented request fields. Why: The header says send prompt_cache_key, but the same line only cites cache_control as a tolerated OpenAI-chat extension and implicit OpenAI/Gemini prefix caching. That does not establish prompt_cache_key as a client control; patch 1 also treated this as Anthropic cache_control + implicit cache, not a cache-key API. Action: Reclassify to the correct key policy (likely strip prompt_cache_key unless docs explicitly define prompt_cache_key), and keep cache_control guidance under cache policy only.
  • [medium] [possible mistake] providers/impossibl/provider.toml:1 - Check: send prompt_cache_key requires a documented/supported cache-key field, not mere unknown-field tolerance. Why: The rationale is that the gateway tolerates extra cache fields / ignores cache_control on non-Anthropic routes (no 400). Patch 1 stated there is no prompt_cache_key field. Tolerance of extras is not evidence that clients should send prompt_cache_key. Action: Verify Impossibl docs for an actual prompt_cache_key control; if absent, use strip (or another evidenced policy) and describe automatic/cache_control behavior only in cache policy.
  • [medium] [possible mistake] providers/infomaniak/provider.toml:3 - Check: Cache policy claims must not contradict the provider’s own documented surface. Why: Patch 1 reported no published cache params, cached-token usage, or cached-input rates (TOLERATES). Patch 2 still strips prompt_cache_key for a closed schema, but asserts “automatic server-side prefix caching” and verification via usage.prompt_tokens_details.cached_tokens. Catalog models also lack cache_read/cache_write. Action: Confirm Infomaniak documents automatic prefix cache and cached_tokens; if not, rewrite cache policy to “no documented prompt cache / upstream-implicit only” (or cite the specific doc that supports the stronger claim).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant