From 28fd1a1f8e0f41244e6ed3e8186adc22851a7205 Mon Sep 17 00:00:00 2001 From: Devansh Dubey Date: Fri, 21 Aug 2026 21:31:09 +0530 Subject: [PATCH 1/3] changelog: 2.19.0 --- changelog/enterprise.mdx | 79 +++++++++++++++++++++++++++++++++++++++- 1 file changed, 78 insertions(+), 1 deletion(-) diff --git a/changelog/enterprise.mdx b/changelog/enterprise.mdx index bde9587d..e14f3c30 100644 --- a/changelog/enterprise.mdx +++ b/changelog/enterprise.mdx @@ -1,6 +1,6 @@ --- title: "Enterprise Gateway" -sidebarTitle: "Enterprise Gateway [2.18.0]" +sidebarTitle: "Enterprise Gateway [2.19.0]" rss: true --- @@ -8,6 +8,83 @@ rss: true Discuss how Portkey's AI Gateway can enhance your organization's AI infrastructure + + +## v2.19.0 + +--- + +### Deepgram Provider + +Deepgram is now a native provider for speech-to-text and text-to-speech, reachable through the unified `/v1/audio/transcriptions` and `/v1/audio/speech` endpoints. The custom-host setup is no longer required, and OpenAI-style parameters such as `response_format` and `voice` map to Deepgram's encodings and models. + +[Deepgram Documentation](/integrations/llms/deepgram) + +### Headroom Context Compression + +New Headroom plugin compresses request context before it reaches the provider to cut input token costs, with separate toggles for system, user, and tool messages. + +[Guardrails Documentation](/product/guardrails) + +### OCR on Vertex AI + +The `POST /v1/ocr` endpoint now supports Vertex AI, routing Mistral OCR models through Vertex's `rawPredict` endpoint. + +[OCR API Reference](/api-reference/inference-api/ocr) · [Vertex AI Documentation](/integrations/llms/vertex-ai#ocr-document-processing) + +### OAuth Client Credentials for Upstream Auth + +Providers can authenticate to upstream endpoints using the OAuth 2.0 client credentials grant, with the gateway fetching and caching tokens. Available for OpenAI-compatible upstreams. + +[Model Catalog Documentation](/product/model-catalog) + +### MCP Registry Proxy + +The gateway now proxies MCP registry requests, letting clients discover MCP servers through the gateway. + +[MCP Registry Documentation](/product/mcp-gateway/mcp-registry) + +### Billing from Provider-Reported Cost + +Cost is now billed from the provider-reported cost when the upstream returns one, falling back to configured pricing otherwise. This improves accuracy for providers such as OpenRouter that return actual spend per request. + +[OpenRouter Documentation](/integrations/llms/openrouter) + +### Span-Level Feedback + +Feedback can now be attached to an individual span rather than only the top-level trace, giving per-step quality signals on multi-step requests. + +[Feedback Documentation](/product/observability/feedback) + +### Provider Updates + +- **Dashscope (Qwen)**: Added Responses and Messages endpoint support, plus additional non-OpenAI parameters +- **Bedrock (Mantle)**: Added Responses API support and fixed Anthropic SSE streaming boundaries +- **Bedrock**: `temperature` is stripped from `inferenceConfig` for Claude 4-series models, where it is no longer accepted +- **Bedrock**: AWS STS session tags can be propagated to the upstream provider, gated behind an environment flag +- **Vertex AI**: `output_config` is now forwarded for Anthropic Messages requests +- **Vertex AI**: Fixed request metadata handling on Cloudflare Workers +- **Azure OpenAI**: Resources auto-resolve for non-default deployments, with further OpenCode compatibility fixes +- **Meshy & Tripo3D**: Added provider configuration and cost tracking for their async generation APIs +- **Anthropic**: Updated the `anthropic-beta` header blocklist +- **OpenAI**: `chat_complete_kwargs` is now forwarded to the provider + +[Providers Documentation](/integrations/llms) + +### Fixes and Improvements + +- **Messages API**: Unique `tool_use` IDs are generated instead of reusing provider tool call IDs +- **Messages & Responses**: Fixed `instructions` handling during transformation +- **Logs**: `output_tokens_details` is preserved in reassembled streaming logs +- **Logs**: Corrected thinking-token attribution for proxy requests +- **Pricing**: Bedrock inference-profile prefixes are preserved for cost lookup, and integration `pricing_adjustments` now apply on the config path +- **MCP**: Issuer URL now follows RFC 8414, extra auth parameters are supported, and `GET`/`DELETE` return 405 since the gateway is stateless +- **Security**: Guardrail check IDs resolved from the prototype chain are now blocked +- **Security**: `x-amz-security-token` is masked in logs +- Updated dependencies to patch security vulnerabilities. + + + ## v2.18.0 From d1db8fd82929bb7910d5ba93fb27173e43052270 Mon Sep 17 00:00:00 2001 From: Devansh Dubey Date: Fri, 21 Aug 2026 21:55:57 +0530 Subject: [PATCH 2/3] update --- changelog/enterprise.mdx | 25 ++++++++----------------- 1 file changed, 8 insertions(+), 17 deletions(-) diff --git a/changelog/enterprise.mdx b/changelog/enterprise.mdx index e14f3c30..c04ef5ed 100644 --- a/changelog/enterprise.mdx +++ b/changelog/enterprise.mdx @@ -38,9 +38,9 @@ Providers can authenticate to upstream endpoints using the OAuth 2.0 client cred [Model Catalog Documentation](/product/model-catalog) -### MCP Registry Proxy +### MCP Registry -The gateway now proxies MCP registry requests, letting clients discover MCP servers through the gateway. +The gateway now serves MCP registry endpoints conforming to the MCP Registry API standard, so any spec-compliant client can discover available MCP servers via `GET /v0.1/servers`. [MCP Registry Documentation](/product/mcp-gateway/mcp-registry) @@ -50,24 +50,16 @@ Cost is now billed from the provider-reported cost when the upstream returns one [OpenRouter Documentation](/integrations/llms/openrouter) -### Span-Level Feedback - -Feedback can now be attached to an individual span rather than only the top-level trace, giving per-step quality signals on multi-step requests. - -[Feedback Documentation](/product/observability/feedback) - ### Provider Updates -- **Dashscope (Qwen)**: Added Responses and Messages endpoint support, plus additional non-OpenAI parameters -- **Bedrock (Mantle)**: Added Responses API support and fixed Anthropic SSE streaming boundaries +- **Dashscope (Qwen)**: Added Responses and Messages endpoint support, plus additional non-OpenAI parameters including `chat_complete_kwargs` +- **Bedrock (Mantle)**: Fixed Anthropic SSE streaming boundaries - **Bedrock**: `temperature` is stripped from `inferenceConfig` for Claude 4-series models, where it is no longer accepted -- **Bedrock**: AWS STS session tags can be propagated to the upstream provider, gated behind an environment flag - **Vertex AI**: `output_config` is now forwarded for Anthropic Messages requests -- **Vertex AI**: Fixed request metadata handling on Cloudflare Workers -- **Azure OpenAI**: Resources auto-resolve for non-default deployments, with further OpenCode compatibility fixes +- **Azure AI Foundry**: Further OpenCode compatibility fixes on the Responses path +- **Azure OpenAI**: Resources auto-resolve for non-default deployments - **Meshy & Tripo3D**: Added provider configuration and cost tracking for their async generation APIs - **Anthropic**: Updated the `anthropic-beta` header blocklist -- **OpenAI**: `chat_complete_kwargs` is now forwarded to the provider [Providers Documentation](/integrations/llms) @@ -77,10 +69,9 @@ Feedback can now be attached to an individual span rather than only the top-leve - **Messages & Responses**: Fixed `instructions` handling during transformation - **Logs**: `output_tokens_details` is preserved in reassembled streaming logs - **Logs**: Corrected thinking-token attribution for proxy requests -- **Pricing**: Bedrock inference-profile prefixes are preserved for cost lookup, and integration `pricing_adjustments` now apply on the config path +- **Pricing**: Bedrock inference-profile models now resolve to the correct rate, and integration pricing adjustments apply to requests routed via configs — [Pricing Adjustments Documentation](/product/model-catalog/pricing-adjustments) - **MCP**: Issuer URL now follows RFC 8414, extra auth parameters are supported, and `GET`/`DELETE` return 405 since the gateway is stateless -- **Security**: Guardrail check IDs resolved from the prototype chain are now blocked -- **Security**: `x-amz-security-token` is masked in logs +- **Security**: Hardened guardrail check resolution and credential masking in logs - Updated dependencies to patch security vulnerabilities. From 545c0bcc4c00472c0e017f167dc01f526a9419cb Mon Sep 17 00:00:00 2001 From: Devansh Dubey Date: Fri, 21 Aug 2026 22:29:09 +0530 Subject: [PATCH 3/3] updates --- changelog/enterprise.mdx | 61 +++++++++++++++++++++------------------- 1 file changed, 32 insertions(+), 29 deletions(-) diff --git a/changelog/enterprise.mdx b/changelog/enterprise.mdx index c04ef5ed..0b32e468 100644 --- a/changelog/enterprise.mdx +++ b/changelog/enterprise.mdx @@ -16,62 +16,65 @@ Discuss how Portkey's AI Gateway can enhance your organization's AI infrastructu ### Deepgram Provider -Deepgram is now a native provider for speech-to-text and text-to-speech, reachable through the unified `/v1/audio/transcriptions` and `/v1/audio/speech` endpoints. The custom-host setup is no longer required, and OpenAI-style parameters such as `response_format` and `voice` map to Deepgram's encodings and models. +New Deepgram provider integration supporting speech-to-text (transcription) and text-to-speech endpoints through the gateway's unified API, with full support for retries, fallbacks, load balancing, caching, logging, and pricing. [Deepgram Documentation](/integrations/llms/deepgram) -### Headroom Context Compression +### Headroom Guardrail -New Headroom plugin compresses request context before it reaches the provider to cut input token costs, with separate toggles for system, user, and tool messages. +New Headroom partner guardrail plugin for request and response scanning, configurable through the standard guardrails workflow. [Guardrails Documentation](/product/guardrails) -### OCR on Vertex AI +### Qwen (DashScope) Responses & Messages API -The `POST /v1/ocr` endpoint now supports Vertex AI, routing Mistral OCR models through Vertex's `rawPredict` endpoint. +Qwen models via DashScope now support the Responses API and Messages API endpoints, along with pass-through of non-OpenAI parameters (`extra_body`) for Qwen-specific features. -[OCR API Reference](/api-reference/inference-api/ocr) · [Vertex AI Documentation](/integrations/llms/vertex-ai#ocr-document-processing) +[DashScope Documentation](/integrations/llms/dashscope) -### OAuth Client Credentials for Upstream Auth +### Vertex AI OCR Support -Providers can authenticate to upstream endpoints using the OAuth 2.0 client credentials grant, with the gateway fetching and caching tokens. Available for OpenAI-compatible upstreams. +The `POST /v1/ocr` endpoint now supports Vertex AI models, bringing OCR under the gateway's full middleware stack for Google Cloud-hosted models. -[Model Catalog Documentation](/product/model-catalog) +[OCR API Reference](/api-reference/inference-api/ocr) · [Vertex AI Documentation](/integrations/llms/vertex-ai) -### MCP Registry +### MCP Server Registry -The gateway now serves MCP registry endpoints conforming to the MCP Registry API standard, so any spec-compliant client can discover available MCP servers via `GET /v0.1/servers`. +New `/v0.1/servers` endpoint exposes a registry of available MCP servers, enabling client-side discovery of configured servers and their capabilities. [MCP Registry Documentation](/product/mcp-gateway/mcp-registry) -### Billing from Provider-Reported Cost +### Span-Level Feedback -Cost is now billed from the provider-reported cost when the upstream returns one, falling back to configured pricing otherwise. This improves accuracy for providers such as OpenRouter that return actual spend per request. +Feedback can now target individual spans within a trace, enabling more granular quality signals at the sub-request level. -[OpenRouter Documentation](/integrations/llms/openrouter) +[Feedback Documentation](/product/observability/feedback) ### Provider Updates -- **Dashscope (Qwen)**: Added Responses and Messages endpoint support, plus additional non-OpenAI parameters including `chat_complete_kwargs` -- **Bedrock (Mantle)**: Fixed Anthropic SSE streaming boundaries -- **Bedrock**: `temperature` is stripped from `inferenceConfig` for Claude 4-series models, where it is no longer accepted -- **Vertex AI**: `output_config` is now forwarded for Anthropic Messages requests -- **Azure AI Foundry**: Further OpenCode compatibility fixes on the Responses path -- **Azure OpenAI**: Resources auto-resolve for non-default deployments -- **Meshy & Tripo3D**: Added provider configuration and cost tracking for their async generation APIs -- **Anthropic**: Updated the `anthropic-beta` header blocklist +- **OpenRouter**: Pricing now uses provider-reported cost when available, falling back to configured pricing tables +- **Qwen (DashScope)**: Responses API and Messages API support with non-OpenAI parameter pass-through +- **Bedrock (Mantle)**: Fixed Anthropic SSE streaming boundary handling for reliable chunk delivery +- **Bedrock**: `temperature` is stripped from `inferenceConfig` for Claude 4-series models that deprecate it +- **Azure AI Foundry**: Additional OpenCode compatibility fixes for Responses API routing +- **Vertex AI**: Fixed `output_config` forwarding for Anthropic Messages requests +- **Vertex AI**: Thinking token attribution corrected for proxy requests +- **Anthropic**: Updated beta parameter blocklist [Providers Documentation](/integrations/llms) ### Fixes and Improvements -- **Messages API**: Unique `tool_use` IDs are generated instead of reusing provider tool call IDs -- **Messages & Responses**: Fixed `instructions` handling during transformation -- **Logs**: `output_tokens_details` is preserved in reassembled streaming logs -- **Logs**: Corrected thinking-token attribution for proxy requests -- **Pricing**: Bedrock inference-profile models now resolve to the correct rate, and integration pricing adjustments apply to requests routed via configs — [Pricing Adjustments Documentation](/product/model-catalog/pricing-adjustments) -- **MCP**: Issuer URL now follows RFC 8414, extra auth parameters are supported, and `GET`/`DELETE` return 405 since the gateway is stateless -- **Security**: Hardened guardrail check resolution and credential masking in logs +- **MCP Gateway**: OAuth issuer URL now follows RFC 8414 format +- **MCP Gateway**: Extra authorization parameters can be passed to upstream OAuth providers +- **Streaming Logs**: `output_token_details` now included in reassembled streaming log entries +- **Pricing**: Integration-level `pricing_adjustments` applied correctly on the config path +- **Pricing**: Bedrock inference-profile prefix preserved for accurate cost attribution +- **Pricing**: Meshy and Tripo3D async API pricing tracked +- **Security**: `x-amz-security-token` masked in log paths +- **Messages API**: Unique `tool_use` IDs generated instead of reusing provider-issued IDs, fixing deduplication issues +- **Messages/Responses**: Instructions parameter handling corrected +- **Forwarding**: `chat_complete_kwargs` forwarded to providers that support it - Updated dependencies to patch security vulnerabilities.