feat: Refactor OpenRouter provider to use Vercel AI SDK #10778
Conversation
dosubot
bot
added
size:XXL
Review of commit 3f935af. No new issues found. 1 previously flagged issue remains unresolved.
Mention @roomote in a comment to request specific changes to this pull request or fix all unresolved issues. |
![roomote[bot]](https://avatars.githubusercontent.com/in/1546624?s=60&v=4){kind=small}
| private client: OpenAI | ||
| protected models: ModelRecord = {} | ||
| protected endpoints: ModelRecord = {} | ||
| private readonly providerName = "OpenRouter" |
There was a problem hiding this comment.
The old implementation had a getReasoningDetails() method and currentReasoningDetails accumulator that Task.ts relies on (see line 959 in Task.ts). This method accumulates reasoning_details from the stream (handling reasoning.text, reasoning.summary, and reasoning.encrypted types) and provides them for persistence in API conversation history. This is required for models like Gemini 3 via OpenRouter that use the reasoning.encrypted format - without it, reasoning context won't be preserved across multi-turn conversations, potentially breaking those model integrations.
Fix it with Roo Code or mention @roomote and request a fix.
daniel-lxs
force-pushed
the
feat/openrouter-ai-sdk
branch
from
5260473 to
d062770
Compare
There was a problem hiding this comment.
Additional Issues Found in Re-review
The previously flagged issue with missing getReasoningDetails() method remains unresolved. Additionally, I've identified a few more regressions in this refactor:
1. Telemetry Loss
Error handling no longer reports exceptions to telemetry.
2. Usage Data Loss
Detailed usage information (cost, cache tokens, reasoning tokens) is no longer yielded.
3. Model-Specific Handling Removed
Special handling for DeepSeek R1, Gemini, and Anthropic models has been removed.
See inline comments for details.
| tools, | ||
| toolChoice: metadata?.tool_choice as any, | ||
| providerOptions, | ||
| }) |
There was a problem hiding this comment.
Usage Data Regression: The previous implementation yielded detailed usage information including:
totalCostfromcost_details.upstream_inference_cost + costcacheReadTokensfromprompt_tokens_details.cached_tokensreasoningTokensfromcompletion_tokens_details.reasoning_tokens
The new implementation only yields basic inputTokens and outputTokens. This loses cost tracking and token detail information that may be important for usage analytics and cost monitoring.
The AI SDK's result.usage may have different properties - please verify what data is available and restore as much usage detail as possible.
| @@ -212,323 +83,59 @@ export class OpenRouterHandler extends BaseProvider implements SingleCompletionH | |||
| metadata?: ApiHandlerCreateMessageMetadata, | |||
There was a problem hiding this comment.
Model-Specific Handling Removed: The previous implementation had important model-specific handling that has been removed:
- DeepSeek R1: Converted messages using
convertToR1Format()which uses user role instead of system role - Gemini Models: Had
sanitizeGeminiMessages()and fake encrypted block injection for preserving reasoning context when switching models - Anthropic Models: Added
x-anthropic-beta: fine-grained-tool-streaming-2025-05-14header - Prompt Caching: Called
addGeminiCacheBreakpoints()oraddAnthropicCacheBreakpoints()for supported models - Reasoning Parameter: The
reasoningparameter (including exclude logic for Gemini 2.5 Pro) is no longer passed to the API
These may cause regressions for specific model providers. Please verify if the AI SDK handles these cases automatically, or if they need to be re-implemented.
- Add getReasoningDetails() method to preserve reasoning context for multi-turn conversations - Restore telemetry reporting with TelemetryService.captureException() in error handlers - Restore detailed usage metrics (totalCost, cacheReadTokens, reasoningTokens, cacheWriteTokens) Addresses review comments on PR #10778
- Add getReasoningDetails() method to preserve reasoning context for multi-turn conversations - Restore telemetry reporting with TelemetryService.captureException() in error handlers - Restore detailed usage metrics (totalCost, cacheReadTokens, reasoningTokens, cacheWriteTokens) Addresses review comments on PR #10778
daniel-lxs
force-pushed
the
feat/openrouter-ai-sdk
branch
from
d8f0c64 to
11fd793
Compare
Review of commit 11fd793. No new issues found. 2 of 5 original issues remain unresolved.
Mention @roomote in a comment to request specific changes to this pull request or fix all unresolved issues. |
- Add getReasoningDetails() method to preserve reasoning context for multi-turn conversations - Restore telemetry reporting with TelemetryService.captureException() in error handlers - Restore detailed usage metrics (totalCost, cacheReadTokens, reasoningTokens, cacheWriteTokens) Addresses review comments on PR #10778
daniel-lxs
force-pushed
the
feat/openrouter-ai-sdk
branch
from
11fd793 to
ae9636e
Compare
- Replace direct OpenAI SDK usage with @openrouter/ai-sdk-provider - Add 'ai' package for streamText() and generateText() functions - Extract reusable AI SDK conversion utilities to src/api/transform/ai-sdk.ts: - convertToAiSdkMessages(): Anthropic messages to CoreMessage format - convertToolsForAiSdk(): OpenAI tools to AI SDK tool format - processAiSdkStreamPart(): AI SDK stream events to ApiStreamChunk - Update tests to mock AI SDK functions instead of OpenAI client - Add comprehensive tests for the new ai-sdk.ts utility module This follows Vercel's AI SDK provider pattern for standardized LLM integration and enables easier migration of other providers in the future.
- Fixed tool result output format to use typed object { type: 'text', value: string }
instead of plain string to satisfy AI SDK validation schema
- Added tool name resolution by building a map of tool call IDs to names
- Updated tests to reflect new output format
- Add getReasoningDetails() method to preserve reasoning context for multi-turn conversations - Restore telemetry reporting with TelemetryService.captureException() in error handlers - Restore detailed usage metrics (totalCost, cacheReadTokens, reasoningTokens, cacheWriteTokens) Addresses review comments on PR #10778
- Add support for 'reasoning' and 'text' event types in AI SDK stream processing - Pass reasoning parameters via createOpenRouter extraBody instead of providerOptions - Support both effort-based (effort: 'high') and budget-based (max_tokens: N) reasoning - Add comprehensive tests for reasoning parameter passing and stream event handling - Fixes reasoning tokens not being displayed for models like DeepSeek R1 and Gemini Thinking Changes: - src/api/transform/ai-sdk.ts: Add 'text' and 'reasoning' event type handlers - src/api/providers/openrouter.ts: Pass reasoning via extraBody in provider creation - Add tests for new event types and reasoning parameter flow - All 53 tests passing
- Re-add DeepSeek R1 format via convertToR1Format with extraBody override - Re-add Gemini sanitization (sanitizeGeminiMessages) and encrypted block injection - Re-add Gemini 2.5 Pro reasoning exclusion when not explicitly configured - Re-add prompt caching (addAnthropicCacheBreakpoints, addGeminiCacheBreakpoints) - Re-add Anthropic beta headers (x-anthropic-beta: fine-grained-tool-streaming) - Re-add reasoning_details handling via consolidateReasoningDetails - Fix topP parameter passthrough to streamText() and generateText() - Fix duplicate toolCallIdToName in ai-sdk.ts from rebase - Update tests for new model-specific behavior and add 9 new tests
daniel-lxs
force-pushed
the
feat/openrouter-ai-sdk
branch
from
ae9636e to
f3181ce
Compare
| } catch (error: any) { | ||
| const errorMessage = error instanceof Error ? error.message : String(error) | ||
| const apiError = new ApiProviderError(errorMessage, this.providerName, modelId, "createMessage") | ||
| TelemetryService.instance.captureException(apiError) | ||
| yield { | ||
| type: "usage", | ||
| inputTokens: lastUsage.prompt_tokens || 0, | ||
| outputTokens: lastUsage.completion_tokens || 0, | ||
| cacheReadTokens: lastUsage.prompt_tokens_details?.cached_tokens, | ||
| reasoningTokens: lastUsage.completion_tokens_details?.reasoning_tokens, | ||
| totalCost: (lastUsage.cost_details?.upstream_inference_cost || 0) + (lastUsage.cost || 0), | ||
| type: "error", | ||
| error: "OpenRouterError", | ||
| message: `${this.providerName} API Error: ${errorMessage}`, | ||
| } | ||
| } |
There was a problem hiding this comment.
This catch yields an error chunk instead of throwing. Task.ts wraps the first chunk from createMessage in a try/catch (line ~4397) that drives retry logic: exponential backoff for auto-approval, context-window-exceeded detection with auto-truncation, and the user-facing retry dialog. By catching the error here and yielding it as a chunk, the first iterator.next() in Task.ts resolves successfully with an error-typed value instead of rejecting, so all of that retry machinery is bypassed. The migrated Gemini handler throws in the equivalent path (line ~203 of gemini.ts), which correctly triggers the retry flow. Consider re-throwing here (after telemetry capture) to preserve the existing error contract.
Fix it with Roo Code or mention @roomote and request a fix.
… provider - Remove buildOpenAiMessages(), extraBody.messages hack, and dummy messages - Wire reasoning_details through AI SDK natively via providerOptions.openrouter - Filter schema-invalid reasoning_details entries (malformed encrypted blocks) - Filter [REDACTED] from thinking UI stream (upstream provider behavior) - Remove unused imports: convertToOpenAiMessages, sanitizeGeminiMessages, consolidateReasoningDetails, convertToR1Format, addAnthropicCacheBreakpoints, addGeminiCacheBreakpoints - All models now use convertToAiSdkMessages() → streamText() natively - Prompt caching deferred (needs providerOptions.openrouter.cacheControl impl)
3 participants
Summary
Migrate the OpenRouter provider from the OpenAI SDK to the Vercel AI SDK for a more standardized and maintainable implementation.
Closes COM-502 (part 2/2)
Problem
The current OpenRouter provider implementation uses the raw OpenAI SDK directly, which requires complex message transformation and reasoning token handling code that is difficult to maintain.
Solution
Refactor the OpenRouter provider to use the
@openrouter/ai-sdk-providerpackage which integrates with Vercel's AI SDK. This provides:streamTextandgenerateTextCoreMessageformatChanges
src/api/providers/openrouter.tsto use AI SDK patternssrc/api/providers/__tests__/openrouter.spec.tswith new test patternsTesting
tsc --noEmitpasses in bothpackages/typesandsrcRelated
Important
Refactor OpenRouter provider to use Vercel AI SDK, simplifying streaming, message conversion, and tool handling, with extensive testing and error handling improvements.
OpenRouterHandlerinopenrouter.tsto use Vercel AI SDK, replacing OpenAI SDK.streamTextandgenerateText.CoreMessageformat.openrouter.spec.tstests to reflect new patterns.TelemetryService.openrouter.spec.tsfor new streaming and message handling.model-params.tsfor reasoning and temperature settings.This description was created by
for 3f935af. You can customize this summary. It will automatically update as commits are pushed.