feat: support OpenRouter / non-Claude models via env-var passthrough (#9)#52
Open
ChrisJr404 wants to merge 1 commit intoknostic:masterfrom
Open
feat: support OpenRouter / non-Claude models via env-var passthrough (#9)#52ChrisJr404 wants to merge 1 commit intoknostic:masterfrom
ChrisJr404 wants to merge 1 commit intoknostic:masterfrom
Conversation
…nostic#9) Closes knostic#9. Routes every anthropic.Anthropic(...) construction through a single helper (utilities.llm_client.get_anthropic_client) that picks up two env vars when set: OPENANT_LLM_BASE_URL=https://openrouter.ai/api/v1 OPENANT_LLM_API_KEY=sk-or-v1-... When unset the SDK uses its existing defaults, so Claude users see no behaviour change. --model now accepts any string. The new resolver: - keeps "opus" / "sonnet" aliases (canonical Claude IDs) - passes a slash-form ID through verbatim (qwen/qwen-3-coder-480b) - strips a leading "openrouter/" prefix per the OpenCode convention Didier flagged on the issue, so openrouter/moonshotai/kimi-k2 just works out of the box. Cost rollups previously fell back to Sonnet rates for unknown model IDs. Now unknown IDs default to {input: 0, output: 0} with a one-time stderr warning, so totals stay honest. A new MODEL_PRICING_OVERRIDE env var (JSON of {model_id: {input, output}} per million tokens) merges over the built-in table for power users who want real OpenRouter pricing. Tests cover: env-var passthrough on/off, kwarg precedence, slash-form resolution, openrouter/ prefix stripping, $0 fallback + warning, and override merge. All 112 tests pass.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #9.
Implements the env-var-passthrough proposal from #9 (comment) plus the
openrouter/prefix convention Didier raised in #9 (comment).What changed
Three env vars + a slightly more permissive
--modelargument are all that's needed to drive Qwen / Kimi / MiniMax / DeepSeek (or any Anthropic-compatible endpoint) without leaving the existing SDK:When
OPENANT_LLM_BASE_URLis unset, every Anthropic client construction falls back to the SDK defaults. Existing Claude setups behave exactly as before.--modelnow acceptsopus,sonnetclaude-opus-4-6qwen/qwen-3-coder-480bopenrouter/moonshotai/kimi-k2openrouter/is stripped per Didier's suggestion (becomesmoonshotai/kimi-k2).Cost tracking
The hardcoded pricing table only knows Claude. Two follow-on bits:
{input: 0, output: 0}with a one-time stderr warning, so cost rollups stay honest instead of silently estimating with Sonnet rates.MODEL_PRICING_OVERRIDEenv var (JSON of{model_id: {input, output}}per million tokens) merges over the built-in table so power users can plug in real OpenRouter pricing without code changes:Implementation notes
utilities.llm_client.get_anthropic_client(**kwargs), owns the env-var logic. Everyanthropic.Anthropic(...)site incli.py,report/generator.py,generate_report.py,utilities/agentic_enhancer/agent.py,utilities/context_enhancer.py,utilities/finding_verifier.py, and theAnthropicClientconstructor inllm_client.pyitself routes through it.resolve_model_id(value)function used bycore/analyzer.pyandcore/enhancer.py.experiment.pyis left alone since it has its own model-selection block aimed at the research entry points.--modelargparsechoices=["opus", "sonnet"]was relaxed to accept any string so slash-form IDs can flow through.ANTHROPIC_API_KEYorOPENANT_LLM_API_KEYso OpenRouter setups don't have to set both.Test coverage
All in
tests/test_llm_provider_routing.py(18 cases) plus an updated case intests/test_token_tracker.py:resolve_model_idcoversopus/sonnetaliases, explicit Claude IDs, slash-form passthrough,openrouter/prefix stripping (including the substring-not-prefix edge), and the empty-string case.get_anthropic_clientis checked with no env vars set (no overrides passed), with both env vars set (passed through), with explicit kwargs winning over env, and with onlyOPENANT_LLM_BASE_URLset.get_pricingcovers known Claude pricing, the unknown-model$0fallback with one-time warning, override merge over the built-in table, override replacing known Claude pricing, invalid JSON ignored with stderr message, and non-object override ignored.TokenTracker.record_callis verified to honourMODEL_PRICING_OVERRIDEend-to-end.pytestfromlibs/openant-core/: 112 passed, 12 skipped, 0 failures.Out of scope
No provider abstraction layer / no OpenAI-SDK path — the issue offered that as a fallback if a bigger refactor was preferred, but the env-var passthrough closes the request with ~50 lines of helper code and zero behaviour change for Claude users. Happy to do the larger refactor in a follow-up if maintainers prefer.