Skip to content

feat: support OpenRouter / non-Claude models via env-var passthrough (#9)#52

Open
ChrisJr404 wants to merge 1 commit intoknostic:masterfrom
ChrisJr404:feat/openrouter-llm-provider-9
Open

feat: support OpenRouter / non-Claude models via env-var passthrough (#9)#52
ChrisJr404 wants to merge 1 commit intoknostic:masterfrom
ChrisJr404:feat/openrouter-llm-provider-9

Conversation

@ChrisJr404
Copy link
Copy Markdown

Closes #9.

Implements the env-var-passthrough proposal from #9 (comment) plus the openrouter/ prefix convention Didier raised in #9 (comment).

What changed

Three env vars + a slightly more permissive --model argument are all that's needed to drive Qwen / Kimi / MiniMax / DeepSeek (or any Anthropic-compatible endpoint) without leaving the existing SDK:

export OPENANT_LLM_BASE_URL=https://openrouter.ai/api/v1
export OPENANT_LLM_API_KEY=sk-or-v1-...
openant scan /path/to/repo --model qwen/qwen-3-coder-480b

When OPENANT_LLM_BASE_URL is unset, every Anthropic client construction falls back to the SDK defaults. Existing Claude setups behave exactly as before.

--model now accepts

Form Example Effect
Alias opus, sonnet Resolves to the canonical Claude ID.
Explicit Claude ID claude-opus-4-6 Used verbatim.
Slash-form ID qwen/qwen-3-coder-480b Used verbatim against the configured endpoint.
OpenCode-style prefix openrouter/moonshotai/kimi-k2 Leading openrouter/ is stripped per Didier's suggestion (becomes moonshotai/kimi-k2).

Cost tracking

The hardcoded pricing table only knows Claude. Two follow-on bits:

  1. Unknown model IDs now default to {input: 0, output: 0} with a one-time stderr warning, so cost rollups stay honest instead of silently estimating with Sonnet rates.
  2. A new MODEL_PRICING_OVERRIDE env var (JSON of {model_id: {input, output}} per million tokens) merges over the built-in table so power users can plug in real OpenRouter pricing without code changes:
export MODEL_PRICING_OVERRIDE='{"qwen/qwen-3-coder-480b": {"input": 0.4, "output": 1.6}}'

Implementation notes

  • One central helper, utilities.llm_client.get_anthropic_client(**kwargs), owns the env-var logic. Every anthropic.Anthropic(...) site in cli.py, report/generator.py, generate_report.py, utilities/agentic_enhancer/agent.py, utilities/context_enhancer.py, utilities/finding_verifier.py, and the AnthropicClient constructor in llm_client.py itself routes through it.
  • Model resolution lives in a single resolve_model_id(value) function used by core/analyzer.py and core/enhancer.py. experiment.py is left alone since it has its own model-selection block aimed at the research entry points.
  • --model argparse choices=["opus", "sonnet"] was relaxed to accept any string so slash-form IDs can flow through.
  • API-key checks now accept either ANTHROPIC_API_KEY or OPENANT_LLM_API_KEY so OpenRouter setups don't have to set both.

Test coverage

All in tests/test_llm_provider_routing.py (18 cases) plus an updated case in tests/test_token_tracker.py:

  • resolve_model_id covers opus / sonnet aliases, explicit Claude IDs, slash-form passthrough, openrouter/ prefix stripping (including the substring-not-prefix edge), and the empty-string case.
  • get_anthropic_client is checked with no env vars set (no overrides passed), with both env vars set (passed through), with explicit kwargs winning over env, and with only OPENANT_LLM_BASE_URL set.
  • get_pricing covers known Claude pricing, the unknown-model $0 fallback with one-time warning, override merge over the built-in table, override replacing known Claude pricing, invalid JSON ignored with stderr message, and non-object override ignored.
  • TokenTracker.record_call is verified to honour MODEL_PRICING_OVERRIDE end-to-end.

pytest from libs/openant-core/: 112 passed, 12 skipped, 0 failures.

Out of scope

No provider abstraction layer / no OpenAI-SDK path — the issue offered that as a fallback if a bigger refactor was preferred, but the env-var passthrough closes the request with ~50 lines of helper code and zero behaviour change for Claude users. Happy to do the larger refactor in a follow-up if maintainers prefer.

…nostic#9)

Closes knostic#9. Routes every anthropic.Anthropic(...) construction through a
single helper (utilities.llm_client.get_anthropic_client) that picks up
two env vars when set:

    OPENANT_LLM_BASE_URL=https://openrouter.ai/api/v1
    OPENANT_LLM_API_KEY=sk-or-v1-...

When unset the SDK uses its existing defaults, so Claude users see no
behaviour change.

--model now accepts any string. The new resolver:
  - keeps "opus" / "sonnet" aliases (canonical Claude IDs)
  - passes a slash-form ID through verbatim (qwen/qwen-3-coder-480b)
  - strips a leading "openrouter/" prefix per the OpenCode convention
    Didier flagged on the issue, so openrouter/moonshotai/kimi-k2 just
    works out of the box.

Cost rollups previously fell back to Sonnet rates for unknown model IDs.
Now unknown IDs default to {input: 0, output: 0} with a one-time stderr
warning, so totals stay honest. A new MODEL_PRICING_OVERRIDE env var
(JSON of {model_id: {input, output}} per million tokens) merges over the
built-in table for power users who want real OpenRouter pricing.

Tests cover: env-var passthrough on/off, kwarg precedence, slash-form
resolution, openrouter/ prefix stripping, $0 fallback + warning, and
override merge. All 112 tests pass.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature]: support of OpenRouter for access to other models

1 participant