Skip to content

feat(agent-langgraph): add orcarouter as a named BOT_PROVIDER - #291

Open
putraperdana1207-pixel wants to merge 1 commit into
CopilotKit:mainfrom
putraperdana1207-pixel:feat/orcarouter-provider
Open

feat(agent-langgraph): add orcarouter as a named BOT_PROVIDER#291
putraperdana1207-pixel wants to merge 1 commit into
CopilotKit:mainfrom
putraperdana1207-pixel:feat/orcarouter-provider

Conversation

@putraperdana1207-pixel

Copy link
Copy Markdown

What this changes

Adds OrcaRouter as a first-class named provider for the LangGraph Bot, agent-langgraph.

OpenBot's framework Bot already lets a deployment choose who answers its model calls via
BOT_PROVIDER, with openai, anthropic and google as named options. This adds orcarouter to
that list, wired to mirror the existing openai branch exactly, because OrcaRouter speaks the same
OpenAI API.

OrcaRouter is an OpenAI-compatible AI gateway built for both models and
agents. Like OpenRouter, it exposes a provider/model namespace across many models — the catalogue
serves namespaced ids such as orcarouter/fusion and openai/gpt-5.5 behind one endpoint — but it
also combines adaptive routing, automatic failover, zero-markup inference, observability,
guardrails, and agent-tool governance behind the same endpoint. Adding orcarouter as a first-class
provider means this project's users can use that stack directly, without treating OrcaRouter as an
anonymous custom base URL.

It also runs gateway-level, zero-trust security for AI agents on the same endpoint — screening every
prompt/response and governing every tool call on a default-deny basis, with no application code
changes.

Why this shape

The repo already documents that any OpenAI-shaped endpoint can stand in for OpenAI via
OPENAI_BASE_URL, and the framework Bot reads that for BOT_PROVIDER=openai. OrcaRouter could be
reached that way. But the repo treats openai, anthropic and google as named providers — each
with its own key variable, base URL and default model — precisely so a deployment never needs a key
for a vendor it does not use. orcarouter is the first provider that is not a model vendor, and it
belongs in that registry rather than as a OPENAI_BASE_URL incantation:

File What changed Mirrors
agent-langgraph/src/index.ts BOT_PROVIDER=orcarouter branch in buildModel(), defaultModelFor(), KEY_VARIABLE, plus ORCAROUTER_BASE_URL the openai branch, which also constructs a ChatOpenAI
.env.example ORCAROUTER_API_KEY, ORCAROUTER_BASE_URL documented next to the other providers' keys and base URLs ANTHROPIC_API_KEY / ANTHROPIC_BASE_URL, GOOGLE_API_KEY / GOOGLE_GENERATIVE_AI_BASE_URL
docs/configuration.md BOT_PROVIDER row extended; ORCAROUTER_API_KEY / ORCAROUTER_BASE_URL rows; a paragraph in "OpenAI-compatible endpoints" the paragraphs already written for the other three providers
docker-compose.yml forwards ORCAROUTER_API_KEY and ORCAROUTER_BASE_URL into the agent-langgraph container the lines already forwarding ANTHROPIC_* / GOOGLE_*
README.md "the LangGraph Bot can use OpenAI, Anthropic, Google, or OrcaRouter" the existing Requirements line
CHANGELOG.md an Unreleased entry the repo's changelog style

Configuration

BOT_PROVIDER=orcarouter
ORCAROUTER_API_KEY=...
BOT_MODEL=orcarouter/fusion

ORCAROUTER_BASE_URL defaults to https://api.orcarouter.ai/v1 and can point at a self-hosted
gateway. Model names travel verbatim, exactly as the repo documents for the other providers.

Where it runs

  • New state that outlives a request? None. This is a configuration-time choice resolved at
    process start, the same way BOT_PROVIDER=anthropic is. Nothing is held in a module-level
    Map or closure.
  • What happens on the second replica? Nothing different. The provider is chosen from the
    environment before the process serves, so every replica reads the same variables and builds
    the same client. No shared state.
  • Anything serialised? No. The Bot's tool loop calls back through the existing
    OPENBOT_TOOL_URL / AGENT_TOOL_TOKEN path unchanged.
  • Anything fanned out to a browser? No.
  • New listener, port, or schedule? No. agent-langgraph still listens on its one port.

Boundary and audit

  • Every acting call still goes through the gateway. This change only picks a model endpoint; it
    does not add a tool path or touch the server gateway, policy, or audit.
  • New refusals and new failures each write a row. None added.
  • Nothing new is trusted from the client that the server can resolve itself.

Changelog

  • Added under Unreleased.

Proof

  • bun run format:check, bun run lint, bun run typecheck — clean.
  • agent-langgraph tests — 15 pass. tests/compose.test.ts — 7 pass.
  • Type-checking agent-langgraph/src/index.ts against the pristine file shows no new errors (the
    package is not part of the root typecheck workspace, matching CI).
  • Live test against the real endpoint: booted agent-langgraph with
    BOT_PROVIDER=orcarouter + ORCAROUTER_API_KEY, and ran a full AG-UI turn through
    /ag-ui. /health reported {"status":"ok","provider":"orcarouter","model":"orcarouter/fusion"},
    and the turn streamed RUN_STARTEDTEXT_MESSAGE_*RUN_FINISHED with HTTP 200. A
    tool-calling turn also streamed TOOL_CALL_START/ARGS/END through the same endpoint.

Note: the full bun test suite needs a PostgreSQL service (CI spins one up); those
ERR_POSTGRES_CONNECTION_REFUSED failures are environmental here, not from this change.

I'm an engineer on the OrcaRouter team.
Discord: discord.gg/YEubt8enRA · X: https://x.com/OrcaRouter

Add orcarouter as a fourth provider for the framework Bot, wired exactly
like the existing openai branch: its own key (ORCAROUTER_API_KEY), a
default endpoint (ORCAROUTER_BASE_URL, https://api.orcarouter.ai/v1) and
a namespaced default model (orcarouter/fusion). OrcaRouter is an
OpenAI-compatible gateway, so the ChatOpenAI integration carries it.

Document the new provider in .env.example, docs/configuration.md and the
README, forward the variables through docker-compose, and note the new
option in the changelog.

Co-Authored-By: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant