Skip to content

fix(provider): discover models from openai-compatible endpoints - #48549

Open
yeager1977 wants to merge 1 commit into
anomalyco:devfrom
yeager1977:provider-model-discovery
Open

fix(provider): discover models from openai-compatible endpoints#48549
yeager1977 wants to merge 1 commit into
anomalyco:devfrom
yeager1977:provider-model-discovery

Conversation

@yeager1977

Copy link
Copy Markdown

Issue for this PR

Closes #27553

Type of change

  • Bug fix
  • New feature
  • Refactor / code improvement
  • Documentation

What does this PR do?

Custom openai-compatible providers configured in opencode.json show a stale model list after options.baseURL changes. opencode never fetches /models from the endpoint; the list is assembled from static models config and the models.dev catalog, both keyed by provider ID. Changing baseURL only changes the request target, not the displayed list. Reproduced with an Ollama server moved between hosts: the old host's models kept showing.

When building provider state, for config providers with npm: "@ai-sdk/openai-compatible", this fetches GET {baseURL}/models and uses the server response as the model list:

  • Server-served models replace the stored list; config entries the server still serves are kept as templates (options/variants/limits preserved).
  • Discovery failure (endpoint down, non-OpenAI shape, timeout) falls back to the configured list.
  • Honors options.baseURL -> api precedence matching resolveSDK, ${VAR} env interpolation, stored auth keys, and filters embedding/rerank entries.
  • Skips discovery when the provider ID matches a models.dev entry with the same api URL (catalog is authoritative there).

How did you verify your code works?

  • New tests in packages/opencode/test/provider/provider.test.ts with a mock OpenAI-compatible server on a random port: discovery replaces the configured list, and discovery failure keeps the configured models. Verified failing before the fix, passing after.
  • test/provider/: 716 pass, 0 fail.
  • bun typecheck clean in packages/opencode.

Screenshots / recordings

N/A (non-UI).

Checklist

  • I have tested my changes locally
  • I have not included unrelated changes in this PR

Custom openai-compatible providers showed stale model lists because
nothing fetched /models from the configured baseURL: the list came from
static config entries and the models.dev catalog keyed by provider ID,
so changing options.baseURL only changed the request target.

Discover GET {baseURL}/models at provider state build for config
providers with npm @ai-sdk/openai-compatible. Server-served models
replace the stored list (keeping config-declared entries the server
still serves); discovery failures keep the configured list. Honors
options.baseURL/api precedence, ${VAR} env interpolation, stored auth
keys, and skips embedding/rerank entries.

(cherry picked from commit 67b44e0)
@github-actions

Copy link
Copy Markdown
Contributor

The following comment was made by an LLM, it may be inaccurate:

Based on my search, I found several related PRs that are worth noting:

Potentially Related PRs:

  1. PR feat(opencode): local LAN provider discovery + auto-discover models #27554 - feat(opencode): local LAN provider discovery + auto-discover models

    • This appears to be directly related as it addresses auto-discovery of models from OpenAI-compatible providers
  2. PR feat(opencode): auto-discover models from OpenAI-compatible providers #32731 - feat(opencode): auto-discover models from OpenAI-compatible providers

    • This is closely related and likely addresses a similar feature for auto-discovering models
  3. PR feat(provider): add dynamic model discovery for custom providers #42660 - feat(provider): add dynamic model discovery for custom providers

    • Related feature for dynamic model discovery from custom providers
  4. PR fix(provider): retain Zen discovery transport metadata #47224 - fix(provider): retain Zen discovery transport metadata

    • Related fix for provider discovery functionality

These PRs all involve model discovery from OpenAI-compatible endpoints, which is the core functionality your PR #48549 is implementing. I recommend checking these (especially #27554 and #32731) to ensure your fix doesn't duplicate work or conflict with existing implementations.

@yeager1977

Copy link
Copy Markdown
Author

Checked all four related PRs the bot flagged. Keeping this PR's focused implementation rather than folding them in — rationale:

This change is the minimal fix for the stale model list in #27553 and is green/mergeable today. Happy to follow up with the useful refinements from #32731 (e.g. a discoverModels: false opt-out or a debug log on discovery failure) in a separate PR rather than expanding this one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FEATURE]: Auto-discover models from OpenAI-compatible providers

1 participant