Repository navigation
fix(provider): OpenAI Compatible 模型验证误报'没有可用模型' / fix model validation false-negative for OpenAI-compatible providers - #779
Open
WenhuaXia wants to merge 1 commit into
Conversation
3 of 4 tasks
…on false-negative for OpenAI-compatible providers 问题 / Problem: onboarding 配置本地模型(vLLM/Ollama 等 OpenAI Compatible)时, '保存并验证模型' 永远报错'该 Provider 没有可用的模型进行测试', 即使远程 /v1/models 正常返回且模型可用。 When configuring local models (vLLM/Ollama, etc.) during onboarding, 'save and validate' always reports 'No models available' even though the remote /v1/models endpoint works. 根因 / Root cause: openai_compatible.py 的 get_models() 只返回 flocks.json 里手动配置的 _config_models,从不请求远程 /v1/models。catalog 中 openai-compatible 的 models 列表为空,所以 isolated 验证时 probe_models 恒为空, 14ms 内就返回 no_models_available,根本没连远程。 get_models() only returns models manually configured in flocks.json and never fetches remote /v1/models. The openai-compatible catalog entry has zero predefined models, so probe_models is always empty during isolated validation and it bails out with no_models_available in ~14ms without ever contacting the upstream. 修复 / Fix: 在 isolated 验证分支中,若 probe_models 为空且用户在表单里明确填了 model_id,则用该 id 合成最小 ModelInfo,让验证真正发起 chat 请求。 In the isolated validation branch, when probe_models is empty and the user explicitly supplied a model_id, synthesize a minimal ModelInfo so the real chat call validates the model against the upstream. 验证 / Verified: 修复前: no_models_available (14ms,未连远程) 修复后: 连接成功, answer=Paris (1015ms,真实推理) Before: no_models_available (14ms, never hit upstream) After: success, answer=Paris (1015ms, real inference)
WenhuaXia
force-pushed
the
fix/openai-compatible-model-validation
branch
from
October 13, 2026 11:03
bdb9a77 to
4576038
Compare
WenhuaXia
added a commit
to WenhuaXia/flocks
that referenced
this pull request
Oct 13, 2026
… to default LLM
问题 / Problem:
从模型管理页添加模型(非 onboarding 向导)后,provider 已配置、凭据已存,
但 default_models.llm 为空。/api/onboarding/status 要求'默认模型已设置
+ 凭据可用'才返回 completed=True,WebUI 在 completed=False 时每次登录
强制弹出初始化向导(前端 dismissed flag 已被测试显式忽略)。
结果:用户明明加了模型,每次登录还是被弹初始化。
Adding a model from the model-management page (not the onboarding
wizard) leaves default_models.llm empty. /api/onboarding/status only
returns completed=True when a default model is set AND credentials
exist, and the WebUI force-opens the setup wizard whenever
completed=False (the frontend dismissed flag is explicitly ignored).
So users who already added a model get the onboarding popup on
every login.
根因 / Root cause:
create_model (custom_provider.py) 只写 provider.{id}.models.{model_id},
从不调 set_default_model。只有 onboarding /apply 路径(1023-1057 行)
会设默认,而该路径此前被 AgentFlocks#779 的验证 bug 挡住,用户只能跳过 onboarding
后手动加模型,陷入'有模型无默认'的中间态。
create_model only upserts the model entry and never calls
set_default_model. Only the onboarding /apply path sets the default,
and that path was previously blocked by the validation bug in AgentFlocks#779 —
users skipped onboarding, added the model manually, and were left with
'a model but no default'.
修复 / Fix:
create_model 新增 _maybe_promote_first_model_to_default:当且仅当
(1) 该 provider 此前无模型(这是第一个模型)、(2) 全局尚无默认 LLM、
(3) 该 provider 已有可用凭据(secret 或 inline apiKey)时,自动把新模型
设为默认 LLM。
- 仅第一个模型才提升 → 不会在后续加第二/第三个模型时覆盖用户已选默认
- 无默认才提升 → 不覆盖 onboarding/其它 provider 已设的默认
- 有凭据才提升 → 与 status.completed 判定对齐,避免'设了默认但跑不通'
的矛盾态(无凭据时不设,用户补 key 后在模型管理页一键设默认)
- 提升失败 try/except 兜底,绝不阻塞模型创建本身
create_model now calls _maybe_promote_first_model_to_default: the new
model is promoted to the default LLM only when (1) the provider had no
models before, (2) no global default LLM exists, and (3) the provider
already has usable credentials. Failures are swallowed — model creation
never blocks on default promotion.
验证 / Verified:
- 新增 5 个回归测试(首个模型+凭据→设默认;inline key→设默认;
无凭据→不设;第二个模型→不覆盖;其它 provider 已有默认→不覆盖),
8/8 passed
- 相邻模块 53 passed / 5 skipped,零回归
- 真实环境数据修复:将 Qwen3.8-27B 设为默认后 /api/onboarding/status
completed=True,登录不再弹
- 8 regression tests: 8/8 passed; neighboring modules: 53 passed / 5 skipped
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
问题描述 / Bug Description
onboarding 配置本地模型(vLLM/Ollama 等 OpenAI Compatible provider)时,点击「保存并验证模型」永远报错 「该 Provider 没有可用的模型进行测试」(
no_models_available),即使远程服务的/v1/models端点正常返回、模型也确实可用。When configuring a local model (vLLM/Ollama, etc. via the OpenAI Compatible provider) in onboarding, clicking "save and validate" always reports "该 Provider 没有可用的模型进行测试" (
no_models_available), even when the remote service's/v1/modelsendpoint returns models correctly.根因 / Root Cause
flocks/provider/sdk/openai_compatible.py的get_models()只返回flocks.json里手动配置的_config_models,从不请求远程/v1/models:而 catalog 中
openai-compatible的预定义模型数为 0(对照threatbook-cn-llm有 11 个)。因此 isolated 验证时probe_models恒为空,在 ~14ms 内就返回no_models_available—— 验证逻辑根本没接触到远程服务。ThreatBook 不受影响,因为它的 catalog 自带模型列表,
_prepare_isolated_provider的 backfill 能补上。修复 / Fix
在
flocks/server/routes/provider.py的 isolated 验证分支中:若probe_models为空、且用户在表单里明确填了model_id,则用该 id 合成一个最小ModelInfo,让验证真正发起 chat 请求去校验该模型。When
probe_modelsis empty and the user explicitly supplied amodel_idin the onboarding form, synthesize a minimalModelInfoso the real chat call validates that model against the upstream instead of failing early.验证 / Verification
直接调用 onboarding 使用的同一函数
_test_provider_credentials_impl(target:http://192.168.250.10:8000/v1, modelQwen3.8-27B, vLLM TP4×4):远程服务健康:
/v1/models带 key 返回Qwen3.8-27B。测试计划 / Test Plan
风险评估 / Risk Assessment
Low — 改动仅 22 行,位于 isolated 验证分支,仅在 probe_models 为空且用户显式提供 model_id 时触发;合成的是临时 ModelInfo,不写入 flocks.json,不影响任何持久状态。
注: uv.lock 的本地环境漂移(aliyun mirror → pypi)已还原,未包含在本 PR。