Skip to content

fix(amazon-bedrock): correct open model catalog - #6794

Closed
rekram1-node wants to merge 3 commits into
devfrom
feat/bedrock-open-models
Closed

fix(amazon-bedrock): correct open model catalog#6794
rekram1-node wants to merge 3 commits into
devfrom
feat/bedrock-open-models

Conversation

@rekram1-node

Copy link
Copy Markdown
Collaborator

Summary

Remaining open-model split of #6637, based on current dev: DeepSeek, Qwen, xAI, NVIDIA, Google, MiniMax, Z.AI, and Moonshot.

  • Audit all 30 existing entries; preserve every existing ID and route.
  • Correct confirmed prices, dates, modalities, open-weight facts, limits, and host reasoning controls.
  • Convert third-party provider definitions to override-only base_model entries.
  • Add three live-verified Gemma 4 Mantle Responses models and complete Gemma 3 lab metadata.
  • Keep Gemma 3 tool_call=false: forced calls can be emitted, but valid native tool-result continuations fail.
  • Keep Bedrock DeepSeek R1 tool_call=false after explicit API rejection.
  • Retain Kimi K2.5’s temperature override after live Converse accepted both temperature=0 and temperature=1.

Unresolved NVIDIA controls remain conservative rather than inventing caller options. New Chat/GovCloud candidates with incomplete controls or client-route support are deferred.

Sources are in leading TOML comments, including AWS model cards and public pricing.

Validation

  • bun validate
  • All new lab entries contain required capabilities, modalities, context, and output
  • git diff --check
  • No model deletions or route changes

@github-actions

Copy link
Copy Markdown
Contributor

Action items

  • [high] [violation] models/google/gemma-3-4b-it.toml:11 - Check: Lab metadata must stay provider-agnostic; host-only capability limits belong on the provider entry. Why: tool_call = false is justified only by Bedrock Converse tool-result failures (PR body + Bedrock comments). Lab files are shared facts about the model; this value will be inherited by any future base_model consumer and contradicts other hosts that already advertise Gemma 3 tool use. Bedrock R1 correctly keeps lab tool_call = true and overrides on the provider. Action: Set lab Gemma 3 IT entries to the model-level capability (likely tool_call = true), and set tool_call = false on providers/amazon-bedrock/models/google.gemma-3-{4b,12b,27b}-it.toml as the host-specific override.
  • [medium] [possible mistake] providers/amazon-bedrock/models/deepseek.v3-v1:0.toml:7 - Check: Provider models that return a reasoning side channel should set interleaved for this API. Why: Leading comments for V3.1 and V3.2 state live Converse returns reasoningContent when thinking is on, but neither file sets interleaved, while the same PR sets interleaved = true for Kimi/GLM peers that use the same side channel. Action: Verify Converse reasoning content shape for deepseek.v3-v1:0 and deepseek.v3.2; add interleaved = true (or field = "reasoning_content") if reasoningContent is returned, or correct the comments if it is not.
  • [medium] [possible mistake] providers/amazon-bedrock/models/zai.glm-4.7-flash.toml:7 - Check: Same-host GLM reasoning peers should mark the reasoning side channel consistently. Why: Live notes say reasoning_config=high returns reasoningContent, and zai.glm-4.7 / zai.glm-5 in this PR set interleaved = true, but flash omits it. Action: Confirm whether flash returns the same reasoning side channel; if yes, add interleaved to match the sibling GLM Bedrock entries.

@rekram1-node

Copy link
Copy Markdown
Collaborator Author

Addressed in 38611e3.

  • Gemma 3 lab metadata now records the provider-agnostic model capability with tool_call = true for 4B/12B/27B.
  • Each Bedrock Gemma 3 provider entry explicitly overrides tool_call = false, preserving the live Bedrock finding that forced tool generation works but native tool-result continuation fails.
  • Added interleaved = true to DeepSeek V3.1, DeepSeek V3.2, and GLM 4.7 Flash. Live Converse evidence already recorded in the headers confirms that thinking mode returns reasoningContent; default/off checks omit it.

bun validate and git diff --check pass.

@github-actions

Copy link
Copy Markdown
Contributor

Action items

  • [high] [possible mistake] providers/amazon-bedrock/models/minimax.minimax-m2.5.toml:4 - Check: Provider interleaved must be set when this host returns a reasoning side channel. Why: The live note says Converse returns reasoningContent for MiniMax M2.5 (and M2/M2.1 share the same always-on Bedrock surface), but none of the MiniMax entries set interleaved, while sibling Bedrock reasoners that emit reasoningContent (DeepSeek V3.x, GLM, Kimi) do. Clients will miss the reasoning channel. Action: Add interleaved = true (or the correct field form) on minimax.minimax-m2.toml, minimax.minimax-m2.1.toml, and minimax.minimax-m2.5.toml, or document why MiniMax does not expose a parseable side channel on this host.
  • [high] [possible mistake] providers/amazon-bedrock/models/qwen.qwen3-32b-v1:0.toml:7 - Check: Provider interleaved must be set when this host returns a reasoning side channel. Why: The live note states reasoning_config=high returns reasoningContent, but the file only sets reasoning_options = [{ type = "toggle" }] and omits interleaved, unlike other Bedrock open models updated in this PR for the same Converse behavior. Action: Add interleaved = true (or the correct field form) for qwen.qwen3-32b-v1:0, or explain why reasoningContent is not a usable interleaved channel here.
  • [medium] [possible mistake] providers/amazon-bedrock/models/qwen.qwen3-vl-235b-a22b.toml:5 - Check: Host structured_output overrides must reflect verified API capability. Why: Lab metadata has structured_output = true, and this PR flips the Bedrock entry to false with only model-card/pricing comments—no live or docs note that structured/JSON output fails on this route. Action: Cite host evidence that structured output is unsupported on Bedrock for this model, or drop the override and inherit the lab value.
  • [low] [possible mistake] providers/amazon-bedrock/models/minimax.minimax-m2.1.toml:8 - Check: Material capability flips should be evidenced. Why: All three MiniMax entries change structured_output from false to true, but first-party MiniMax entries do not claim it and the leading comments only cover thinking/pricing/limits—not structured output. Action: Add a short leading note of the Bedrock check that confirms structured output, or leave the field unset/false until verified.

@rekram1-node

Copy link
Copy Markdown
Collaborator Author

Addressed the second review pass in 7b23de6 with direct Converse verification:

  • Added interleaved = true to MiniMax M2, M2.1, and M2.5. All three returned reasoningContent without a caller reasoning setting.
  • Added interleaved = true to Qwen 3 32B, matching the existing live high/low evidence.
  • Confirmed structured output on all three MiniMax models: a contradictory prose instruction plus outputConfig.textFormat produced valid schema-constrained JSON.
  • Confirmed the same structured-output behavior on Qwen 3 VL 235B, so removed its incorrect structured_output = false host override and now inherit the lab’s true capability.
  • Added concise live-evidence notes to the affected headers.

Sanitized results: MiniMax M2/M2.1/M2.5 and Qwen VL all returned HTTP 200; MiniMax responses included reasoning blocks and all four returned schema-valid {ok: boolean} JSON.

bun validate and git diff --check pass.

@rekram1-node

Copy link
Copy Markdown
Collaborator Author

Superseded by three smaller PRs based on current dev after the broad reviewer repeatedly hung:

All findings from both completed review passes are included, and each split passes bun validate independently.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant