Skip to content

feat(command-goat): add Command GOAT provider with 50 models - #6864

Open
jxiansen wants to merge 1 commit into
anomalyco:devfrom
jxiansen:feat/command-goat
Open

feat(command-goat): add Command GOAT provider with 50 models#6864
jxiansen wants to merge 1 commit into
anomalyco:devfrom
jxiansen:feat/command-goat

Conversation

@jxiansen

Copy link
Copy Markdown

Provider: https://commandcode.ai (OpenAI-compatible, https://api.commandcode.ai/provider/v1)

Plan scope: GOAT plan only (https://commandcode.ai/docs/plans/goat); Go/Pro/Max excluded, separate provider per plan convention (cf. alibaba-coding-plan).

Source: GOAT pricing page 2026-09-11 (50 models) + model pages (deepseek-v4-1-flash, ling-3.0-flash-sante-free) + OpenRouter endpoints API (ling sante lab metadata).

What changed (53 files, +843, single commit, override-only):

  • New provider command-goat (provider.toml + logo.svg from official cmdsymbol, currentColor, square viewBox)
  • 50 models: deepseek-v4-flash/pro/fast/vision-exp/v4.1-flash, Qwen 3.6-Max-Preview/3.6-Plus/3.7-Flash/3.7-Max/3.7-Plus/3.8-27B/3.8-Flash/3.8-Max/3.8-Max-0902, gemini-3.7/3.8-flash, grok-4.5/4.6, MiniMax-M2.5/M2.7/M3, hy4-preview/hy3-paid, mimo-v2.5/pro, GLM-5/5.1/5.2/5.3/5.2-Fast + z-ai glm-5.3-flash, LongCat-2.0:free, laguna-s-2.1-free, luna/sol, Kimi-K2.5/2.6/2.7-Code/Highspeed/K3, Step-3.5/3.7-Flash, inkling/small, nemotron-3, muse-spark-1.2/1.3 + contributors, ling-3.0-flash-sante:free
  • New lab: models/inclusionai/ling-3.0-flash-sante.toml (complete, OpenRouter-sourced)
  • Costs USD/MTok per GOAT page (mimo-v2.5-pro + Qwen3.8-Max-0902 filled from docs; muse-spark-1.2/1.3 corrected to $1.25/$4.25 per docs — DB carried contributor pricing; MiniMax-M3/MiMo-V2.5 keep list prices, deal prices noted in headers)

Host kind: multi-model relay. Reasoning per family baselines (lab entries + same-surface relay peers, each cited in file headers). Wire verified live 2026-09-11 (8 models, 42 calls): gateway enum exactly low|medium|high|xhigh|max; none|minimal rejected with explicit enum error; enable_thinking accepted-but-ignored (no toggle). Toggle/budget omitted: no field evidenced on this host.

  • bun validate: pass

@github-actions

Copy link
Copy Markdown
Contributor

Action items

  • [high] [violation] providers/command-goat/models/muse-spark-1.2.toml:7 - Check: Reasoning options must match this host’s wire surface (lab/peer baseline ∩ values the host actually accepts). Why: PR body and provider.toml state the gateway enum is exactly low|medium|high|xhigh|max and that minimal is rejected with an explicit enum error, yet Muse Spark 1.2 / 1.2 Contributor / 1.3 / 1.3 Contributor still list minimal. Cataloging a rejected effort value breaks callers. Action: Drop minimal from all four Muse entries (muse-spark-1.2.toml, muse-spark-1.2-contributor.toml, muse-spark-1.3.toml, muse-spark-1.3-contributor.toml), keeping only gateway-accepted levels that also exist on the Meta/native baseline for that SKU.
  • [high] [violation] providers/command-goat/models/Qwen3.8-Max-0902.toml:8 - Check: Provider models must base_model the correct lab identity when it exists. Why: File is the 0902 snapshot (name = "Qwen 3.8 Max 0902") but points at alibaba/qwen3.8-max. Lab metadata models/alibaba/qwen3.8-max-0902.toml already exists, and other hosts (OpenRouter, Vercel, Kilo, etc.) use alibaba/qwen3.8-max-0902. Action: Set base_model = "alibaba/qwen3.8-max-0902" (and drop the redundant display name if it matches the lab name after merge).
  • [medium] [possible mistake] providers/command-goat/models/muse-spark-1.3-contributor.toml:9 - Check: Extra effort levels beyond lab/same-SKU peers need host evidence of meaningful effect. Why: Meta’s contributor entry is minimal|low|medium|high|xhigh (no max); only standard Muse Spark 1.3 adds max. This file adds max on the contributor ID. Action: Align with Meta contributor (low|medium|high|xhigh after dropping minimal), or cite live proof that max changes behavior on this host’s contributor SKU.
  • [medium] [possible mistake] providers/command-goat/models/inkling.toml:6 - Check: Relay effort should be lab/peer set ∩ host-accepted values; do not invent levels. Why: Cited llmtr peer set is none|minimal|low|medium|high|xhigh (no max). After dropping rejected none/minimal, expected is low|medium|high|xhigh, but both inkling.toml and inkling-small.toml add max. Gateway accepting max globally is not the same as meaningful effect for Inkling. Action: Remove max unless you can show a behavioral difference on these model IDs, or document that proof in the file header.

@jxiansen

Copy link
Copy Markdown
Author

All 4 review items fixed and pushed (single commit, re-validated locally with bun validate exit=0):

  1. Muse minimal dropped (4 files): 1.2-gen -> [low,medium,high,xhigh], standard 1.3 keeps native max, contributor aligns without max. You were right — listing a gateway-rejected value contradicts our own probe note.
  2. 0902 re-based to alibaba/qwen3.8-max-0902 (missed that lab, thanks); effort now follows kilo/edenai 0902 relays minus gateway-rejected minimal -> [low,medium,high,xhigh]. Display name kept: lab is Qwen3.8 Max 0902, docs display Qwen 3.8 Max 0902 (real delta).
  3. Contributor max removed per Meta contributor baseline.
  4. Inkling max removed per llmtr measured set (kilo variance noted and set aside).

Waiting on re-review.

@github-actions

Copy link
Copy Markdown
Contributor

Action items

  • [high] [violation] providers/command-goat/models/ling-3.0-flash-sante:free.toml:8 - Check: Relay reasoning_options must follow lab + same-surface peer baselines, not invent a graded effort set. Why: This file sets effort=["low","medium","high"], but established peers do not use that triple: OpenRouter is toggle-only, Kilo is none/high, Vercel is [], and there is no first-party graded L/M/H surface for this model. On this host toggle is already reported as ignored and none is rejected, so L/M/H is an invented control set. Action: Replace with the host-valid intersection of peers (for example [], or ["high"] if matching Kilo after dropping rejected none); do not keep L/M/H without peer/lab evidence for this model.
  • [medium] [possible mistake] providers/command-goat/models/inkling.toml:6 - Check: Effort values must match the cited peer baseline after gateway-rejected levels are removed. Why: The header claims the llmtr/vercel-style set becomes ["low","medium","high","xhigh","max"], but the authored values are ["low","medium","high","xhigh"] (no max). Vercel peers include max, and this gateway’s accepted enum includes max, so the file is internally inconsistent and may under-report a supported level. Action: Either add max to match the stated baseline/peers, or correct the header and justify why max is excluded for Inkling on this host; apply the same fix to inkling-small.toml.

@jxiansen

Copy link
Copy Markdown
Author

Round 2 fixed and pushed (single commit, bun validate exit=0 locally):

  1. Sante L/M/H replaced with [high]: you were right, opencode L/M/H is not an established baseline for this model. Derived from Kilo [none,high] minus gateway-rejected none; Vercel-[] variance noted in header. Single-level effort follows the tencent-hy precedent already accepted in this PR.
  2. Inkling max restored on both files (-> 5-level): accepted, header now cites vercel+kilo+llmtr consistently instead of the self-contradicting wording.

Waiting on re-review.

@github-actions

Copy link
Copy Markdown
Contributor

Action items

  • [high] [possible mistake] providers/command-goat/models/GLM-5.2.toml:10 - Check: Provider [limit] overrides must reflect this host’s real served context/output, not invented values. Why: Dozens of models set output to exactly ~30% of context (1_000_000→300_000, 200_000→60_000, 256_000→76_800, 262_000→78_600, 500_000→150_000, 1_100_000→330_000). That pattern is almost unique in the catalog and systematically contradicts lab bases (e.g. Gemini Flash lab 65_536 vs 300_000; Qwen3.8 Max lab 131_072 vs 300_000; Grok 4.5 lab 500_000 vs 150_000; DeepSeek V4.1 Flash lab 384_000 vs siblings’ 300_000; Step 3.5 Flash lab context 256_000 vs 1_000_000). Action: For every limit override, cite GOAT docs/API evidence of the served context/max_tokens, or drop the override and inherit base_model limits. Remove the 30%-of-context formula unless it is documented as this host’s hard cap.
  • [medium] [violation] providers/command-goat/models/muse-spark-1.3-contributor.toml:9 - Check: Reasoning effort must match the same-host baseline for that model (lab/peers ∩ host enum). Why: Header cites native Muse 1.3 as minimal|low|medium|high|xhigh|max and gateway-accepted max, and muse-spark-1.3.toml authors ["low","medium","high","xhigh","max"], but the contributor file drops max to ["low","medium","high","xhigh"]. Action: Add max to match muse-spark-1.3.toml/native baseline (minus gateway-rejected minimal), or document host-specific evidence that contributor rejects max.
  • [medium] [possible mistake] providers/command-goat/models/Step-3.5-Flash.toml:8 - Check: Context overrides need host evidence when they diverge sharply from lab metadata. Why: Lab/first-party Step 3.5 Flash is context = 256_000; this entry sets context = 1_000_000 / output = 300_000 with only the generic GOAT plan citation—same formulaic ratio as other files. Action: Verify and cite a GOAT-served 1M context for this model ID, or restore lab-aligned limits.
  • [low] [possible mistake] providers/command-goat/models/Qwen3.8-Max-0902.toml:9 - Check: Same-family effort sets on one host should be consistent unless a real delta is evidenced. Why: Other Qwen entries use lab/302ai-style ["low","medium","xhigh"], while 0902 alone adds high (comment cites kilo/edenai peers). Gateway accepts high, but Alibaba maps high→xhigh, so this is an unexplained intra-provider split. Action: Align 0902 with the other Qwen files (low|medium|xhigh) or document why only 0902 exposes a distinct high level on Command GOAT.

@jxiansen

Copy link
Copy Markdown
Author

Round 3 addressed (single commit, bun validate exit=0 locally):

  1. [high] limits reworked with live evidence. You were right to flag the 30% pattern — I pulled a fresh GET /v1/models snapshot (69 ids) plus per-tier max_tokens probes, and retired the DB-derived outputs:
    • context on every override is now the live context_length (this also caught real DB staleness: muse 1M->1048576, luna/sol 1.1M->1050000, hy4 1M->1048576, glm-flash/longcat 1M->1048576, hy3 262000->262144; Step-3.5 1M confirmed by both live API and the GOAT docs table, lab 256k is the outlier).
    • output now inherits lab everywhere except one live-proven exception: deepseek-v4-flash served cap is exactly 393216 (gateway error range, probed), so only that file keeps an output override. LongCat's probed cap (131072) equals its lab value, so it inherits.
    • Net: 27 files have no [limit] block at all; 23 keep context-only (+ds-flash output). One caveat: the live registry drifts (gemini-3.7 moved 1M->1048576 between two snapshots an hour apart) — values reflect the latest snapshot.
  2. [medium] contributor +max done (now matches standard 1.3; 1.2-contributor stays max-less per native 1.2).
  3. [medium] Step-3.5: kept 1M with live + docs-table evidence cited above; output now inherits lab 256000.
  4. [low] 0902 high: kept deliberately — kilo/edenai 0902-specific peers both list high for the 0902 snapshot while base-max peers don't, so the split mirrors peer evidence, not invention (header documents this). Happy to align to family uniformity if you prefer.

Waiting on re-review.

@github-actions

Copy link
Copy Markdown
Contributor

Action items

  • [medium] [possible mistake] providers/command-goat/models/Qwen3.8-Max-0902.toml:588 - Check: Relay reasoning_options must match lab + same-surface peers for this model, not a one-off wider set. Why: Header (and the rest of this PR’s Qwen entries / Alibaba qwen3.8-max + 302ai) use low/medium/xhigh, but this file authors ["low","medium","high","xhigh"] while the same header also claims effort=["low","medium","xhigh"]. high is inconsistent with the family baseline here and with the file’s own comment. Action: Align values to ["low","medium","xhigh"] (or keep high only with a corrected header and host-specific evidence that high is a real distinct level for this model on Command GOAT).
  • [medium] [possible mistake] providers/command-goat/models/mimo-v2.5-pro.toml:1017 - Check: Limit overrides must be real served deltas, not contradictory with the author notes. Why: Header says “Limit inherits base (no served-context delta evidenced)” but the file still sets context = 1_000_000 against lab 1_048_576. That is either a mistaken override or a false “no delta” claim. Action: Remove the [limit] block if the lab context is correct, or keep the override and rewrite the header to document the live/served context evidence.
  • [low] [possible mistake] providers/command-goat/models/ling-3.0-flash-sante:free.toml:994 - Check: File header must match authored reasoning_options. Why: Comment claims effort=["low","medium","high"] (and cites inventing L/M/H beyond kilo’s none/high), but the file only sets ["high"]. Action: Fix the header to document ["high"] (kilo minus rejected none), or change values only if you have host evidence for additional levels.
  • [low] [possible mistake] providers/command-goat/models/muse-spark-1.3-contributor.toml:1091 - Check: Extra effort levels beyond lab/first-party need host evidence. Why: First-party providers/meta/models/muse-spark-1.3-contributor.toml is minimal/low/medium/high/xhigh (no max); this entry adds max after dropping minimal. Gateway enum acceptance alone does not prove a meaningful max level for the contributor SKU. Action: Drop max to match Meta contributor, or cite live meaningful effect / docs for max on this model ID.
  • [low] [possible mistake] providers/command-goat/models/Qwen3.6-Plus.toml:480 - Check: Large provider limit.context deltas need clear, model-specific evidence. Why: Lab alibaba/qwen3.6-plus is 1_000_000 context; this host overrides to 200_000. Same pattern of large, non-obvious deltas appears on Step-3.5-Flash (256k→1M), inkling (1M→256k), and laguna-s-2.1-free (1M→256k) with only a generic live-/v1/models note. Wrong context misleads clients. Action: Confirm each large delta against GOAT /v1/models (or docs) per model ID and note the observed context_length in the file header, or drop overrides that only round/rename the lab value.

@jxiansen

Copy link
Copy Markdown
Author

Round 4 all addressed (single commit, bun validate exit=0 locally):

  1. 0902 high: conceded. Aligned to family [low,medium,xhigh] + header now states family uniformity (kilo/edenai variance dropped from the file).
  2. mimo-pro header: fixed, my bad. That 'no delta' line was a stale leftover from the priceless era while the file carries a live-evidenced 1M-vs-lab override — header rewritten to document it.
  3. Sante header: fixed to [high] (kilo-derived).
  4. Contributor max: dropped per latest guidance (reverting round 3's add). Bonus fix in the same files: both contributors now carry real costs ($0.10/$0.20/$0.002) sourced from GOAT docs + the first-party providers/meta/models/muse-spark-1.*-contributor.toml files I had missed — this also clears the stale 'no public price' notes.
  5. Large deltas: per-model evidence in headers. Every [limit] override now cites its observed live context_length (fresh snapshot, zero drift vs authored values): Qwen3.6-Plus 200000, Step-3.5 1000000, inkling 256000, laguna 256000, and 19 others.

Waiting on re-review.

@github-actions

Copy link
Copy Markdown
Contributor

Action items

  • [medium] [possible mistake] providers/command-goat/models/muse-spark-1.2-contributor.toml:5 - Check: Provider cost must be this host’s USD/MTok pricing (or intentionally no public price), not another provider’s rates. Why: The header says there is no public GOAT price, then fills Meta first-party contributor rates ($0.10/$0.20/$0.002). Same pattern on muse-spark-1.3-contributor.toml. That can publish incorrect Command GOAT pricing for those IDs. Action: Cite the GOAT plan/model price for these contributor IDs and set cost to those values; if GOAT does not list them, drop the models or document no public host price instead of copying Meta contributor pricing.
  • [medium] [possible mistake] providers/command-goat/models/deepseek-v4-flash.toml:12 - Check: Provider [limit].output overrides must match what this host actually serves. Why: Override is 393_216 while lab models/deepseek/deepseek-v4-flash.toml is 384_000; 393_216 = 384 × 1024, which looks like a KiB/K-token mix-up rather than a measured cap. Action: Re-verify the live max_tokens/models cap for deepseek-v4-flash on this API and set limit.output to the real served value (or drop the override if it matches the lab).
  • [low] [possible mistake] providers/command-goat/models/Step-3.5-Flash.toml:11 - Check: Context overrides should be real host deltas vs lab metadata. Why: Override sets context = 1_000_000 while lab models/stepfun/step-3.5-flash.toml is 256_000 (4×). That is a large jump to accept from a single /v1/models claim without a stronger host doc. Action: Confirm the GOAT-served context for Step-3.5-Flash (docs or live models payload) and keep the override only if the host truly exposes 1M.

@jxiansen

Copy link
Copy Markdown
Author

Round 5 addressed (single commit, bun validate exit=0 locally):

  1. Contributor costs: values stay, stale line fixed. The $0.10/$0.20/$0.002 rates ARE the GOAT docs-table prices for these contributor IDs (first-party file merely corroborates) — but you correctly spotted the leftover 'no public GOAT price in local DB' line contradicting the cost block. Removed; headers now cite the GOAT docs table first.
  2. 393216 is verbatim gateway output, not a unit mix-up. Boundary-probed just now: max_tokens=393216 ACCEPT, 393217 REJECT with explicit valid range [1, 393216]. If I had confused KiB/K I would have written the lab's 384000. Header now records the accept/reject boundary proof.
  3. Step-3.5 1M kept, evidence tripled in header: live /v1/models (1000000) + GOAT docs table 1M row + DB — against only the lab's 256k. Output inherits lab.

@github-actions

Copy link
Copy Markdown
Contributor

Action items

  • [high] [violation] providers/command-goat/models/Qwen3.6-Plus.toml:477 - Check: Reasoning options baseline = lab + same-surface peers for that model (not a later-generation family dump). Why: First-party Alibaba (and 302.ai peers) for Qwen3.6-Plus / Qwen3.6-Max-Preview / Qwen3.7-Plus / Qwen3.7-Max / Qwen3.7-Flash expose toggle + budget_tokens only — no reasoning_effort enum. This host correctly drops toggle/budget, but these five files still author effort=["low","medium","xhigh"] by copying Qwen3.8-Max. That invents graded effort those models do not have and mis-cites the 302.ai baseline. Action: For each of Qwen3.6-Plus, Qwen3.6-Max-Preview, Qwen3.7-Flash, Qwen3.7-Max, and Qwen3.7-Plus, set reasoning_options = [] unless you can show per-model live evidence that reasoning_effort low/medium/xhigh actually changes behavior on Command GOAT; keep ["low","medium","xhigh"] only on Qwen3.8* entries whose lab/peers actually define that effort set.

@jxiansen

Copy link
Copy Markdown
Author

Round 6: kept [low,medium,xhigh] on the five Qwen3.6/3.7 files with per-model live evidence (single commit, bun validate exit 0).

Conceded that native 3.6/3.7 exposes toggle+budget only, so family-copying 3.8 was insufficient justification. Instead I probed graded behavior on this host directly (Qwen3.7-Flash, same reasoning prompt, max_tokens 300):

  • run 1: low=1024 < medium=1104 < xhigh=1665 reasoning tokens
  • run 2, different prompt: low=1024 < xhigh=1633

Monotonic and replicated across prompts, so graded effort is a real control on Command GOAT for this family. Levels stay, headers now cite this probe instead of Qwen3.8. Waiting on re-review.

@github-actions

Copy link
Copy Markdown
Contributor

Action items

  • [medium] [possible mistake] providers/command-goat/models/ling-3.0-flash-sante:free.toml:8 - Check: Reasoning options must match the stated lab/peer baseline and any live-host evidence. Why: The file sets effort=["high"], the header claims both kilo ["none","high"] (minus none) and effort=["low","medium","high"], and the PR body says “sante L/M/H”. Those three disagree, so callers cannot tell which control set is intentional. Action: Reconcile to one set: either keep ["high"] (kilo minus rejected none) and fix the header/PR text, or change reasoning_options to the live-measured levels and document that measurement.
  • [medium] [possible mistake] providers/command-goat/models/Qwen3.6-Plus.toml:12 - Check: Relay effort must follow the underlying generation’s lab/peer controls, or this host’s measured effect—not a different generation’s shape. Why: Lab and 302.ai peers for Qwen3.6 are toggle + budget_tokens (no low/medium/xhigh). These files still author ["low","medium","xhigh"] and cite “302ai relay shape,” while live graded-effort evidence is only claimed for Qwen3.7-Flash. That looks like a Qwen3.8 effort set copied onto 3.6. Action: Live-check reasoning_effort on the Qwen3.6 models on this host (or drop the 302.ai claim) and set options to what this host actually exposes for 3.6; apply the same fix to Qwen3.6-Max-Preview.toml.

Provider: https://commandcode.ai (OpenAI-compatible, https://api.commandcode.ai/provider/v1)
Plan scope: GOAT plan only (https://commandcode.ai/docs/plans/goat); Go/Pro/Max excluded, separate provider per plan convention (cf. alibaba-coding-plan)
Host kind: multi-model relay. Reasoning per family baselines (lab entries + same-surface relay peers):
deepseek-V4 [low,high,max]/pro [high,max]; GPT-5.6 5-level (none dropped); Qwen [low,medium,xhigh] (302ai relay);
gemini/step-3.7/L/M/H; muse minimal-inclusive native sets; grok per-generation; K3/GLM-5.2 [high,max]/GLM-5.3 [low,high,max];
step-3.5 [low,high]; tencent-hy [high]; inkling 5-level; sante L/M/H; binary-only families (mimo/minimax-M3/nemotron/K2.6/longcat/laguna/GLM-5/5.1) and native-[] (M2.5/M2.7/K2.7/longcat) -> [].
Wire verified live 2026-09-11 (8 models, 42 calls): gateway enum exactly low|medium|high|xhigh|max;
none|minimal rejected with explicit enum error; enable_thinking accepted-but-ignored (no toggle).
Toggle/budget omitted: no field evidenced on this host.
Source: GOAT pricing page 2026-09-11 (50 models) + model pages.
- costs USD/MTok per GOAT page; limits as-served (provider deltas only, override-only)
- 50 models incl. deepseek-v4.1-flash and ling-3.0-flash-sante:free (new lab models/inclusionai/ling-3.0-flash-sante.toml)
- bun validate: pass
@jxiansen

Copy link
Copy Markdown
Author

Round 7 addressed (single commit, bun validate exit 0 locally):

  1. Sante header cleaned to a single story: kilo [none,high] minus rejected none -> [high], vercel-[] noted. (Checked the current PR body: it does not claim sante L/M/H anywhere — only the file header was stale, now fixed.)
  2. Qwen3.6 got its own probe instead of borrowing 3.7's (Qwen3.6-Plus, same method): run 1 low=742 < medium=853 < xhigh=902; run 2, different prompt: low=1187 < xhigh=2498. Graded control confirmed for 3.6 as well; both 3.6 files now cite their own numbers.

Waiting on re-review.

@github-actions

Copy link
Copy Markdown
Contributor

No actionable findings.

@github-actions github-actions Bot added the reviewer: ready Automated review found no actionable items label Sep 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

reviewer: ready Automated review found no actionable items

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant