[Klaud Cold] Enact the August 3, 2026 scenario and precision deprecations / 执行 2026 年 8 月 3 日场景与精度下线 - #2493
Conversation
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献!请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md,然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人,你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板,包括保留英文语句 如需进行 PR 验证,请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动(flake),重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
Archive the deprecated master-config entries into configs/deprecated/ and move their benchmark scripts into the sibling deprecated/ directories, per the Deprecation Notice in MODELS.md. 54 config keys move out of the active NVIDIA and AMD master configs: minimaxm3-8k1k 39 Single-turn 8k1k (agentic coding stays active) kimik2.5-agentic 6 agentic coding (8k1k stays until August 6, 2026) qwen3.5-bf16 9 all bf16 recipes, every scenario The speculative-decoding A/B retirements are NOT enacted here. Removing the non-spec-decode agentic arm would leave MiniMax-M3 and GLM-5.2 with no active config at all, since their replacement EAGLE3/MTP arms have not landed, and would drop all AMD and SGLang agentic coverage for dsv4 and qwen3.5. That round is deferred to a follow-up once the spec-decode arms exist. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Mark the first deprecation table as enacted and update the matrix rows for minimaxm3, kimik2.5 and qwen3.5. The speculative-decoding A/B table gets an explicit "not yet enacted" status note explaining why, and the dsv4 / glm5.2 rows no longer claim MTP-only coverage that is not in force. Mirrored in MODELS_zh.md. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ecc5f97 to
e607399
Compare
There was a problem hiding this comment.
Additional findings (outside current diff — PR may have been updated during review):
-
🔴
configs/deprecated/nvidia-agentic-non-spec-master.yaml:92-121— configs/deprecated/nvidia-agentic-non-spec-master.yaml archives four GB300 dynamo-sglang agentic keys (qwen3.5-fp4-gb300-dynamo-sglang-agentic-agg/-disagg, dsv4-fp4-gb300-dynamo-sglang-agentic-agg/-disagg) and removes them from configs/nvidia-master.yaml, but every search-space row in all four uses spec-decoding: mtp — there is no non-MTP sibling for these keys.Extended reasoning...
configs/deprecated/nvidia-agentic-non-spec-master.yamlis meant to hold only the non-speculative-decode arm of the dsv4/qwen3.5/glm5.2 agentic A/B pairs — its own header comment says so explicitly: "Deprecated non-speculative-decoding agentic-coding entries archived from nvidia-master.yaml ... the spec-decode arm is the better Pareto frontier, so only that arm is run and published." That contract is violated by four of the entries this PR moved into the file:qwen3.5-fp4-gb300-dynamo-sglang-agentic-agg— both search-space rows setspec-decoding: "mtp"(lines ~105, 121 of the archive)qwen3.5-fp4-gb300-dynamo-sglang-agentic-disagg— all 7 rows setspec-decoding: "mtp"dsv4-fp4-gb300-dynamo-sglang-agentic-agg— its single row setsspec-decoding: mtpdsv4-fp4-gb300-dynamo-sglang-agentic-disagg— all rows setspec-decoding: mtp
These are the spec-decode (MTP) arm, not the non-spec arm the file is supposed to collect. Contrast with the genuinely non-spec entries correctly archived in the same file (e.g.
dsv4-fp4-gb300-dynamo-vllm-agentic, thedsv4-fp4-gb200-dynamo-vllm-agentic-*entries), which all setspec-decoding: none/omit it.The Deprecation Notice in
MODELS.mdthat this PR itself enacts is explicit: for bothqwen3.5anddsv4, only the non-MTP agentic arm is deprecated; "Agentic coding, MTP" is the arm that "remains"/is published. Grepping the activeconfigs/nvidia-master.yamlafter this PR confirms none of these fourgb300-dynamo-sglang-agentickeys remain active, and there is no like-for-like GB300 dynamo-sglang replacement — the block that stayed active for qwen3.5 GB300-class agentic isqwen3.5-fp4-b300-sglang-agentic-mtp, which is a different recipe (single-node b300, not multi-node gb300-dynamo-sglang), and dsv4 keeps onlygb300-dynamo-vllmagentic MTP, notgb300-dynamo-sglang. So this PR silently drops real, currently-published GB300 dynamo-sglang MTP agentic coverage for both models with no replacement.The likely root cause: the migration script/author probably classified entries into "non-spec" vs "spec" buckets by key-name suffix (looking for a trailing
-mtp) rather than by the actualspec-decodingfield inside the search-space rows. These four keys don't have an-mtpsuffix in their names (unlike e.g.qwen3.5-fp4-b300-sglang-agentic-mtp), even though their contents are MTP recipes — so they fell through the suffix-based filter into the non-spec archive.Proof, step by step:
- Open
configs/deprecated/nvidia-agentic-non-spec-master.yamlin this diff and findqwen3.5-fp4-gb300-dynamo-sglang-agentic-agg:. - Its
search-spacerows read- spec-decoding: "mtp" ... conc-list: [1]and- spec-decoding: "mtp" ... conc-list: [96]— both explicitly MTP, not "none". - Compare to the file's own header: "only the spec-decode arm is run and published" is deprecated-file language for removing the non-spec arm — but these rows ARE the spec-decode arm.
- Check
configs/nvidia-master.yamlpost-PR:grep gb300-dynamo-sglang-agenticreturns nothing — the key is gone from the active config with no replacement of the same platform/engine/multinode shape. - Check
MODELS.md's own updated matrix in this PR: for qwen3.5, "Agentic coding (MTP only from 2026-08-03)" is listed as the active/remains scenario, and for dsv4 the same. Removing the MTP entries directly contradicts what the PR's own documentation says should stay active.
Impact: this is a benchmark-coverage regression, not a crash — no user-facing failure occurs, but real, currently-published GB300 multi-node dynamo-sglang MTP agentic benchmark coverage for two models (qwen3.5 and dsv4) silently disappears from the sweep, with the archive's header and MODELS.md's matrix now factually describing coverage that no longer exists on that platform.
Fix: move these four keys (and their search-space content) back into
configs/nvidia-master.yamlas active configs — they are the arm the Deprecation Notice says to keep — or, if their removal was actually intended for a different reason, updateMODELS.mdand the archive header to correctly describe why the GB300 dynamo-sglang MTP arm no longer runs.
|
|
||
| **Monday, August 3, 2026** is the last day for the scenarios, precisions, and recipe variants listed below; they are deprecated after that date. | ||
|
|
||
| **Enacted on 2026-08-04** in [#2493](https://github.com/SemiAnalysisAI/InferenceX/pull/2493): 80 config keys were removed from the active master configs and archived under [`configs/deprecated/`](configs/deprecated/), and their benchmark scripts moved to the sibling `deprecated/` directories. Kimi-K3 is the one exception — its non-DSpark agentic arm is listed below but was deliberately kept live in that PR, so both arms still run pending a follow-up. | ||
|
|
||
| Scenario and precision retirements: | ||
|
|
||
| | Model | Deprecated | Remains | |
There was a problem hiding this comment.
🟡 The self-referential PR link uses a literal PRNUM placeholder instead of 2493 in both MODELS.md (lines 15, 56, 57, 61) and the mirrored lines in MODELS_zh.md, producing 8 broken /pull/PRNUM links (404) in the docs that are this PR's own deliverable. Fix with a simple find-and-replace of PRNUM → 2493 in both files.
Extended reasoning...
This PR's whole purpose is to record the enactment of the August 3, 2026 deprecation notice and link back to the PR that did it. Four places in MODELS.md (lines 15, 56, 57, 61) and the four mirrored lines in MODELS_zh.md write [#PRNUM](https://github.com/SemiAnalysisAI/InferenceX/pull/PRNUM) where the actual PR number, 2493, should be. This is a leftover template placeholder — the author correctly filled in the real numbers for every other PR reference on those lines (e.g. #2268, #1724, #734), but couldn't have known this PR's own number ahead of time and never went back to substitute it once the PR was opened.
I verified this against the actual committed file content, not the PR diff view: grep -n PRNUM MODELS.md MODELS_zh.md on the current working tree returns all 8 occurrences, and git show HEAD:MODELS.md confirms the same text is present in the commit that shipped this change (5b5606e, "docs(MODELS): record the August 3, 2026 deprecations as enacted"). The diff shown during review substitutes the placeholder with #2493 for display purposes, which is why it can look already-fixed there — but the file actually on disk/in HEAD still has the raw placeholder.
Concretely, anyone who clicks the "Enacted on 2026-08-04 in #PRNUM" link at MODELS.md:15, or either of the "no active config as of 2026-08-04 (#PRNUM)" callouts at lines 56-57, or the "Agentic coding (removed 2026-08-04, #PRNUM)" note at line 61, lands on https://github.com/SemiAnalysisAI/InferenceX/pull/PRNUM, which is not a valid PR number and 404s. Same for the four mirrored Chinese-language lines in MODELS_zh.md. Since the whole point of these notes is to let a future reader jump straight to the PR that performed the archival/removal, a dead link defeats that purpose, even though it doesn't break any code, CI, or benchmark behavior.
Fix is mechanical: replace PRNUM with 2493 in both files (8 substitutions total, matching the #2493 display already shown in the PR's own diff view). Since this is a documentation-only defect with no functional or CI impact, I'm marking it a nit rather than a blocking issue — but it should be fixed before merge since it's trivial and the whole point of the change is to leave working self-referential links.
Summary
Today is Tuesday, August 4, 2026, so the August 3, 2026 block of the Deprecation Notice in
MODELS.mdis now past its last run day. This PR enacts the scenario and precision retirements from that block.54 config keys move out of the active NVIDIA and AMD master configs into
configs/deprecated/, and their 45 benchmark scripts move into the siblingdeprecated/directories.minimaxm3-8k1kkimik2.5-agenticqwen3.5-bf16Archive files:
configs/deprecated/{nvidia,amd}-{minimaxm3-8k1k,kimik2.5-agentic,qwen3.5-bf16}-master.yaml.Not in this PR: the speculative-decoding A/B retirements
The second table in the August 3 block retires the non-spec-decode agentic arm of five models. That round is deliberately deferred, because the replacement arms do not exist yet:
Every non-spec-decode agentic config therefore stays active here.
MODELS.mdcarries an explicit "Status: not yet enacted" note under that table with the reasoning, and thedsv4/glm5.2matrix rows no longer advertise MTP-only coverage that is not actually in force.Verification
active ∪ archived == origin/mainat the key level,active ∩ archived == ∅, and every entry's parsed value is byte-identical toorigin/main. Blocks were moved as verbatim text, so comments and formatting are preserved. NVIDIA 157 → 125 active + 32 archived; AMD 78 → 56 active + 22 archived.dsv4-fp4-b200-vllm-agentic,glm5.2-fp4-b300-sglang-agentic,minimaxm3-fp4-mi355x-vllm-agentic,minimaxm3-fp8-h100-vllm-agentic,qwen3.5-fp8-mi355x-sglang-agentic,kimik3-fp4-b300-vllm-agentic.(prefix, precision, runner-family, scenario)fingerprint against all 181 surviving configs — zero collisions.benchmark_lib.shsource,INFERENCEX_ROOT,chat_templates/). Every relative reference resolves to an existing file and all passbash -n.kimik2.5_fp4_b200.sh,models_vllm.yaml,KLAUD_DEBUG.md); all repointed.configs/ci-priority.yamlandconfigs/CONFIGS.mdreference none of the deprecated keys.srt-slurm-recipes/YAMLs are left in place for deprecated models, and noperf-changelog.yamlentry is added.中文说明
今日为 2026 年 8 月 4 日(星期二),
MODELS.md弃用公告中 2026 年 8 月 3 日 一节已过最后运行日。本 PR 执行其中的场景与精度下线:54 个配置键从启用的 NVIDIA 与 AMD master 配置移入configs/deprecated/,其 45 个基准测试脚本移入同级deprecated/目录。投机解码 A/B 下线本轮不执行:其替代分支尚未合入 —— 若此刻移除,MiniMax-M3 与 GLM-5.2 将没有任何启用配置,DeepSeek-V4-Pro 与 Qwen3.5 在 AMD 及 SGLang 上的智能体覆盖亦将全部消失。所有非投机解码智能体配置在本 PR 中保持启用,
MODELS.md已注明该表「尚未执行」及其原因。