feat(glm5.2-fp4-mi355x-sglang): add agentic-coding MTP benchmark config on MI355X / 新增 GLM-5.2 FP4 MI355X SGLang Agentic MTP 基准测试配置

[giovanniguastiamd]

Summary

Add an agentic-coding benchmark configuration for GLM-5.2 in FP4 precision on AMD MI355X using SGLang with Multi-Token Prediction (MTP) speculative decoding.

Config key: glm5.2-fp4-mi355x-sglang-agentic-mtp

Key parameters:

• Image: lmsysorg/sglang-rocm:v0.5.16-rocm720-mi35x-20260728
• Tensor parallelism: TP=4, Expert parallelism: EP=4
• KV offloading: DRAM via hicache backend
• Concurrency sweep: [1, 2, 4, 8, 10]
• Spec decoding: MTP

中文说明

为 AMD MI355X 上的 GLM-5.2 FP4 精度模型新增 SGLang 框架的 agentic-coding 基准测试配置，启用多词元预测（MTP）投机解码。

配置键： glm5.2-fp4-mi355x-sglang-agentic-mtp

主要参数：

• 镜像：lmsysorg/sglang-rocm:v0.5.16-rocm720-mi35x-20260728
• 张量并行：TP=4，专家并行(EP)：EP=4
• KV 卸载：通过 hicache 后端卸载至 DRAM
• 并发扫描：[1, 2, 4, 8, 10]
• 投机解码：MTP

[github-actions]

Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase As a PR reviewer and CODEOWNER, I have reviewed this and have.

For PR verification, add the full-sweep-fail-fast label (strongly recommended) to this PR — the benchmark sweep only runs on labeled PRs. Use full-sweep-enabled only if you need matrix jobs to keep running past a failure.

PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs

---

感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 As a PR reviewer and CODEOWNER, I have reviewed this and have。

如需进行 PR 验证，请为此 PR 添加 full-sweep-fail-fast 标签（强烈推荐）— 基准测试 sweep 仅在带有标签的 PR 上运行。仅当需要矩阵任务在失败后继续运行时才使用 full-sweep-enabled。

PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档

[claude]

🔴 Line 81 checks require_agentic_kv_offload_backend mooncake, but this recipe's only config arm in amd-master.yaml sets kv-offload-backend: { name: hicache }, which the launcher exports as KV_OFFLOAD_BACKEND=hicache. Since the check requires an exact string match under KV_OFFLOADING=dram, it hard-exits (exit 1) before the SGLang server ever starts, failing every concurrency point in the sweep. All 8 sibling single-node SGLang agentic scripts that use hicache offload call require_agentic_kv_offload_backend hicache, confirming mooncake here is a copy-paste mistake — should be hicache.

Extended reasoning...

The bug: benchmarks/single_node/agentic/glm5.2_fp4_mi355x_sglang_mtp.sh:81 calls:

if require_agentic_kv_offload_backend mooncake; then

But the config entry this script serves — glm5.2-fp4-mi355x-sglang-agentic-mtp in configs/amd-master.yaml — has exactly one search-space arm:

- { tp: 4, ep: 4, kv-offloading: dram, kv-offload-backend: { name: hicache }, conc-list: [1, 2, 4, 8, 10], spec-decoding: mtp }

How it fails at runtime: The sweep launcher (run-sweep.yml) reads matrix.config['kv-offload-backend'].name and exports it verbatim as KV_OFFLOAD_BACKEND (mapped through benchmark-tmpl.yml), so at runtime KV_OFFLOAD_BACKEND=hicache and KV_OFFLOADING=dram.

require_agentic_kv_offload_backend in benchmark_lib.sh:38-56 handles the dram case like this:

dram)
    if [[ "${KV_OFFLOAD_BACKEND:-}" != "$expected_backend" ]]; then
        echo "Error: expected KV_OFFLOAD_BACKEND=$expected_backend when KV_OFFLOADING=dram, got '${KV_OFFLOAD_BACKEND:-}'" >&2
        exit 1
    fi
    ...

This is a hard exit 1, not a return, so it terminates the whole script — even though it's being called inside an if condition, which is normally where set -e scripts avoid tripping on non-zero returns. The explicit exit bypasses that entirely.

Step-by-step proof of failure:

1. CI schedules the config's only arm: kv-offloading: dram, kv-offload-backend: { name: hicache }.
2. run-sweep.yml exports KV_OFFLOAD_BACKEND=hicache into the job environment.
3. The script sources benchmark_lib.sh, then at line 81 runs require_agentic_kv_offload_backend mooncake.
4. Inside the function: KV_OFFLOADING=dram → enters the dram) branch → compares KV_OFFLOAD_BACKEND ("hicache") against expected_backend ("mooncake") → mismatch → prints Error: expected KV_OFFLOAD_BACKEND=mooncake when KV_OFFLOADING=dram, got 'hicache' and calls exit 1.
5. The script dies immediately, before resolve_trace_source, before install_agentic_deps, and long before the SGLang server is launched.
6. Since kv-offloading: dram is the only arm in this recipe's search space, every single job in this sweep (all 5 concurrency points: 1, 2, 4, 8, 10) fails identically at startup.

Why nothing else catches this: there's no validation anywhere that cross-checks the script's expected offload backend against the yaml config's kv-offload-backend.name at config-authoring time — the check only fires at runtime inside the launched job, and by then it's a guaranteed failure on the very first run.

Why mooncake is clearly the wrong string: looking at every sibling single-node SGLang agentic script that offloads via hicache — dsv4_fp4_b300_sglang.sh, dsv4_fp4_b200_sglang.sh, qwen3.5_fp8_mi355x_sglang.sh, qwen3.5_fp4_b300_sglang_mtp.sh, glm5.2_fp4_b300_sglang.sh, qwen3.5_fp8_h100.sh, qwen3.5_fp8_b300_sglang.sh, qwen3.5_fp4_b200_sglang_mtp.sh — all of them call require_agentic_kv_offload_backend hicache to match their yaml's kv-offload-backend: { name: hicache }. This script is the sole outlier calling mooncake while still configured with hicache. It's a plausible copy/paste artifact: the script does use Mooncake as SGLang's internal --hicache-storage-backend mooncake (the L3 storage tier under HiCache), which is a distinct, SGLang-internal identifier from the InferenceX-level KV_OFFLOAD_BACKEND value — the two are easy to conflate but are not the same thing.

Fix: change line 81 to require_agentic_kv_offload_backend hicache, matching the yaml config and every sibling recipe.

[claude]

🔴 Trailing whitespace after the backslash on line 199 (--speculative-num-draft-tokens 6 \ ) means bash does NOT treat it as a line continuation inside the SGLANG_CMD array — it escapes only the first following space, injecting a stray single-space array element between the speculative-decoding args and ${CACHE_ARGS[@]}. This bogus positional token gets passed to python3 -m sglang.launch_server, which will reject it via argparse ("unrecognized arguments"), breaking server startup on every arm of this new config.

Extended reasoning...

The bug: In the SGLANG_CMD array literal (lines 186-199), each array element is normally just placed on its own line — no backslash needed, since newlines already separate array elements in a bash (...) literal. Lines 196-198 add trailing backslashes anyway (harmless, since the backslash is the last character on those lines and bash's line-continuation logic swallows the following newline cleanly). Line 199 is different: --speculative-num-draft-tokens 6 \ has trailing spaces after the backslash, so the backslash is no longer the last character on the line. In that position, bash does not perform line-continuation — instead the backslash escapes only the very next character (a space), producing a literal-space word. The remaining unescaped spaces on that line then terminate the word, so it becomes a standalone array element consisting of a single space character, sitting between 6 and ${CACHE_ARGS[@]}.\n\nReproduced locally with the exact construct from the file:\n\nARR=(\n --speculative-algorithm EAGLE \\n --speculative-num-steps 5 \\n --speculative-eagle-topk 1 \\n --speculative-num-draft-tokens 6 \ \n --enable-hierarchical-cache\n --watchdog-timeout 1800\n)\n\nyields:\n\n[6]=<--speculative-num-draft-tokens>\n[7]=<6>\n[8]=< > <-- stray literal-space element\n[9]=<--enable-hierarchical-cache>\n\nexactly matching what four independent verifiers reproduced against the actual file content.\n\nWhy nothing catches it: set -x and set -eo pipefail are enabled, but neither detects an extra positional argv token — it's syntactically valid bash, just semantically wrong. printf '%q ' "${SGLANG_CMD[@]}" (line 202) would actually reveal it in sglang_command.txt as a stray ' ' (quoted empty-looking space) between the draft-tokens value and the cache args, but that's only visible after the fact in a log file, not something that fails CI before the run.\n\nImpact: SGLANG_CMD is invoked unconditionally at line 209 ("${SGLANG_CMD[@]}" > "$SERVER_LOG" 2>&1 &) for the sole arm of this new config (tp:4, ep:4, DP_ATTENTION=false). sglang.launch_server uses argparse with no positional arguments defined in ServerArgs, so a stray token that doesn't start with - is treated as an unexpected positional and argparse raises error: unrecognized arguments: ' '", causing the process to exit immediately. wait_for_server_ready` will then time out waiting for the port to come up (or detect the dead PID) and the whole benchmark run fails before any request is ever served — on every concurrency point of the sweep, since the server args are shared across all of them.\n\nFix: remove the trailing whitespace after the backslash on line 199 (or, more robustly, just drop the backslash entirely like the other elements in this array, since newlines already delimit array elements and the backslashes on 196-198 aren't functionally required either).

[claude]

🟡 The new perf-changelog.yaml entry for glm5.2-fp4-mi355x-sglang-agentic-mtp sets pr-link to pull/2463, but this PR is #2488. Please update the link to point to #2488, matching the convention every other entry follows (e.g. the preceding entry correctly links to #2446).

Extended reasoning...

The newly added perf-changelog.yaml entry for glm5.2-fp4-mi355x-sglang-agentic-mtp (lines 5428-5434) sets:

pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2463

but per the PR metadata, this PR is #2488, created 2026-08-04. #2463 is an unrelated, earlier PR number that has nothing to do with this GLM-5.2 agentic-coding config.

Why this is a bug: every other entry in perf-changelog.yaml links to the PR that actually introduced that entry — that's the whole point of the file, it's a provenance/changelog record. The entry immediately preceding this one (lines 5421-5427) links to pull/2446, and #2446 is indeed the PR titled "Update MinimaxM3 with TP2EP1" that added that TP2EP1 search-space line — the link is self-consistent. By contrast, the new GLM-5.2 entry's link does not point back to the PR that added it (#2488); it points to a different, older PR. This is almost certainly a copy-paste artifact — the author likely reused an older changelog block as a template and forgot to update the PR number to the one actually being opened.

Proof by walkthrough:

1. Look at PR metadata: <pr number="2488">, created 2026-08-04T14:30:37Z.
2. Look at the diff hunk added to perf-changelog.yaml in this same PR: it adds config-keys: [glm5.2-fp4-mi355x-sglang-agentic-mtp] with pr-link: .../pull/2463.
3. Compare with the convention used elsewhere in the file (e.g., the entry just above, for dsv4-fp8-mi325x-vllm-mtp search-space additions, links to .../pull/2446 — and recent git history confirms commit 50bd859 is titled "[AMD][MI355X] Update MinimaxM3 with TP2EP1 ([AMD][MI355X] Update MinimaxM3 with TP2EP1 #2446)", i.e., that entry's link is correct and self-referential).
4. Therefore the new entry should link to .../pull/2488 (this PR) instead of .../pull/2463, to match the same self-referential convention.

Impact: This is purely a documentation/traceability defect in perf-changelog.yaml. It does not affect the benchmark config parsing, the launcher script, or any runtime behavior — pr-link is metadata consumed only by humans (or downstream changelog tooling) trying to trace which PR introduced a given config-key. A wrong link makes that traceability slightly misleading (pointing readers to an unrelated PR) but causes no functional failure, so it doesn't need to block merge — just a one-line fix to the URL.

Fix: Change pull/2463 to pull/2488 in the new perf-changelog.yaml entry.