perf(agentx): enable MI355X MiniMax M3 EAGLE3 / perf(agentx)：为 MI355X MiniMax M3 启用 EAGLE3 - #2458
perf(agentx): enable MI355X MiniMax M3 EAGLE3 / perf(agentx)：为 MI355X MiniMax M3 启用 EAGLE3#2458cquil11 wants to merge 21 commits into
Conversation
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
e49338f to
94074fb
Compare
cquil11
changed the title
94074fb to
100b98b
Compare
|
Claude finished @cquil11's task in 1m 24s —— View job Review
LGTM - no blocking issues found Verified: the new entry is appended at the end of |
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
LGTM aside from the inline nit — this is a minor, low-risk config-only change.
Extended reasoning...
Overview
The PR adds a single new changelog entry to perf-changelog.yaml (lines 5359-5363), registering the minimaxm3-fp4-mi355x-vllm-agentic config for its initial MI355X MiniMax M3 submission via the new AgentX harness. No other files are touched. I confirmed this is genuinely the first changelog entry for this config key (only other reference is the config definition itself in configs/amd-master.yaml), so the 'Initial' description is accurate and the ruled-out mislabeling concern does not apply.
Security risks
None. This is a declarative YAML changelog entry with no code execution, credentials, or user input involved.
Level of scrutiny
Low. This follows the repo's well-established append-only changelog pattern (list of config-keys/description/pr-link entries) with no schema changes or logic modifications. The change is purely additive and mechanical.
Other factors
The one inline finding is a nit about the PR title/description missing the bilingual (English/Chinese) format mandated by AGENTS.md — a real, actionable process gap, but purely metadata and not something that affects correctness or functionality. Recent merged PRs (#2457, #2455) show this convention is not currently CI-enforced, so it's not a blocker here either. No outstanding unaddressed reviewer comments exist on this PR.
| - config-keys: | ||
| - minimaxm3-fp4-mi355x-vllm-agentic | ||
| description: | ||
| - "Initial MI355X MiniMax M3 with new AgentX harness" | ||
| pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2458 |
There was a problem hiding this comment.
🟡 This PR's title and description are English-only, but AGENTS.md mandates bilingual PR titles (<English title> / <中文标题>) and a ## 中文说明 section in the body for every PR / 本 PR 的标题和描述仅为英文，但 AGENTS.md 要求所有 PR 标题必须为双语格式（<英文标题> / <中文标题>），并在正文中包含 ## 中文说明 部分。请补充中文标题和说明以符合仓库规范。
Extended reasoning...
AGENTS.md line 7 (a checked-in, mandatory repository convention, not subjective style) states verbatim: "PR and GitHub-issue titles & descriptions must be bilingual — include a Simplified Chinese version in addition to English. Title format: <English title> / <中文标题>. In the PR/issue body, follow the English content with its Chinese translation (e.g. a ## 中文说明 section mirroring the summary...). ... This applies to every PR and every issue."
This PR's title is perf(agentx): refresh MI355X MiniMax M3 — there is no / <中文标题> suffix. Its body has only ## Summary and ## Validation sections, both in English, with no ## 中文说明 section translating the summary. The only documented exceptions in AGENTS.md are the CODEOWNER sign-off template (which must stay English-verbatim so the sign-off verifier bot can match it) and bot-generated comments following their own workflow templates — neither applies here, since this is a human-authored PR title and description.
Step-by-step proof:
- AGENTS.md:7 requires: title format
<English> / <中文>, and body with an English section followed by a## 中文说明mirror. - PR perf(agentx): enable MI355X MiniMax M3 EAGLE3 / perf(agentx)：为 MI355X MiniMax M3 启用 EAGLE3 #2458 title:
perf(agentx): refresh MI355X MiniMax M3→ no/separator, no Chinese text at all. - PR perf(agentx): enable MI355X MiniMax M3 EAGLE3 / perf(agentx)：为 MI355X MiniMax M3 启用 EAGLE3 #2458 body:
## Summary(3 bullets, English) +## Validation(2 bullets, English) → no## 中文说明section exists anywhere in the body. - Checking the stated exceptions: CODEOWNER sign-off template (not applicable — no sign-off here) and bot comment templates (not applicable — this is the PR author's own title/description). Neither exempts this PR.
- Conclusion: the PR violates the checked-in bilingual convention as written, with no applicable exception.
Why existing tooling doesn't catch this: there's no CI gate that lints PR titles/descriptions for a Chinese section — this convention is enforced only by human/agent review, not automation. That's corroborated by recent history: PR #2457 (fix(agentx): pin AIPerf v1 timing watchdog) and PR #2455 also merged with English-only titles, showing the rule is currently unenforced by any hard gate.
Impact of merging as-is: purely PR-metadata hygiene — the underlying YAML change (adding the MiniMax M3 config entry) is unaffected and nothing breaks functionally. Fix is trivial: the author (or a maintainer/bot) can edit the PR title to perf(agentx): refresh MI355X MiniMax M3 / perf(agentx)：更新 MI355X MiniMax M3 and append a ## 中文说明 section mirroring the Summary/Validation bullets.
Severity: since this causes no runtime failure, crash, or data loss, and AGENTS.md does not declare the bilingual rule a hard merge-blocker (nor is it CI-enforced), this is a nit — a real, actionable convention violation worth flagging, but not one that should hold up the merge.
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30777851549 |
…m3-agentx # Conflicts: # perf-changelog.yaml
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30777851549 |
Update the MI355X MiniMax M3 AgentX recipe to the latest published official vLLM ROCm nightly and record the image bump in the performance changelog. 中文：将 MI355X MiniMax M3 AgentX 配方更新至最新发布的官方 vLLM ROCm nightly 镜像，并在性能变更日志中记录此次镜像升级。
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30834201753 |
functionstackx
left a comment
functionstackx
left a comment
There was a problem hiding this comment.
@cquil11 can this be with EAGLE?
Route the MI355X AgentX recipe through EAGLE3-GQA with three speculative tokens. Pin throughput to the committed thinking-on golden acceptance length of 2.78 while retaining real verification for evals. 中文：为 MI355X AgentX 配方启用 EAGLE3-GQA，使用 3 个投机 token。吞吐量测试固定采用 thinking-on 黄金接受长度 2.78，评估任务继续使用真实目标模型校验。
cquil11
changed the title
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30843862008 |
|
Updated the MI355X recipe to EAGLE3-GQA with 3 speculative tokens and the committed thinking-on golden AL 2.78 for throughput; eval-only retains real verification. Replacement sweep: https://github.com/SemiAnalysisAI/InferenceX/actions/runs/30846281719 / 已将 MI355X 配方更新为 EAGLE3-GQA，使用 3 个投机 token；吞吐量测试采用已提交的 thinking-on 黄金接受长度 2.78，eval-only 保留真实校验。替代扫描：https://github.com/SemiAnalysisAI/InferenceX/actions/runs/30846281719 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30846281719 |
Use the Triton attention backend for the MiniMax-M3 EAGLE3-GQA draft so it shares the mandatory 128-token cache block size with the target model.\n\n中文：MI355X MiniMax-M3 EAGLE3-GQA 草稿模型改用 Triton 注意力后端，使其与目标模型共用必需的 128-token KV 缓存块大小。
Support both GPU-resident and native DRAM KV-cache modes through the shared AgentX validators. Keep prefix caching and thinking mode explicit, pin the official unshuffled ROCm KV layout, honor the launcher-selected port, and clean up the server process on every exit. 中文：通过共享 AgentX 校验函数支持 GPU 常驻和原生 DRAM KV 缓存两种模式。显式启用前缀缓存与 thinking 模式，固定使用官方 ROCm 非重排 KV 布局，沿用启动器分配的端口，并确保退出时清理服务进程。
Keep TP4 concurrency 1, 4, 8, and 16 GPU-resident, where the prior sweep showed ample KV headroom and negligible external-cache use. Reserve native DRAM KV offload for the extended concurrency 24 and 32 points. 中文：TP4 并发 1、4、8、16 保持 KV 缓存驻留 GPU；上一轮扫描显示这些点具有充足 KV 余量，外部缓存命中可忽略。仅在新增的并发 24、32 测试点启用原生 DRAM KV 卸载。
Use the supported vLLM router for DP-attention configurations. Map AIPerf correlation IDs to session IDs so every turn of a trajectory remains on the same DP rank, and include the router in normal service readiness and cleanup. 中文：DP-attention 配置使用受支持的 vLLM router。将 AIPerf correlation ID 映射为 session ID，确保同一轨迹的各轮请求固定路由至同一 DP rank，并将 router 纳入标准就绪检查与退出清理流程。
Retain the PR-specific performance changelog entry at the new tail while incorporating the current AgentX AIPerf submodule and repository changes. 中文：合并最新 main 分支，并将本 PR 的性能变更日志条目重新追加到文件末尾，同时纳入当前 AgentX AIPerf 子模块及仓库更新。
Add focused TEP8 c20 and DEP8 c24 GPU-resident candidates for throughput validation. Size DEP max-num-seqs per rank with AgentX handoff headroom instead of applying the global root concurrency to every rank. 中文：新增聚焦吞吐量验证的 TEP8 c20 与 DEP8 c24 GPU 常驻候选配置。DEP 的 max-num-seqs 按 rank 设置并为 AgentX handoff 预留余量，避免每个 rank 都按全局根轨迹并发度配置。
Round per-rank DEP max-num-seqs upward so future non-divisible concurrency values retain AgentX handoff headroom and never truncate the scheduler capacity. 中文：DEP 的每 rank max-num-seqs 改为向上取整，确保后续不可整除的并发值仍保留 AgentX handoff 余量，不会截断调度容量。
Reduce the MiniMax-M3 native KV offload pool from 80% to 20% of host DRAM. This keeps a meaningful 299 GB cache extension while avoiding the 1.2 TB pinned-memory registration failure observed on MI355X. 中文：将 MiniMax-M3 原生 KV 卸载池从主机内存的 80% 降至 20%。这仍可提供约 299 GB 的缓存扩展，同时避免 MI355X 上 1.2 TB 锁页内存注册失败。
Add one matched high-concurrency point for the validated TEP8 and DEP8 topologies so the final sweep can retain the better scaling path. 中文：为已验证的 TEP8 和 DEP8 拓扑增加一个 c28 同并发对比点，以便最终扫描保留扩展性更好的方案。
Add TP4 c12 and c20 without KV offload. The validated c16 point used about 58% of GPU KV capacity, so these points remain on the device-resident arm before native offload begins at c24. 中文：为 TP4 增加不启用 KV 卸载的 c12 和 c20。已验证的 c16 仅使用约 58% 的 GPU KV 容量，因此这些点继续使用显存驻留方案，并从 c24 开始启用原生卸载。
Add one isolated GPU-resident TEP8 stress point above c28 to identify the practical high-throughput limit without duplicating the TP4 offload or DEP arms. 中文：在 c28 之上增加一个独立的显存驻留 TEP8 压力测试点，用于确定实际高吞吐上限，同时避免重复 TP4 卸载或 DEP 方案。
Keep TP4 GPU-resident through c24, reserve native offload for c32, and retain TEP8 for higher throughput. Remove the slower DEP candidate and its unused router path after direct production comparison. 中文：TP4 在 c24 及以下保持显存驻留，仅在 c32 使用原生 KV 卸载，并保留 TEP8 作为更高吞吐方案。根据生产环境直接对比结果，移除较慢的 DEP 候选及其不再使用的路由路径。
Extend the supported TP4 native DRAM KV-offload band to concurrency 40 after the reduced 20% host allocation cleared server initialization and c32 profiling. 中文：在降低为 20% 的主机内存分配已通过服务启动和 c32 性能测试后，将受支持的 TP4 原生 DRAM KV 卸载范围扩展到并发 40。
Test TP4 concurrency 32 without offload after the native connector stored terabytes of KV but produced no cache loads at this point. Keep native DRAM offload for concurrency 40. 中文：原生连接器在 c32 写入数 TB KV 但没有产生缓存回读，因此改为验证 TP4 并发 32 的纯 GPU KV 配置，并仅在并发 40 使用原生 DRAM KV 卸载。
Move the final TP4 native-offload candidate to concurrency 28. Concurrency 32 was oversaturated: its TTFT coverage fell below the existing 98% validity gate and native offload produced no cache loads. 中文：将最终 TP4 原生 KV 卸载候选点调整为并发 28。并发 32 已过度饱和，其 TTFT 覆盖率低于现有 98% 有效性门槛，且原生卸载未产生缓存回读。
Add one upper TEP8 concurrency point after c36 sustained higher throughput with bounded TTFT and ample GPU KV headroom. 中文：c36 在 TTFT 可控且 GPU KV 余量充足的情况下保持更高吞吐量，因此新增一个 TEP8 并发 44 上限测试点。
2 participants
Summary
TRITON_ATTN, FP8 KV cache, AITER MoE, and the baseline unshuffled KV layout.Validation
中文说明
TRITON_ATTN、FP8 KV 缓存、AITER MoE，并固定使用基线非重排 KV 布局。验证