Skip to content

perf(agentx): refresh MI355X MiniMax M3 / perf(agentx)：刷新 MI355X MiniMax M3 #2458

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
cquil11 wants to merge 2 commits into
base: main
Choose a base branch
from
+66 −21
Open

perf(agentx): refresh MI355X MiniMax M3 / perf(agentx)：刷新 MI355X MiniMax M3 #2458

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
6ebbf3d
perf(agentx): refresh MI355X MiniMax M3
cquil11 Aug 4, 2026
73e2f4f
chore(perf): record MI355X MiniMax M3 refresh
cquil11 Aug 4, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
71 changes: 54 additions & 17 deletions ...ngle_node/agentic/minimaxm3_fp4_mi355x.sh → .../agentic/minimaxm3_fp4_mi355x_vllm_mtp.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,11 @@
set -euo pipefail
set -x

# Agentic trace replay benchmark for Minimax-M3 FP4 on MI355X using vLLM.
# Agentic trace replay benchmark for MiniMax-M3 FP4 on MI355X using vLLM
# EAGLE3 speculative decoding.
#
# Required env vars:
# MODEL, MODEL_PATH, TP, CONC, KV_OFFLOADING, KV_OFFLOAD_BACKEND,
# MODEL, MODEL_PATH, TP, CONC, KV_OFFLOADING,
# TOTAL_CPU_DRAM_GB, RESULT_DIR, DURATION, EP_SIZE, DP_ATTENTION

source "$(dirname "$0")/../../benchmark_lib.sh"
Expand All @@ -17,11 +18,15 @@ source "$(dirname "$0")/../../benchmark_lib.sh"
# setting it here makes the effective framework always lm-eval, never swebench.
export EVAL_FRAMEWORK="lm-eval"

check_env_vars MODEL TP CONC KV_OFFLOADING KV_OFFLOAD_BACKEND TOTAL_CPU_DRAM_GB RESULT_DIR DURATION EP_SIZE DP_ATTENTION
check_env_vars MODEL TP CONC KV_OFFLOADING TOTAL_CPU_DRAM_GB RESULT_DIR DURATION EP_SIZE DP_ATTENTION

echo "MODEL=$MODEL TP=$TP CONC=$CONC KV_OFFLOADING=$KV_OFFLOADING TOTAL_CPU_DRAM_GB=$TOTAL_CPU_DRAM_GB RESULT_DIR=$RESULT_DIR DURATION=$DURATION EP_SIZE=$EP_SIZE DP_ATTENTION=$DP_ATTENTION"

PORT=8888
DRAFT_MODEL="Inferact/MiniMax-M3-EAGLE3-GQA"
NUM_SPEC_TOKENS=3
# golden_al_distribution/minimaxm3_eagle3_gqa.yaml:
# minimax-m3.thinking_on[3]
SYNTHETIC_ACCEPT_LEN=2.78

if [[ -n "${SLURM_JOB_ID+x}" ]]; then
echo "JOB $SLURM_JOB_ID running on $SLURMD_NODENAME"
Expand All @@ -40,6 +45,7 @@ else
hf download "$MODEL"
export MODEL_PATH="$MODEL"
fi
hf download "$DRAFT_MODEL"

rocm-smi || true
amd-smi || true
Expand All @@ -49,16 +55,33 @@ install_agentic_deps

# ---- Server config ----------------------------------------------------------
SERVER_LOG="$RESULT_DIR/server.log"
LMCACHE_LOG="$RESULT_DIR/lmcache_server.log"
mkdir -p "$RESULT_DIR"

OFFLOAD_ARGS=(--no-enable-prefix-caching)

case "$KV_OFFLOAD_BACKEND" in
vllm-simple)
SERVER_PID=""
cleanup_agentic_services() {
local exit_code=$?
trap - EXIT INT TERM
set +e
stop_background_process_tree "$SERVER_PID" "vLLM server" 60
exit "$exit_code"
}
trap cleanup_agentic_services EXIT
trap 'exit 130' INT
trap 'exit 143' TERM

# AgentX replays growing multi-turn prefixes, so keep prefix caching enabled
# for both GPU-resident and native-offload configurations.
OFFLOAD_ARGS=()

case "${KV_OFFLOAD_BACKEND:-}" in
"")
require_agentic_kv_offload_none
;;
vllm-native)
require_agentic_kv_offload_backend vllm-native
unset VLLM_USE_SIMPLE_KV_OFFLOAD
# Use vLLM's regular native KV-offload path (OffloadingConnector),
# NOT the SimpleCPUOffloadConnector. The "native" backend resolves to
# NOT the SimpleCPUOffloadConnector. The "vllm-native" backend resolves to
# OffloadingConnector by default; setting VLLM_USE_SIMPLE_KV_OFFLOAD=1
# would switch it to SimpleCPUOffloadConnector. We intentionally leave
# that env var UNSET here so the regular OffloadingConnector path is
Expand All @@ -73,28 +96,39 @@ case "$KV_OFFLOAD_BACKEND" in
--kv_offloading_size "$TOTAL_CPU_DRAM_GB"
)
;;
*)
echo "Unsupported KV_OFFLOAD_BACKEND: ${KV_OFFLOAD_BACKEND:-}" >&2
exit 1
;;
esac

# ---- LLM server config ----------------------------------------------------------
PARALLEL_ARGS=(--tensor-parallel-size "$TP")
if [ "${DP_ATTENTION}" = "true" ]; then
PARALLEL_ARGS=(
--tensor-parallel-size 1
--data-parallel-size "$TP"
--enable-expert-parallel
)
elif [ "$EP_SIZE" -gt 1 ]; then
if [ "$EP_SIZE" -gt 1 ]; then
PARALLEL_ARGS+=(--enable-expert-parallel)
fi

# Synthetic acceptance standardizes throughput against the committed golden
# EAGLE3-GQA curve. Accuracy evals must use real target verification.
if [ "${EVAL_ONLY}" = "true" ]; then
SPEC_CONFIG="{\"method\": \"eagle3\", \"model\": \"$DRAFT_MODEL\", \"num_speculative_tokens\": $NUM_SPEC_TOKENS, \"attention_backend\": \"TRITON_ATTN\"}"
else
SPEC_CONFIG="{\"method\": \"eagle3\", \"model\": \"$DRAFT_MODEL\", \"num_speculative_tokens\": $NUM_SPEC_TOKENS, \"attention_backend\": \"TRITON_ATTN\", \"rejection_sample_method\": \"synthetic\", \"synthetic_acceptance_length\": $SYNTHETIC_ACCEPT_LEN}"
fi

echo "Starting vllm server..."
export PYTHONNOUSERSITE=1

export VLLM_ENGINE_READY_TIMEOUT_S=3600
export VLLM_EXECUTE_MODEL_TIMEOUT_SECONDS=1800
export VLLM_USE_BREAKABLE_CUDAGRAPH=0
export VLLM_ROCM_USE_AITER=1
export VLLM_ROCM_USE_AITER_MOE=1
export VLLM_ROCM_USE_AITER_FUSION_SHARED_EXPERTS=1
# The official MiniMax-M3 MI355X EAGLE3 baseline uses the native page-128
# layout. The shuffled AITER page-16 path is a separate high-concurrency mode
# and must only be enabled after that band is validated independently.
export VLLM_ROCM_SHUFFLE_KV_CACHE_LAYOUT=0
# INT4 quantized all-reduce for the (~1.5 MB) decode all-reduces, which are the
# single biggest decode kernel at high concurrency. The MIN_SIZE_KB override is
# required: vLLM's default INT4 quick-reduce size gate for (bf16, TP4) is 16 MB,
Expand All @@ -113,11 +147,13 @@ VLLM_CMD=(
--block-size 128
--gpu-memory-utilization 0.85
--language-model-only
--enable-prefix-caching
--attention-backend TRITON_ATTN
--moe-backend aiter
--kv-cache-dtype fp8
--tool-call-parser minimax_m3
--enable-auto-tool-choice
--default-chat-template-kwargs '{"thinking_mode":"enabled"}'
# NOTE: --reasoning-parser minimax_m3 is intentionally OMITTED.
# MiniMax-M3 is an interleaved-thinking model: its <mm:think>...</mm:think>
# block MUST be round-tripped back into the conversation history every turn
Expand All @@ -129,6 +165,7 @@ VLLM_CMD=(
# inline in message.content, so the client preserves it across turns. The
# tool-call parser above still extracts tool calls from the full output.
--max-num-seqs "$CONC"
--speculative-config "$SPEC_CONFIG"
"${OFFLOAD_ARGS[@]}"
)
printf '%q ' "${VLLM_CMD[@]}" | tee "$RESULT_DIR/vllm_command.txt"
Expand Down
10 changes: 6 additions & 4 deletions configs/amd-master.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2231,8 +2231,8 @@ minimaxm3-fp8-mi325x-vllm-agentic:
- { tp: 8, ep: 8, kv-offloading: dram, kv-offload-backend: { name: mooncake, version: "0.3.11.post1" }, conc-list: [10, 12, 14, 16, 18, 20, 22, 24, 26, 28, 32] }
- { tp: 8, ep: 8, dp-attn: true, kv-offloading: dram, kv-offload-backend: { name: mooncake, version: "0.3.11.post1" }, conc-list: [24, 32, 36, 40, 44, 48, 52, 56, 60, 64, 72, 80, 96], router: { name: vllm-router, version: "0.1.14" } }

minimaxm3-fp4-mi355x-vllm-agentic:
image: vllm/vllm-openai-rocm:nightly-dcfebf93f4eccf30f71872283331eee757915daf
minimaxm3-fp4-mi355x-vllm-agentic-mtp:
image: vllm/vllm-openai-rocm:nightly-5e35a6f4f9bbc217c599692157ca985c894373f7
model: amd/MiniMax-M3-MXFP4
model-prefix: minimaxm3
runner: cluster:mi355x-amds
Expand All @@ -2241,9 +2241,11 @@ minimaxm3-fp4-mi355x-vllm-agentic:
multinode: false
scenarios:
agentic-coding:
- dram-utilization: 0.80
- dram-utilization: 0.20
search-space:
- { tp: 4, kv-offloading: dram, kv-offload-backend: { name: vllm-simple }, conc-list: [1, 4, 8, 16] }
- { tp: 4, spec-decoding: mtp, kv-offloading: none, conc-list: [1, 4, 8, 12, 16, 20, 24] }
- { tp: 4, spec-decoding: mtp, kv-offloading: dram, kv-offload-backend: { name: vllm-native }, conc-list: [28] }
- { tp: 8, ep: 8, spec-decoding: mtp, kv-offloading: none, conc-list: [20, 28, 36] }

dsv4-fp4-mi355x-sglang-disagg-agentic-hicache:
image: lmsysorg/sglang-rocm:v0.5.14-rocm720-mi35x-20260710
Expand Down
6 changes: 6 additions & 0 deletions perf-changelog.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5425,3 +5425,9 @@
description:
- "Extend the search space to include the TP2EP1 configuration for fixed seq len 8k1k"
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2446

- config-keys:
- minimaxm3-fp4-mi355x-vllm-agentic-mtp
description:
- "Initial MI355X MiniMax M3 with new AgentX harness"
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2458
Comment on lines +5429 to +5433

@claude claude Bot Aug 3, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 This PR's title and description are English-only, but AGENTS.md mandates bilingual PR titles (<English title> / <中文标题>) and a ## 中文说明 section in the body for every PR / 本 PR 的标题和描述仅为英文，但 AGENTS.md 要求所有 PR 标题必须为双语格式（<英文标题> / <中文标题>），并在正文中包含 ## 中文说明 部分。请补充中文标题和说明以符合仓库规范。

Extended reasoning...

AGENTS.md line 7 (a checked-in, mandatory repository convention, not subjective style) states verbatim: "PR and GitHub-issue titles & descriptions must be bilingual — include a Simplified Chinese version in addition to English. Title format: <English title> / <中文标题>. In the PR/issue body, follow the English content with its Chinese translation (e.g. a ## 中文说明 section mirroring the summary...). ... This applies to every PR and every issue."

This PR's title is perf(agentx): refresh MI355X MiniMax M3 — there is no / <中文标题> suffix. Its body has only ## Summary and ## Validation sections, both in English, with no ## 中文说明 section translating the summary. The only documented exceptions in AGENTS.md are the CODEOWNER sign-off template (which must stay English-verbatim so the sign-off verifier bot can match it) and bot-generated comments following their own workflow templates — neither applies here, since this is a human-authored PR title and description.

Step-by-step proof:

  1. AGENTS.md:7 requires: title format <English> / <中文>, and body with an English section followed by a ## 中文说明 mirror.
  2. PR perf(agentx): refresh MI355X MiniMax M3 / perf(agentx)：刷新 MI355X MiniMax M3 #2458 title: perf(agentx): refresh MI355X MiniMax M3 → no / separator, no Chinese text at all.
  3. PR perf(agentx): refresh MI355X MiniMax M3 / perf(agentx)：刷新 MI355X MiniMax M3 #2458 body: ## Summary (3 bullets, English) + ## Validation (2 bullets, English) → no ## 中文说明 section exists anywhere in the body.
  4. Checking the stated exceptions: CODEOWNER sign-off template (not applicable — no sign-off here) and bot comment templates (not applicable — this is the PR author's own title/description). Neither exempts this PR.
  5. Conclusion: the PR violates the checked-in bilingual convention as written, with no applicable exception.

Why existing tooling doesn't catch this: there's no CI gate that lints PR titles/descriptions for a Chinese section — this convention is enforced only by human/agent review, not automation. That's corroborated by recent history: PR #2457 (fix(agentx): pin AIPerf v1 timing watchdog) and PR #2455 also merged with English-only titles, showing the rule is currently unenforced by any hard gate.

Impact of merging as-is: purely PR-metadata hygiene — the underlying YAML change (adding the MiniMax M3 config entry) is unaffected and nothing breaks functionally. Fix is trivial: the author (or a maintainer/bot) can edit the PR title to perf(agentx): refresh MI355X MiniMax M3 / perf(agentx)：更新 MI355X MiniMax M3 and append a ## 中文说明 section mirroring the Summary/Validation bullets.

Severity: since this causes no runtime failure, crash, or data loss, and AGENTS.md does not declare the bilingual rule a hard merge-blocker (nor is it CI-enforced), this is a nit — a real, actionable convention violation worth flagging, but not one that should hold up the merge.

Loading