Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -56,9 +56,9 @@ zip_override_tp4_ep1:
agg:
gpus: 4
args:
cuda-graph-max-bs-decode: [2, 4, 8, 16, 32, 64, 128, 256, 512, 320, 384, 448, 640]
scheduler-recv-interval: [10, 10, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30]
cuda-graph-max-bs-decode: [1, 2, 4, 8, 16, 32, 64, 128, 256, 512, 320, 384, 448, 640]
scheduler-recv-interval: [10, 10, 10, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30, 30]
tensor-parallel-size: 4
benchmark:
env:
CONC: ['2', '4', '8', '16', '32', '64', '128', '256', '512', '320', '384', '448', '640']
CONC: ['1', '2', '4', '8', '16', '32', '64', '128', '256', '512', '320', '384', '448', '640']
2 changes: 1 addition & 1 deletion inferencex-e2e/configs/nvidia-master.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -1142,7 +1142,7 @@ qwen3.5-fp8-b200-sglang:
srt-recipe: 8k1k.yaml
- tp: 4
ep: 1
conc-start: 2
conc-start: 1
conc-end: 512
srt-recipe: 8k1k.yaml
- tp: 4
Expand Down
19 changes: 19 additions & 0 deletions inferencex-e2e/configs/powerx-article-b200.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
qwen3.5-fp8-b200-sglang:
image: lmsysorg/sglang:nightly-dev-cu13-20260918-20518d85
model: Qwen/Qwen3.5-397B-A17B-FP8
model-prefix: qwen3.5
runner: cluster:b200-nscale
precision: fp8
framework: sglang
multinode: false
srt-recipe-dir: qwen3.5/sglang/b200-fp8
scenarios:
fixed-seq-len:
- isl: 8192
osl: 1024
search-space:
- tp: 4
ep: 1
conc-start: 1
conc-end: 128
srt-recipe: 8k1k.yaml
8 changes: 8 additions & 0 deletions inferencex-e2e/perf-changelog.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -10052,3 +10052,11 @@
- "Offload every Muse Spark Open H200 point to host DRAM with SGLang HiCache (120 GB per rank so the target and DFlash draft host pools fit the node cgroup, dram-utilization 0.80), keeping the concurrency 1-64 sweep."
- "Keep the GPU-resident settings unchanged: mem-fraction-static 0.85, matching running-request and decode-graph caps."
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/4199

- config-keys:
- qwen3.5-fp8-b200-sglang
scenario-type:
- fixed-seq-len
description:
- "Add TP4/EP1 concurrency 1 to the Qwen3.5 FP8 B200 SGLang 8k1k recipe zip_override (cuda-graph-max-bs-decode 1, scheduler-recv-interval 10) so the c1 matrix point matches, and add the configs/powerx-article-b200.yaml selector for TP4/EP1 c1-c128 article repeats. The reused sweep measured and validated GPU power at every point."
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/3378
Comment thread
edwingao28 marked this conversation as resolved.
Loading