Repository navigation
[AgentX] offload Muse Spark Open H200 to HiCache / [AgentX] Muse Spark Open H200 全部并发启用 HiCache 卸载 - #4199
Conversation
|
Thanks for the contribution!
中文感谢你的贡献！
|
Oseltamivir
added
the
non-canary-full-sweep-enabled
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
| description: | ||
| - "Offload every Muse Spark Open H200 point to host DRAM with SGLang HiCache (160 GB per rank, dram-utilization 0.80), keeping the concurrency 1-64 sweep." | ||
| - "Keep the GPU-resident settings unchanged: mem-fraction-static 0.85, matching running-request and decode-graph caps." | ||
| pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/XXXX |
There was a problem hiding this comment.
🔴 This PR's changelog check will fail, and the merge-prep step will raise, because the new entry's placeholder link is pull/XXXX (four X's). The validator only accepts XXX or .../pull/XXX (three X's) at PR_LINK_PLACEHOLDERS in validate_perf_changelog.py:22-25. It also accepts the real .../pull/<number>, but XXXX matches none of these. Fix: change the line to pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/XXX or to the real PR number.
Why this was flagged
The new block appended at inferencex-e2e/perf-changelog.yaml:10045 has pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/XXXX. On a PR run, validate_added_pr_link (inferencex-e2e/infx/workflows/validate_perf_changelog.py:140-150) raises "new PR entry must use ... or an XXX placeholder". It does so because the link is neither in PR_LINK_PLACEHOLDERS (only "XXX" and ".../pull/XXX") nor equal to .../pull/{pr_number}. At merge time, prepare_perf_changelog_merge.py:108-111 raises "appended entry N has unexpected pr-link" for the same reason instead of rewriting the placeholder to the canonical link. On the base branch, both existing placeholder entries (lines 5598 and 9727) use the accepted three-X form. The documented template in CONTRIBUTING.md:206 also uses pull/XXX. Nothing normalizes XXXX, so the changelog gate blocks this PR and the benchmark rerun it triggers.
Verification: The new entry in inferencex-e2e/perf-changelog.yaml ends with pull/XXXX, which is not in PR_LINK_PLACEHOLDERS (validate_perf_changelog.py:22-25). merge_with_reuse.py:631 calls canonicalize_changelog(pr, repo), which raises ChangelogValidationError before the placeholder is swapped. The PR-run check (run-sweep.yml:97-117) never calls compare_entries, so it passes.
| enable-hierarchical-cache: true | ||
| hicache-size: 160 |
There was a problem hiding this comment.
🟡 (optional) Every H200 point now pins about 1280 GB of host KV pools. The DRAM budget this config declares is 1234 GB. h200-dgxc is the only Muse Spark HiCache runner whose pools exceed the budget. hicache-size: 160 is allocated per TP rank, so TP8 gives 8 x 160 = 1280 GB. Matrix generation gives total-cpu-dram-gb = 1,471,356 MiB x 0.80 = 1234 GB. Fix: keep 8 x hicache-size within the budget, e.g. hicache-size: 150 (1200 GB, cap 154). Or raise dram-utilization in nvidia-master.yaml to at least 0.83 on purpose.
Why this was flagged
Each c1-c64 point of musespark-fp4-h200-sglang-agentic-dflash now selects zip_override_tp8_hicache. That override starts SGLang with enable-hierarchical-cache and hicache-size 160, in GB, for each of the 8 TP ranks, so about 1280 GB of pinned host memory. The runner is cluster:h200-dgxc, and runners.yaml:392 lists available-cpu-dram-mib 1_471_356 (about 1543 GB). infx/matrix/generate.py:516-519 multiplies that by dram-utilization 0.80, giving TOTAL_CPU_DRAM_GB = 1234. That value goes to the run and is recorded as allocated_cpu_dram_gb (infx/results/agentic/init.py:192). The pools therefore use about 83% of the node, not the declared 80%. On the base branch these points used kv-offloading none and pinned no host pool. No validator checks hicache-size x TP against TOTAL_CPU_DRAM_GB, because the recipe does not set TOTAL_CPU_DRAM_GB (single_node.py:138).
Verification: nit. agentic.yaml:68-69 sets hicache-size: 160 with tensor-parallel-size: 8 (line 25), so the pools total 8 x 160 = 1280 GB. generate.py:516-519 computes 1,471,356 MiB x 0.80 = 1234 GB for h200-dgxc. That value is published as allocated_cpu_dram_gb (results/agentic/init.py:192). The recorded budget therefore understates the real host pinning by 46 GB.
|
View unofficial run (performance): https://inferencex.semianalysis.com/inference?unofficialRun=38941442268 View unofficial run (accuracy): https://inferencex.semianalysis.com/evaluation?unofficialRun=38941442268 |
|
/reuse-sweep-run 38941442268 |
# Conflicts: # inferencex-e2e/perf-changelog.yaml
2 participants
Moves every point of the Muse Spark Open H200 sweep (c1–c64) to a single SGLang HiCache arm with host-DRAM KV offload (160 GB per rank, dram-utilization 0.80). The GPU-resident settings stay the same: mem-fraction-static 0.85 and the existing running-request / decode-graph caps. This mirrors #4182 for H100.
H200 has about 20 GB of free GPU memory at 0.85, so the H100 HiCache out-of-memory problem shouldn't recur here.