Repository navigation
[AgentX] offload all Muse Spark Open H100 points to HiCache / [AgentX] Muse Spark Open H100 全部并发点启用 HiCache 卸载 - #4182
Conversation
|
Thanks for the contribution!
中文感谢你的贡献！
|
Oseltamivir
added
the
non-canary-full-sweep-enabled
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
Besides the inline finding, I checked two other things and found no problems. In the recipe, all three zipped lists (max-running-requests, cuda-graph-max-bs-decode and CONC) have 4 entries, and they match the master conc-list [1, 2, 4, 8]. The perf-changelog change is a pure append at the end of the file, and no existing bytes changed.
Extended reasoning...
This PR changes one srt-slurm recipe, one master config entry and the perf-changelog for a single MuseSpark H100 agentic benchmark. It does not touch any security-sensitive code. The one confirmed finding (the placeholder pr-link fails the changelog validator) is posted inline, so this note only records the other checks that came back clean.
| description: | ||
| - "Run every point with HiCache host-DRAM KV offload at 160 GB per rank, sweeping concurrency 1, 2, 4 and 8; drop the GPU-resident arm and the concurrency 16 queueing point." | ||
| - "Keep the batch-8 cap on running requests and decode graphs." | ||
| pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/XXXX |
There was a problem hiding this comment.
🔴 The PR's changelog check will fail, which blocks the sweep for this change. The new entry's placeholder pull/XXXX has four X's, and the validator accepts only the exact pull/XXX form. Fix: set pr-link to https://github.com/SemiAnalysisAI/InferenceX/pull/XXX or to this PR's real number.
Why this was flagged
On pull requests, .github/workflows/run-sweep.yml:106 runs python -m infx.workflows.validate_perf_changelog. For each newly appended entry, validate_added_pr_link in inferencex-e2e/infx/workflows/validate_perf_changelog.py:147 only allows a link in PR_LINK_PLACEHOLDERS ("XXX" or https://github.com/SemiAnalysisAI/InferenceX/pull/XXX, lines 22-25) or the exact PR URL. The appended entry at perf-changelog.yaml:10024 uses .../pull/XXXX, which matches neither. The check raises "new PR entry must use ... or an XXX placeholder", so check-changelog fails and the sweep jobs that depend on it (needs: check-changelog) never run for this config change. The placeholder set is an exact-string match, so the extra X is not tolerated.
Verification: The trigger is merging this PR through the supported reuse merge path (merge_with_reuse). The pull/XXXX placeholder is rejected there. PR_LINK_PLACEHOLDERS (validate_perf_changelog.py:22-25) contains only "XXX" and ".../pull/XXX". merge_with_reuse.py:631 calls canonicalize_changelog, which reaches validate_perf_changelog.py:147 and raises.
|
View unofficial run (performance): https://inferencex.semianalysis.com/inference?unofficialRun=38925660934 View unofficial run (accuracy): https://inferencex.semianalysis.com/evaluation?unofficialRun=38925660934 |
0.95 shrank the GPU KV pool to 98k tokens and rejected ~196k-token trace inputs.
|
/reuse-sweep-run 38925660934 |
2 participants
Follow-up to #4149 and #4181, which merged with a HiCache arm at concurrency 2–8 alongside the GPU-resident c1–c16 sweep.
This makes HiCache host-DRAM KV offload the only arm and sweeps concurrency 1, 2, 4 and 8:
enable-hierarchical-cacheandhicache-size: 160GB per rank.[2, 4, 8, 8]), since eager batches above it OOM in the MoE.Why: at TP8 each H100 holds ~67 GB of target weights, leaving a GPU KV pool of a few GB, so agentic prefixes evict between turns; HiCache keeps them in host DRAM (~1.83 TiB on h100-dsxe).
Risk: HiCache's GPU staging buffers must fit in the ~1 GB/GPU headroom at mem-fraction-static 0.97.