[None][test] Enable gen_only + ctx_only DeepSeek-V4-Pro perf-sanity on GB300

[chenfeiz0326]

Summary

Follow-up to #16540. That PR added the four DeepSeek-V4-Pro (FP4, 1.6T MoE) disaggregated perf-sanity shapes on GB300 (con8 / con180 / con666 / con4301) but registered only the e2e variant of each — the gen_only and ctx_only variants were left commented out in the test-db yamls:

• gen_only — deferred pending gen_only runtime characterization on the target CI cluster.
• ctx_only — deferred behind a shared-stage OOM-cascade investigation (the con4301 ctx_only case passed cleanly in isolation on OCI-AGA, so the leak was judged orthogonal to these configs).

This PR enables both remaining modes so all three test modes (aggr_upload-ctx_only-*, disagg_upload-gen_only-*, disagg_upload-e2e-*) run for every shape.

Why this needs no config/model changes

The four disagg config yamls, the deepseek_v4_pro_fp4 → DeepSeek-V4-Pro model registration, the role-specific ctx_worker_env_var / gen_worker_env_var handling, and the in-repo MoE load-balancer yamls all landed in #16540 and are unchanged here. parse_test_string() derives benchmark_mode from the test-ID prefix and resolves the same on-disk config for all three modes (only the launched worker fleet differs), so enabling gen_only/ctx_only is purely a test-list registration change.

Changes

File	Change
l0_gb300_multi_gpus_perf_sanity.yml	Uncomment the 4 aggr_upload-ctx_only-* DeepSeek-V4-Pro entries (8 → 12 active ctx_only tests)
l0_gb300_multi_nodes_perf_sanity_ctx1_node1_gpu4_gen4_node2_gpu8.yml (con8)	Uncomment the gen_only entry (now 1 gen_only + 1 e2e)
l0_gb300_multi_nodes_perf_sanity_ctx6_node1_gpu4_gen1_node4_gpu16.yml (con666)	Uncomment the gen_only entry
l0_gb300_multi_nodes_perf_sanity_ctx3_node1_gpu4_gen1_node8_gpu32.yml (con180)	Uncomment the gen_only entry
l0_gb300_multi_nodes_perf_sanity_ctx12_node1_gpu4_gen1_node2_gpu8.yml (con4301)	Uncomment the gen_only entry
jenkins/L0_Test.groovy	GB300-4_GPUs PerfSanity-Post-Merge split 2 → 3 (keeps ~4 tests/shard for the heavier ctx_only set, giving the DSv4-Pro cases more per-node isolation given the #16540 shared-stage OOM note); the 4 GB300 DeepSeek-V4-Pro multi_nodes Disagg stages testCount 1 → 2 (gen_only + e2e)

Validation

• All 5 edited test-db yamls parse as valid YAML; each multi_nodes file holds exactly 1 gen_only + 1 e2e entry, matching testCount=2.
• All 12 registered DeepSeek-V4-Pro test IDs resolve to an existing config file under tests/scripts/perf-sanity/disaggregated/.
• multi_gpus ctx_only count (12) vs split (3) → ~4 tests/shard.

Test plan

• gen_only + ctx_only local functional runs on GB300 (aws-cmh)
• CI post-merge stages green (gen_only/ctx_only)

Note (carried from #16540): slurm_install.sh still hard-codes --cuda_architectures '100-real' (GB200); GB300 source builds need 103-real. Not addressed here.

Dev Engineer Review

• Enabled four DeepSeek-V4-Pro FP4 ctx_only tests and four multi-node gen_only tests.
• Increased GB300 4-GPU sharding from 2 to 3 and raised the four relevant multi-node stage testCount values from 1 to 2.
• Changes are limited to intended test coverage and scheduling configuration; existing model, worker, and load-balancer settings remain unchanged.
• YAML parsing and test-ID/config resolution were validated successfully. No duplicates or formatting issues were identified.

QA Engineer Review

Modified test-list files:

• l0_gb300_multi_gpus_perf_sanity.yml: enabled four ctx_only entries.
• l0_gb300_multi_nodes_perf_sanity_ctx12_node1_gpu4_gen1_node2_gpu8.yml: enabled one gen_only entry.
• l0_gb300_multi_nodes_perf_sanity_ctx1_node1_gpu4_gen4_node2_gpu8.yml: enabled one gen_only entry.
• l0_gb300_multi_nodes_perf_sanity_ctx3_node1_gpu4_gen1_node8_gpu32.yml: enabled one gen_only entry.
• l0_gb300_multi_nodes_perf_sanity_ctx6_node1_gpu4_gen1_node4_gpu16.yml: enabled one gen_only entry.

The entries use valid existing test configurations and preserve the corresponding timeout settings. Multi-GPU distribution was validated at approximately four tests per shard.

Verdict: needs follow-up — CBTS coverage data and local/CI execution results are pending.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 68264744-2890-410a-a50a-0cf4c610ca22

📥 Commits

Reviewing files that changed from the base of the PR and between 96360f8 and 544d14c.

📒 Files selected for processing (6)

• jenkins/L0_Test.groovy
• tests/integration/test_lists/test-db/l0_gb300_multi_gpus_perf_sanity.yml
• tests/integration/test_lists/test-db/l0_gb300_multi_nodes_perf_sanity_ctx12_node1_gpu4_gen1_node2_gpu8.yml
• tests/integration/test_lists/test-db/l0_gb300_multi_nodes_perf_sanity_ctx1_node1_gpu4_gen4_node2_gpu8.yml
• tests/integration/test_lists/test-db/l0_gb300_multi_nodes_perf_sanity_ctx3_node1_gpu4_gen1_node8_gpu32.yml
• tests/integration/test_lists/test-db/l0_gb300_multi_nodes_perf_sanity_ctx6_node1_gpu4_gen1_node4_gpu16.yml

---

Walkthrough

GB300 PerfSanity CI configuration now includes a third PyTorch split and generates two active tests per DeepSeek-V4-Pro disaggregated stage. Five YAML test lists enable previously commented-out gen_only or ctx_only cases.

Changes

GB300 PerfSanity coverage

Layer / File(s)	Summary
Expand PyTorch split coverage jenkins/L0_Test.groovy	Adds the third GB300 PyTorch PerfSanity split and updates the total split count from 2 to 3.
Enable disaggregated test coverage jenkins/L0_Test.groovy, tests/integration/test_lists/test-db/*.yml	Updates GB300 disaggregated stage generation from one to two tests per shape and activates the corresponding gen_only and ctx_only YAML entries.

Estimated code review effort: 2 (Simple) | ~10 minutes

Possibly related PRs

• NVIDIA/TensorRT-LLM#9138: Adds the multi-node disaggregated PerfSanity CI path and related configurations.

Suggested reviewers: niukuo, mzweilz

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title is concise and accurately summarizes enabling gen_only and ctx_only perf-sanity on GB300.
Description check	✅ Passed	The description covers the summary, rationale, changes, validation, and test plan, matching the template well.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 Checkov (3.3.8)

tests/integration/test_lists/test-db/l0_gb300_multi_gpus_perf_sanity.yml

Traceback (most recent call last):
File "/usr/local/bin/checkov", line 2, in
from checkov.main import Checkov
ModuleNotFoundError: No module named 'checkov'

tests/integration/test_lists/test-db/l0_gb300_multi_nodes_perf_sanity_ctx12_node1_gpu4_gen1_node2_gpu8.yml

Traceback (most recent call last):
File "/usr/local/bin/checkov", line 2, in
from checkov.main import Checkov
ModuleNotFoundError: No module named 'checkov'

tests/integration/test_lists/test-db/l0_gb300_multi_nodes_perf_sanity_ctx1_node1_gpu4_gen4_node2_gpu8.yml

Traceback (most recent call last):
File "/usr/local/bin/checkov", line 2, in
from checkov.main import Checkov
ModuleNotFoundError: No module named 'checkov'

• 2 others

---

_{Comment @coderabbitai help to get the list of available commands.}