[None][perf] Preserve default V2 KV cache pool sizing

[2ez4bz]

Dev Engineer Review

• KV cache warmup sizing logic (V2): Updated KVCacheManagerV2._build_base_config so allocator fallback remains the default: typical_step and warmup constraints are only constructed when kv_cache_config.avg_seq_len is explicitly set (i.e., avg_seq_len gates the warmup constraint path). The avg_seq_len > max_seq_len validation is now conditional on avg_seq_len being provided.
• DeepSeek-V4 compatibility: Added DeepseekV4CacheManager._get_typical_seq_len(...) override to preserve DeepSeek-V4’s prior behavior by using kv_cache_config.avg_seq_len when set, otherwise falling back to self.max_seq_len.
• Gemma4 SWA/pool regression coverage:
  • Added new Gemma4 (26B) B200 performance-sanity test config: tests/scripts/perf-sanity/aggregated/gemma4_26b_a4b_nvfp4_blackwell.yaml.
  • Extended model-path mapping with gemma_4_26b_a4b_nvfp4 in tests/integration/defs/perf/_model_paths.py.
  • Added CI stage plumbing for single-GPU B200 PerfSanity in jenkins/L0_Test.groovy:
    • DGX_B200-PyTorch-PerfSanity-1 (pre-merge)
    • DGX_B200-PyTorch-PerfSanity-Post-Merge-1 (post-merge)
    • Both reference l0_b200_perf_sanity on auto:dgx-b200-flex with gpuCount=1.
• Tests/fixtures: Unit tests updated to reflect the new warmup gating semantics, and a DeepSeek-V4 unit test validates fallback behavior.

QA Engineer Review

Test files changed

• tests/integration/test_lists/test-db/l0_b200_perf_sanity.yml
  • Added coverage for Gemma4 test_e2e[aggr_upload-gemma4_26b_a4b_nvfp4_blackwell-gemma4_26b_a4b_nvfp4_tp1_1k1k] under:
    • stage: pre_merge
    • stage: post_merge
  • Format/structure appears consistent (single-GPU *b200*, ubuntu*, x86_64, backend pytorch, orchestrator mpi, TIMEOUT (60)).

Test code changed (unit tests)

• tests/unittest/_torch/executor/test_kv_cache_manager_v2.py
  • Added: test_default_uses_allocator_fallback
  • Added: test_avg_seq_len_builds_warmup_constraints
  • Modified: test_extra_tokens_are_in_context_capacity (now uses KvCacheConfig(avg_seq_len=264) explicitly)
  • (Replaced old warmup test per diff summary: test_builds_warmup_constraints → test_default_uses_allocator_fallback)
• tests/unittest/_torch/attention/sparse/deepseek_v4/test_deepseek_v4_cache_manager.py
  • Added: test_typical_seq_len_preserves_deepseek_v4_fallback

Coverage linkage to integration/test-db

• The unit tests above are not directly referenced in tests/integration/test_lists/test-db/; the end-to-end Gemma4 regression is exercised via l0_b200_perf_sanity.yml.

Verdict: needs follow-up (CBTS coverage availability not provided in the gathered context).

Description

• Why?

Using max_seq_len as the default typical sequence length over-
allocates Gemma4's full-attention pool and exhausts its SWA pool.
This prevents CUDA graph padding from allocating dummy requests
and forces eager decode, leading to hugely degraded performance.

• What?

Only derive generic warmup constraints from an explicitly configured
avg_seq_len, preserving allocator-derived pool sizing by default.
Keep DeepSeek-V4's max-length fallback in its specialized cache manager.

This restores the behavior for Gemma4 prior to 697738c1f6 introducing
the new sizing logic for DeepSeek-V4, and adds an E2E perf regression test
for it.

Test Coverage

PR Checklist

Please review the following before submitting your PR:

• PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.

• PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.

• Test cases are provided for new code paths (see test instructions)

• If PR introduces API changes, an appropriate PR label is added - either api-compatible or api-breaking. For api-breaking, include BREAKING in the PR title.

• Any new dependencies have been scanned for license and vulnerabilities

• CODEOWNERS updated if ownership changes

• Documentation updated as needed

• Update tava architecture diagram if there is a significant design change in PR.

• The reviewers assigned automatically/manually are appropriate for the PR.

• Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see a list of available CI bot commands, please comment /bot help.

[2ez4bz]

/bot run

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: c61634e0-a829-41b1-907a-4c3c4b33b823

📥 Commits

Reviewing files that changed from the base of the PR and between 0b93b6f and 291449c.

📒 Files selected for processing (8)

• jenkins/L0_Test.groovy
• tensorrt_llm/_torch/attention_backend/sparse/deepseek_v4/cache_manager.py
• tensorrt_llm/_torch/pyexecutor/kv_cache_manager_v2.py
• tests/integration/defs/perf/_model_paths.py
• tests/integration/test_lists/test-db/l0_b200_perf_sanity.yml
• tests/scripts/perf-sanity/aggregated/gemma4_26b_a4b_nvfp4_blackwell.yaml
• tests/unittest/_torch/attention/sparse/deepseek_v4/test_deepseek_v4_cache_manager.py
• tests/unittest/_torch/executor/test_kv_cache_manager_v2.py

🚧 Files skipped from review as they are similar to previous changes (6)

• tests/integration/test_lists/test-db/l0_b200_perf_sanity.yml
• tests/unittest/_torch/attention/sparse/deepseek_v4/test_deepseek_v4_cache_manager.py
• tests/integration/defs/perf/_model_paths.py
• tensorrt_llm/_torch/attention_backend/sparse/deepseek_v4/cache_manager.py
• jenkins/L0_Test.groovy
• tests/unittest/_torch/executor/test_kv_cache_manager_v2.py

---

Walkthrough

The PR makes KV-cache warmup configuration conditional on an explicitly provided average sequence length, preserves DeepSeek-V4 fallback behavior, and adds Gemma 4 NVFP4 B200 performance-sanity coverage for pre-merge and post-merge CI runs.

Changes

KV cache typical sequence handling

Layer / File(s)	Summary
Conditional KV cache warmup configuration tensorrt_llm/_torch/pyexecutor/kv_cache_manager_v2.py, tests/unittest/_torch/executor/test_kv_cache_manager_v2.py	KV-cache warmup constraints are built only when avg_seq_len is configured, with tests covering default and explicit configurations.
DeepSeek-V4 fallback preservation tensorrt_llm/_torch/attention_backend/sparse/deepseek_v4/cache_manager.py, tests/unittest/_torch/attention/sparse/deepseek_v4/test_deepseek_v4_cache_manager.py	DeepSeek-V4 uses avg_seq_len when provided and otherwise falls back to max_seq_len, with parametrized test coverage.

B200 performance-sanity coverage

Layer / File(s)	Summary
Gemma 4 NVFP4 performance configuration tests/integration/defs/perf/_model_paths.py, tests/scripts/perf-sanity/aggregated/*	Adds the Gemma 4 NVFP4 model path and its B200 server/client performance-sanity configuration.
B200 PerfSanity CI wiring tests/integration/test_lists/test-db/l0_b200_perf_sanity.yml, jenkins/L0_Test.groovy	Adds single-GPU B200 pre-merge and post-merge performance-sanity test definitions and Jenkins stages.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Possibly related PRs

• NVIDIA/TensorRT-LLM#10598: Modifies overlapping PerfSanity stage configuration in jenkins/L0_Test.groovy.
• NVIDIA/TensorRT-LLM#16725: Consumes the new B200 PerfSanity stage names when selecting the SLURM job environment.

Suggested reviewers: tburt-nv, longlee0622

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 12.50% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title is concise and accurately summarizes the main change to default V2 KV cache pool sizing.
Description check	✅ Passed	The description explains the problem, solution, and context well, but the Test Coverage section is left empty.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@jenkins/L0_Test.groovy`:
- Around line 4896-4898: Update the perfMode predicate near the stage
configuration logic to recognize stage keys containing "-PerfSanity-" in
addition to "-Perf-". Ensure both DGX_B200-PyTorch-PerfSanity-1 and
DGX_B200-PyTorch-PerfSanity-Post-Merge-1 receive performance mode and run with
--perf.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 591c9744-f352-4e46-a3fd-399cb1acaf56

📥 Commits

Reviewing files that changed from the base of the PR and between c6e0c98 and 0b93b6f.

📒 Files selected for processing (8)

• jenkins/L0_Test.groovy
• tensorrt_llm/_torch/attention_backend/sparse/deepseek_v4/cache_manager.py
• tensorrt_llm/_torch/pyexecutor/kv_cache_manager_v2.py
• tests/integration/defs/perf/_model_paths.py
• tests/integration/test_lists/test-db/l0_b200_perf_sanity.yml
• tests/scripts/perf-sanity/aggregated/gemma4_26b_a4b_nvfp4_blackwell.yaml
• tests/unittest/_torch/attention/sparse/deepseek_v4/test_deepseek_v4_cache_manager.py
• tests/unittest/_torch/executor/test_kv_cache_manager_v2.py

[tensorrt-cicd]

PR_Github #61222 [ run ] triggered by Bot. Commit: 0b93b6f Link to invocation

[2ez4bz]

/bot run

[tensorrt-cicd]

PR_Github #61224 [ run ] triggered by Bot. Commit: 0b93b6f Link to invocation

[tensorrt-cicd]

PR_Github/16783-0b93b6f #61222 was force-killed by a newer pipeline run.
L0 job information not available (job may not have been triggered yet).

Link to superseding invocation

[tensorrt-cicd]

PR_Github #61224 [ run ] completed with state SUCCESS. Commit: 0b93b6f
/LLM/main/L0_MergeRequest_PR pipeline #49465 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

[chzblych]

Approved for the CI changes.

[2ez4bz]

/bot run

[tensorrt-cicd]

PR_Github #61341 [ run ] triggered by Bot. Commit: 291449c Link to invocation

[tensorrt-cicd]

PR_Github #61341 [ run ] completed with state FAILURE. Commit: 291449c
/LLM/main/L0_MergeRequest_PR pipeline #49569 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation