[None][test] Waive failed cases for main in QA CI

[crazydemo]

Auto-generated Waive PR

Created by: TensorRT LLM CI Report (requested by qa@nvidia.com)

Target branch: main, 8xA100
Jenkins build: https://prod.blsm.nvidia.com/swqa-tensorrt-qa-test/job/LLM_FUNCTION_TEST/2117/
Bug(s): 6087946, 6112497

Target branch: main, 8xL40S
Jenkins build: https://prod.blsm.nvidia.com/swqa-tensorrt-qa-test/job/LLM_FUNCTION_TEST/2118/
Bug(s): 6112497, 6112500

Target branch: main, 8xH100
Jenkins build: https://prod.blsm.nvidia.com/swqa-tensorrt-qa-test/job/LLM_FUNCTION_TEST/2135/
Bug(s): 5981122, 6112497

Target branch: main, B200
Jenkins build: https://prod.blsm.nvidia.com/swqa-tensorrt-qa-test/job/LLM_FUNCTION_CLUSTER_TEST/1403/
Bug(s): 5705199, 6114464, 6114610, 6114612
Jenkins build: https://prod.blsm.nvidia.com/swqa-tensorrt-qa-test/job/LLM_FUNCTION_CLUSTER_TEST/1404/
Bug(s): 6114608

Target branch: main, GB200
Jenkins build: https://prod.blsm.nvidia.com/swqa-tensorrt-qa-test/job/LLM_FUNCTION_CLUSTER_TEST/1394/
Bug(s): 6011317, 6094102, 6114139, 6114140, 6114141, 6114142, 6114464

Target branch: main, B300
Jenkins build: https://prod.blsm.nvidia.com/swqa-tensorrt-qa-test/job/LLM_FUNCTION_CLUSTER_TEST/1395/
Bug(s): 5702795, 6114608, 6115560, 6115562

Target branch: main, GB300
Jenkins build: https://prod.blsm.nvidia.com/swqa-tensorrt-qa-test/job/LLM_FUNCTION_CLUSTER_TEST/1401/
Bug(s): 6114139, 6114140, 6114141, 6114142

Waive entries added

accuracy/test_llm_api_autodeploy.py::TestNemotronNanoV3::test_accuracy[fp8-4-trtllm] SKIP (https://nvbugs/6112500)
accuracy/test_llm_api_pytorch.py::TestLlama3_1_8BInstruct::test_bfloat16_4gpus[pp4-attn_backend=TRTLLM-torch_compile=False] SKIP (https://nvbugs/6112497)
accuracy/test_llm_api_pytorch.py::TestGemma3_1BInstruct::test_fp8_prequantized[torch_compile=False] SKIP (https://nvbugs/6087946)
accuracy/test_llm_api_pytorch.py::TestGemma3_1BInstruct::test_fp8_prequantized[torch_compile=True] SKIP (https://nvbugs/6087946)
disaggregated/test_disaggregated.py::test_disaggregated_ctxtp2_genpp2[TinyLlama-1.1B-Chat-v1.0] SKIP (https://nvbugs/6114140)
accuracy/test_dwdp_disaggregated_serving.py::TestDwdpDeepSeekV3Lite::test_dwdp_accuracy SKIP (https://nvbugs/6094102)
disaggregated/test_disaggregated.py::test_disaggregated_overlap_gen_first[ctx_pp4-TinyLlama-1.1B-Chat-v1.0] SKIP (https://nvbugs/6114142)
disaggregated/test_workers.py::test_workers_kv_cache_events[TinyLlama-1.1B-Chat-v1.0] SKIP (https://nvbugs/6114139)
accuracy/test_llm_api_pytorch_multimodal.py::TestQwen3VL_MOE::test_auto_dtype SKIP (https://nvbugs/6114464)
disaggregated/test_disaggregated.py::test_disaggregated_trtllm_sampler[TinyLlama-1.1B-Chat-v1.0] SKIP (https://nvbugs/6114141)
disaggregated/test_disaggregated.py::test_disaggregated_ctxtp2pp2_gentp2pp2[TinyLlama-1.1B-Chat-v1.0] SKIP (https://nvbugs/6114610)
disaggregated/test_disaggregated.py::test_disaggregated_ctxpp4_gentp4[TinyLlama-1.1B-Chat-v1.0] SKIP (https://nvbugs/6114610)
disaggregated/test_disaggregated.py::test_disaggregated_ctxpp4_genpp4[TinyLlama-1.1B-Chat-v1.0] SKIP (https://nvbugs/6114612)
test_e2e.py::test_multi_nodes_eval[DeepSeek-R1/DeepSeek-R1-0528-FP4-tp16-mmlu] SKIP (https://nvbugs/6114608)
test_e2e.py::test_multi_nodes_eval[Qwen3/saved_models_Qwen3-235B-A22B_nvfp4_hf-tp16-mmlu] SKIP (https://nvbugs/6114608)
test_e2e.py::test_multi_nodes_eval[Kimi-K2-Thinking-NVFP4-tp16-mmlu] SKIP (https://nvbugs/6114608)
test_e2e.py::test_multi_nodes_eval[nemotron-nas/Llama-3_1-Nemotron-Ultra-253B-v1-tp16-mmlu] SKIP (https://nvbugs/6114608)
test_e2e.py::test_multi_nodes_eval[Qwen3/Qwen3-235B-A22B-tp16-mmlu] SKIP (https://nvbugs/6115560)
accuracy/test_llm_api_pytorch.py::TestQwen3NextInstruct::test_nvfp4[tp1-cutlass] SKIP (https://nvbugs/6116088)
test_e2e.py::test_openai_disagg_multi_nodes_completion_service_discovery[http] SKIP (https://nvbugs/6115562)
accuracy/test_llm_api_pytorch.py::TestDeepSeekV3Lite::test_bfloat16_python_scheduler[mtp_nextn=2-attention_dp=True-cuda_graph=True-overlap_scheduler=True-enable_chunked_prefill=False] SKIP (https://nvbugs/5981122)

Summary by CodeRabbit

• Tests
  • Added test waivers for expanded test coverage across multiple model families and configurations, including autodeploy accuracy, multi-GPU setups, disaggregated workflows, serving accuracy tests, multimodal model dtype validation, and multi-node evaluation scenarios.

Description

Test Coverage

PR Checklist

Please review the following before submitting your PR:

• PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.

• PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.

• Test cases are provided for new code paths (see test instructions)

• Any new dependencies have been scanned for license and vulnerabilities

• CODEOWNERS updated if ownership changes

• Documentation updated as needed

• Update tava architecture diagram if there is a significant design change in PR.

• The reviewers assigned automatically/manually are appropriate for the PR.

• Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see a list of available CI bot commands, please comment /bot help.

[crazydemo]

/bot run --skip-test

[coderabbitai]

📝 Walkthrough

Walkthrough

Updates the test waiver list by adding 19 new SKIP entries for various test scenarios including autodeploy accuracy, disaggregated workflows, DWDP serving, multimodal MoE dtype, and E2E multi-node evaluation cases across different model families and configurations.

Changes

Cohort / File(s)	Summary
Test Waivers tests/integration/test_lists/waives.txt	Added 19 new SKIP entries covering: Nemotron Nano V3 fp8-4-trtllm autodeploy, Llama 3.1 8B bfloat16 cases, TinyLlama disaggregated workflows, DWDP DeepSeek V3 Lite serving, Qwen 3 VL MoE auto dtype, and multi-node E2E evaluation scenarios with various TP/MMLU configurations.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~3 minutes

Possibly related PRs

• [None][infra] Waive 2 failed cases for main in post-merge #13335: Another PR that appends SKIP entries to the same waiver list file.

Suggested reviewers

• EmmaQiaoCh
• jieli-matrix

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The PR title '[None][test] Waive failed cases for main in QA CI' clearly and specifically describes the main change: adding test waivers for failing QA CI cases on the main branch.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description check	✅ Passed	The PR description provides comprehensive information about auto-generated waive entries, including target branches, Jenkins builds, NVBugs references, and specific test entries being added.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@tests/integration/test_lists/waives.txt`:
- Line 436: Remove the duplicate waiver line
"accuracy/test_llm_api_pytorch.py::TestLlama3_1_8BInstruct::test_bfloat16_4gpus[pp4-attn_backend=TRTLLM-torch_compile=False]
SKIP (https://nvbugs/6112497)" from tests/integration/test_lists/waives.txt (the
identical entry already exists earlier in the file around the original waiver
for the same node id), leaving only the existing waiver entry to avoid
redundancy and NVBugs tracking ambiguity.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 6e047414-1d90-4422-a35b-dbfe2b001a7d

📥 Commits

Reviewing files that changed from the base of the PR and between 2ea0e63 and d5a79cf.

📒 Files selected for processing (1)

• tests/integration/test_lists/waives.txt

[coderabbitai]

⚠️ Potential issue | 🟡 Minor

Duplicate waiver entry already exists earlier in this file.

The exact same node id is already waived at Line 169, so this new line is redundant and can cause tracking ambiguity across NVBugs.

Suggested cleanup

-accuracy/test_llm_api_pytorch.py::TestLlama3_1_8BInstruct::test_bfloat16_4gpus[pp4-attn_backend=TRTLLM-torch_compile=False] SKIP (https://nvbugs/6112497)

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change

accuracy/test_llm_api_pytorch.py::TestLlama3_1_8BInstruct::test_bfloat16_4gpus[pp4-attn_backend=TRTLLM-torch_compile=False] SKIP (https://nvbugs/6112497)

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@tests/integration/test_lists/waives.txt` at line 436, Remove the duplicate
waiver line
"accuracy/test_llm_api_pytorch.py::TestLlama3_1_8BInstruct::test_bfloat16_4gpus[pp4-attn_backend=TRTLLM-torch_compile=False]
SKIP (https://nvbugs/6112497)" from tests/integration/test_lists/waives.txt (the
identical entry already exists earlier in the file around the original waiver
for the same node id), leaving only the existing waiver entry to avoid
redundancy and NVBugs tracking ambiguity.

[tensorrt-cicd]

PR_Github #45721 [ run ] triggered by Bot. Commit: d5a79cf Link to invocation

[crazydemo]

/bot run --skip-test

[tensorrt-cicd]

PR_Github #45727 [ run ] triggered by Bot. Commit: d5a79cf Link to invocation

[tensorrt-cicd]

PR_Github #45727 [ run ] completed with state SUCCESS. Commit: d5a79cf
/LLM/main/L0_MergeRequest_PR pipeline #35925 (Partly Tested) completed with status: 'SUCCESS'

CI Report

Link to invocation