Bug Description
tests/py/dynamo/models/test_models_export.py::test_fp8_mha_fused_kernel fails on H100. It's currently worked around with a skip on release/ngc/26.09 (not yet upstreamed to main):
@unittest.skipIf(
torch.cuda.get_device_capability() < (8, 9),
"FP8 quantization requires compute capability 8.9 or later",
)
@unittest.skipIf(
torch.cuda.get_device_capability() == (9, 0),
"test_fp8_mha_fused_kernel is failing on H100",
)The root cause was never diagnosed — this issue is to track investigating and properly fixing it rather than leaving it permanently skipped on H100.
The test hand-constructs an FX pattern mirroring modelopt's FP8 MHA export (quantize_op on Q/K/V feeding scaled_dot_product_attention), compiles it with use_explicit_typing=True, and asserts:
- The compiled engine contains a fused
mha layer (i.e. _gemm_mha_v2), confirming FP8 MHA fusion triggered.
- Numerics match PyTorch reference SDPA within
cosine_similarity > 0.99.
It passes on other compute-capability ≥ 8.9 GPUs; only H100 (compute capability (9,0)) is excluded.
To Reproduce
pytest tests/py/dynamo/models/test_models_export.py::test_fp8_mha_fused_kernel -v
on an H100 GPU, with the H100-specific skipIf above removed.
Expected behavior
Test should pass on H100 (compute capability 8.9+), or if H100 genuinely can't support the fused FP8 MHA kernel (_gemm_mha_v2/normalization_quantize path), the test/converter should document why and skip with a precise, understood reason rather than an unexplained one.
Environment
- GPU: H100
- TensorRT: 11.3.x (dynamo
use_explicit_typing=True path)
Additional context
Skip was added opportunistically while triaging a batch of CI failures; nobody has looked at the actual failure signature (assertion failure on the mha layer-name check vs. numeric mismatch vs. a build-time error) yet.
Bug Description
tests/py/dynamo/models/test_models_export.py::test_fp8_mha_fused_kernelfails on H100. It's currently worked around with a skip onrelease/ngc/26.09(not yet upstreamed tomain):The root cause was never diagnosed — this issue is to track investigating and properly fixing it rather than leaving it permanently skipped on H100.
The test hand-constructs an FX pattern mirroring modelopt's FP8 MHA export (
quantize_opon Q/K/V feedingscaled_dot_product_attention), compiles it withuse_explicit_typing=True, and asserts:mhalayer (i.e._gemm_mha_v2), confirming FP8 MHA fusion triggered.cosine_similarity > 0.99.It passes on other compute-capability ≥ 8.9 GPUs; only H100 (compute capability (9,0)) is excluded.
To Reproduce
on an H100 GPU, with the H100-specific
skipIfabove removed.Expected behavior
Test should pass on H100 (compute capability 8.9+), or if H100 genuinely can't support the fused FP8 MHA kernel (
_gemm_mha_v2/normalization_quantizepath), the test/converter should document why and skip with a precise, understood reason rather than an unexplained one.Environment
use_explicit_typing=Truepath)Additional context
Skip was added opportunistically while triaging a batch of CI failures; nobody has looked at the actual failure signature (assertion failure on the
mhalayer-name check vs. numeric mismatch vs. a build-time error) yet.