Repository navigation
[release/1.6] Lower fused CUDA attention variants - #24491
Merged
JacobSzwejbka merged 2 commits intoNov 3, 2026
Merged
JacobSzwejbka merged 2 commits into
JacobSzwejbka merged 2 commits into
Conversation
Cherry-pick #24426, retaining the release missing-kernel regression test and importing tempfile for the new attention test. Prepared with OpenAI Codex.
Preserve the test unchanged while avoiding a merge conflict with the preceding CUDA indexing pick. Prepared with OpenAI Codex.
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/24491
Note: Links to docs will display an error until the docs builds have been completed. ❌ 1 New FailureAs of commit 11a1092 with merge base 4db778e ( NEW FAILURE - The following job has failed:
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
meta-cla
Bot
added
the
CLA Signed
JacobSzwejbka
deployed
to
upload-benchmark-results
November 3, 2026 12:50 — with 
GitHub Actions
Active
GitHub Actions
Active
shoumikhin
approved these changes
Nov 3, 2026
JacobSzwejbka
deleted the
codex/release-1.6-clean-cuda-fused-attention-20261103
branch
November 3, 2026 17:04
JacobSzwejbka
added a commit
that referenced
this pull request
Nov 3, 2026
…ductions (#24492) Supersedes #24377 with a clean release pick from #24294. Fixes scalar tensors, float32 CPU defaults, dynamic output buffers and integer reductions. Includes the related #24396 runtime, folded-cumsum, parameter-count and CI runner follow-ups previously spread across #24377 and #24446. This pick contains the complete core OpenVINO fix and its required follow-ups. Bool-op handling and masked-read metadata are isolated in the next clean pick. The source float32 default can cost speed on CPUs with bfloat16 hardware; inference_precision=bf16 remains available. ## Scope and landing order Land after #24491 in the verified landing order. The OpenVINO pair follows the CUDA picks; these backend groups can otherwise be reviewed separately. This branch starts directly at release/1.6 commit `dfb78b9ecd933a5fee33ebc94f69ed4b71cbb1e7`. Its GitHub diff contains 16 files for this pick; it does not carry duplicate #24298 history or the excluded unmerged QNN work. Dependencies are documented here rather than embedded as other cherry-pick commits. ## Verification All changed Python files compile and pass Ruff F821 checks. All seven replacements merge cleanly in the documented sequence, including simulated squash merges. No QNN files are changed. The combined OpenVINO production code matches the previously reviewed final #24446 code; the bool test class is preserved unchanged. Native CUDA, OpenVINO and portable C++ suites were not run locally on this macOS host; PR CI remains required before landing. Prepared with OpenAI Codex.
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Supersedes #24455 with a clean release pick from #24426.
Restores inference-only fused flash, efficient and cuDNN attention calls to scaled_dot_product_attention when the source pass can preserve their semantics.
The fused-attention test is moved unchanged to avoid conflicts with the earlier indexing pick. The existing release missing-kernel regression test is retained.
Scope and landing order
Land after #24490 in the verified landing order.
This branch starts directly at release/1.6 commit
dfb78b9ecd933a5fee33ebc94f69ed4b71cbb1e7. Its GitHub diff contains 5 files for this pick; it does not carry duplicate #24298 history or the excluded unmerged QNN work. Dependencies are documented here rather than embedded as other cherry-pick commits.Verification
All changed Python files compile and pass Ruff F821 checks. All seven replacements merge cleanly in the documented sequence, including simulated squash merges. No QNN files are changed. The existing CPU graph rewrite test passed its nine cases using the real inherited AotiPartitioner implementation. The GPU import/pass-registration test could not run locally because Triton is unavailable on macOS.
Native CUDA, OpenVINO and portable C++ suites were not run locally on this macOS host; PR CI remains required before landing.
Prepared with OpenAI Codex.