[Bugfix] Fix persistent_masked_m_silu_mul_quant tests #28366
Conversation
Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
varun-sundar-rabindranath
requested review from
WoosukKwon,
mgoin,
pavanimajety,
tlrmchlsmth and
yewentao256
as code owners
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces two important bug fixes for the persistent_masked_m_silu_mul_quant kernel. The first fix correctly handles cases where the number of groups is not divisible by the number of warps by falling back to a 1-warp configuration. The second fix adds a parameter to control ue8m0 scale ceiling to align the kernel's behavior with the reference implementation in tests. The changes are logical and well-targeted. I've found one critical issue in an updated validation check that needs to be addressed.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
mgoin
added
bug
…8366) Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com> Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com> Signed-off-by: xuebwang-amd <xuebwang@amd.com>
…8366) Signed-off-by: Varun Sundar Rabindranath <vsundarr@redhat.com> Co-authored-by: Varun Sundar Rabindranath <vsundarr@redhat.com>
3 participants
Purpose
(8, 128, 128 * 33, fp8_dtype). This is because when hidden size is>= 4096we launch a kernel with 8 warps and the kernel requires that the NUM_GROUPS (hidden_size / 128) divides evenly between all warps. For this case, the PR just falls back to the 1 warp case.Test Plan
test_silu_mul_fp8_quant_deep_gemm.pylocallyTest Result