[Cherry-Pick][Optimization]Kernel fusion: cast+sigmoid+bias+noauxtc(#7777)#7782
[Cherry-Pick][Optimization]Kernel fusion: cast+sigmoid+bias+noauxtc(#7777)#7782ShaneGZhu wants to merge 3 commits into
Conversation
* conflict * support more cast type * modify test * add type check * fix config issues * enable more backend * modify 2025->2026 * only support gpu backend and fix test issues * support gpu backend * modify format * modify except type * conflict fix * format * fix dynamic_load_weight config issues * fix dynamic_load_weight config issue * fix fake get_moe_scores config issues * add mock test * modify RL control env * Modify code standardization * fix test issue * fix test issue * fix conflict issue * fix mockdata config issue * enhance test converage * fix test
|
Thanks for your contribution! |
PaddlePaddle-bot
left a comment
PaddlePaddle-bot
left a comment
There was a problem hiding this comment.
🤖 Paddle-CI-Agent | pr_review |
2026-05-12 11:39:28
📋 Review 摘要
PR 概述：为 MoE 模型（如 DeepSeek-V3）实现 cast+sigmoid+bias+noaux_tc 融合内核优化，减少显存带宽消耗和 kernel 启动开销
变更范围：custom_ops/gpu_ops/、fastdeploy/model_executor/layers/moe/
影响面 Tag：[OP]
📝 PR 规范检查
标题格式基本符合 Cherry-Pick 规范，但 [Optimization] 与描述之间缺少空格（当前：[Cherry-Pick][Optimization]Kernel，应为 [Cherry-Pick][Optimization] Kernel）。PR 描述所有段落均为空，仅保留了模板注释占位符，需补全。
标题建议（可直接复制）：
[Cherry-Pick][Optimization] Kernel fusion: cast+sigmoid+bias+noauxtc(#7777)
PR 描述建议（可直接复制，必须复刻 checklist §D2 模板的完整结构）：
## Motivation
为 MoE 模型（如 DeepSeek-V3）的 gate 路由实现内核融合优化，将 cast+sigmoid+bias+grouped_topk（noaux_tc）融合为单个 GPU kernel `grouped_topk`，减少显存带宽消耗和 kernel 启动开销；同时提供独立的 `fused_cast_sigmoid_bias` 算子供仅执行 cast+sigmoid+bias 的场景使用。Cherry-pick 自 #7777。
## Modifications
- `custom_ops/gpu_ops/grouped_topk_kernels.cu`：新增融合 CUDA kernel，将 cast+sigmoid+bias+grouped topk (noaux_tc) 合并为单 kernel
- `custom_ops/gpu_ops/fused_cast_sigmoid_bias.cu`：新增融合 CUDA kernel，执行 cast+sigmoid+bias
- `custom_ops/gpu_ops/cpp_extensions.cc`：注册 `grouped_topk` 与 `fused_cast_sigmoid_bias` 的 pybind11 绑定
- `custom_ops/setup_ops.py`：将两个新 `.cu` 文件加入编译列表（GPU 编译的两处位置）
- `fastdeploy/model_executor/layers/moe/fused_cast_sigmoid_bias.py`：新增 Python 封装模块
- `fastdeploy/model_executor/layers/moe/moe.py`：`get_moe_scores` 新增 `use_fused_cast` 参数，支持走融合 kernel 路径（`grouped_topk`）
- `fused_moe_cutlass_backend.py`、`fused_moe_deepgemm_backend.py`、`fused_moe_triton_backend.py`：在 CUDA + 非 RL 模式下自动启用融合内核
- 新增单测：`tests/layers/test_fused_cast_sigmoid_bias.py`、`tests/layers/test_deepgemm_fused_moe.py`、`tests/layers/test_fused_moe_cutlass_backend.py`、`tests/layers/test_fused_moe_triton_backend.py`、`tests/operators/test_grouped_topk_op.py`
## Usage or Command
N/A
## Accuracy Tests
N/A
## Checklist
- [x] Add at least a tag in the PR title.
- Tag list: [`[FDConfig]`,`[APIServer]`,`[Engine]`, `[Scheduler]`, `[PD Disaggregation]`, `[Executor]`, `[Graph Optimization]`, `[Speculative Decoding]`, `[RL]`, `[Models]`, `[Quantization]`, `[Loader]`, `[OP]`, `[KVCache]`, `[DataProcessor]`, `[BugFix]`, `[Docs]`, `[CI]`, `[Optimization]`, `[Feature]`, `[Benchmark]`, `[Others]`, `[XPU]`, `[HPU]`, `[GCU]`, `[DCU]`, `[Iluvatar]`, `[Metax]`]
- You can add new tags based on the PR content, but the semantics must be clear.
- [ ] Format your code, run `pre-commit` before commit.
- [x] Add unit tests. Please write the reason in this PR if no unit tests.
- [ ] Provide accuracy results.
- [x] If the current PR is submitting to the `release` branch, make sure the PR has been submitted to the `develop` branch, then cherry-pick it to the `release` branch with the `[Cherry-Pick]` PR tag.问题
| 级别 | 文件 | 概述 |
|---|---|---|
| 📝 PR 规范 | PR 描述 | 所有段落均为空，仅保留模板注释 |
| 🟡 建议 | fastdeploy/model_executor/layers/moe/moe.py:50 |
死代码块 if current_platform.is_cuda(): pass 应删除 |
总体评价
融合内核设计合理，grouped_topk 将 cast+sigmoid+bias+noaux_tc 四步合并为单 kernel，正确处理了 float32 中间精度保证与边界情况；编译注册、Python 调用侧、单测均已同步添加。主要问题是 PR 描述段落为空以及 moe.py 中残留的死代码块（if current_platform.is_cuda(): pass），建议修复后合入。
| import numpy as np | ||
|
|
||
| if current_platform.is_cuda(): | ||
| pass |
There was a problem hiding this comment.
🟡 建议 死代码块，应删除
if current_platform.is_cuda(): pass 是一个空操作块，对运行时没有任何效果。疑似在重构过程中遗留，未在 body 内填入实际逻辑（例如条件导入 fused_cast_sigmoid_bias）。
建议直接删除这两行：
# 删除以下两行:
if current_platform.is_cuda():
pass
CI报告基于以下代码生成（30分钟更新一次）: 1 任务总览CI 仍有任务运行中：1 个 required 任务失败（
2 任务状态汇总2.1 Required任务 : 3/10 通过
2.2 可选任务 — 21/25 通过
3 失败详情（仅 required）Approval — 审批流程（置信度: 高）Approval
根因详情: 关键日志: 修复建议:
修复建议摘要: 请 FastDeploy RD 和 PaddlePaddle RD 各自在 PR 上 Approve 链接: 查看日志 |
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## release/2.6 #7782 +/- ##
==============================================
Coverage ? 71.89%
==============================================
Files ? 379
Lines ? 53967
Branches ? 8442
==============================================
Hits ? 38801
Misses ? 12396
Partials ? 2770
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
4 participants
Motivation
Modifications
Usage or Command
Accuracy Tests
Checklist
[FDConfig],[APIServer],[Engine],[Scheduler],[PD Disaggregation],[Executor],[Graph Optimization],[Speculative Decoding],[RL],[Models],[Quantization],[Loader],[OP],[KVCache],[DataProcessor],[BugFix],[Docs],[CI],[Optimization],[Feature],[Benchmark],[Others],[XPU],[HPU],[GCU],[DCU],[Iluvatar],[Metax]]pre-commitbefore commit.releasebranch, make sure the PR has been submitted to thedevelopbranch, then cherry-pick it to thereleasebranch with the[Cherry-Pick]PR tag.