[Graph Optimization]Revert blockwise CUDAGraph and support piecewise CUDAGraph in prefill#7969
Conversation
CI报告基于以下代码生成（30分钟更新一次）: 1 Required任务 : 4/10 通过
2 失败详情🔴 Run FastDeploy Unit Tests and Coverage / run_tests_with_coverage — PR问题（置信度: 高）分析器: ci_analyze_unittest_fastdeploy 失败用例:
关键日志:
修复建议:
关联变更: 🔴 Run Stable Tests / stable_tests — PR问题（置信度: 高）失败用例:
关键日志:
修复建议:
关联变更: 本 PR 修改 🔴 Extracted partial CE model tasks to run in CI. / run_ce_cases — PR问题（置信度: 高）失败用例:
关键日志:
修复建议:
关联变更: 本 PR 删除 🔴 Approval — 需要 Approval（置信度: 高）该 Job 需要人工 Approval，完成审批后 CI 才会继续执行。 🔴 xpu_8cards_case_test / run_xpu_8cards_cases — PR问题（置信度: 中）失败用例:
关键日志:
修复建议:
关联变更: 本 PR 修改 PD prefill piecewise CUDAGraph 控制流；失败测试位于 |
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## develop #7969 +/- ##
==========================================
Coverage ? 67.45%
==========================================
Files ? 474
Lines ? 66920
Branches ? 10296
==========================================
Hits ? 45138
Misses ? 18891
Partials ? 2891
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
ZhangX-21
force-pushed
the
piecewise_cudagraph
branch
from
29f8a1e to
342bf2d
Compare
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
ZhangX-21
force-pushed
the
piecewise_cudagraph
branch
from
65bec02 to
8bd2e90
Compare
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
ZhangX-21
changed the title
|
Thanks for your contribution! |
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
ZhangX-21
force-pushed
the
piecewise_cudagraph
branch
from
48fc5c0 to
638a81c
Compare
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
PaddlePaddle-bot
left a comment
PaddlePaddle-bot
left a comment
There was a problem hiding this comment.
🤖 Paddle-CI-Agent | pr_review |
2026-07-15 14:44:32 Asia/Shanghai
📋 Review 摘要
PR 概述：回退 block-wise CUDAGraph，并为 prefill/mixed 路径接入 piecewise CUDAGraph，同时调整 FlashAttention/MoE custom op 的 SOT 兼容逻辑。
变更范围：CUDAGraph 配置和 worker capture 流程、FlashAttention py-op 包装、MoE/TopK custom op infer_meta、图优化相关测试。
影响面 Tag：[Graph Optimization] [OP] [FDConfig]
问题
| 级别 | 文件 | 概述 |
|---|---|---|
| 🔴 Bug | custom_ops/gpu_ops/noaux_tc_redundant.cu:81 |
noaux_tc_redundant 的静态 infer_meta 仍只声明 3 个输出，和 4 输出注册不一致 |
历史 Findings 修复情况
| Finding | 问题 | 状态 |
|---|---|---|
| F3 | PD prefill piecewise 与 CPU cache 兼容性检查 | |
| F4 | WFP8AFP8 Triton MoE 中 paddle.sum(..., out=...) |
|
| F5 | batch_id_per_token 的 -1 padding sentinel |
✅ 已修复 |
| F6 | mixed batch 下 merge_prefill_decode_output 结果被丢弃 |
|
| F7 | python_op_wfp8afp8_triton_moe 未加入 CUDA Graph blacklist |
📝 PR 规范检查
标题缺少官方 Tag，描述结构完整但 Usage or Command 和 Accuracy Tests 仅为占位注释。
标题建议（可直接复制）：
[Graph Optimization] Revert blockwise CUDAGraph and support piecewise CUDAGraph in prefill
PR 描述建议（点击展开，可直接复制）
## Motivation
This PR reverts the previous blockwise CUDAGraph implementation and adds support for piecewise CUDAGraph in the prefill phase. The blockwise approach captured per-layer graphs which fragmented SOT-compiled graphs; the piecewise approach captures reusable sub-graph segments during prefill without graph fragmentation.
## Modifications
- Revert blockwise CUDAGraph related logic (remove `cuda_graph_op.py`, env vars `FD_USE_BLOCK_WISE_CUDA_GRAPH` / `FD_BLOCK_WISE_CUDA_GRAPH_SIZES`).
- Add `prefill_cudagraph_guard` around prefill capture.
- Extend prefill capture sizes up to 8192 tokens in `config.py`.
- Refactor `RMSNorm.forward`: remove dtype cast, use dynamic `max_chunk_tokens` for allreduce fusion.
- Pass `max_token_num` to `flashinfer_allreduce_residual_rmsnorm` to match workspace allocation.
- Keep decode CUDAGraph behavior unchanged.
## Usage or Command
N/A
## Accuracy Tests
N/A
## Checklist
- [x] Add at least a tag in the PR title.
- Tag list: [`[FDConfig]`,`[APIServer]`,`[Engine]`, `[Scheduler]`, `[PD Disaggregation]`, `[Executor]`, `[Graph Optimization]`, `[Speculative Decoding]`, `[RL]`, `[Models]`, `[Quantization]`, `[Loader]`, `[OP]`, `[KVCache]`, `[DataProcessor]`, `[BugFix]`, `[Docs]`, `[CI]`, `[Optimization]`, `[Feature]`, `[Benchmark]`, `[Others]`, `[XPU]`, `[HPU]`, `[GCU]`, `[DCU]`, `[Iluvatar]`, `[Metax]`]
- You can add new tags based on the PR content, but the semantics must be clear.
- [x] Format your code, run `pre-commit` before commit.
- [ ] Add unit tests. Please write the reason in this PR if no unit tests.
- [ ] Provide accuracy results.
- [ ] If the current PR is submitting to the `release` branch, make sure the PR has been submitted to the `develop` branch, then cherry-pick it to the `release` branch with the `[Cherry-Pick]` PR tag.总体评价
本轮按风险优先审查了 CUDAGraph prefill/mixed、FlashAttention Python op、MoE custom python op、noaux_tc static op metadata、GPU worker warmup/config 联动；未全量覆盖纯删除的 block-wise graph tests。当前新增的 static op infer_meta 不一致会阻断 SOT/custom op 构图，需要修复后再合入。
| const std::vector<int64_t>&, | ||
| const std::vector<int64_t>&, | ||
| const int n_group, | ||
| const int topk_group, |
There was a problem hiding this comment.
🔴 Bug noaux_tc_redundant 的静态 infer_meta 仍只返回 3 个输出。
这次签名补齐了额外输入和 attrs，但 PD_BUILD_STATIC_OP(noaux_tc_redundant) 注册了 4 个输出（包含 inplace 的 tokens_per_expert_stats_list_out），NoauxTcRedundantInferShape 仍只返回 3 个 shape，NoauxTcRedundantInferDtype 也只返回 3 个 dtype 且签名仍只覆盖前两个输入。进入 SOT/static custom op 路径时，infer_meta 和注册输出不一致会导致 op 构图失败或丢失 inplace 输出元信息。
建议把 shape/dtype infer 都补齐到与注册一致，例如返回第四个 tokens_per_expert_stats_list 的 shape/dtype，并让 InferDtype 接收新增 3 个输入 dtype 及 attrs 参数。
6 participants
Motivation
This PR reverts the previous blockwise CUDAGraph implementation and adds support for piecewise CUDAGraph in the prefill phase.
Modifications
Usage or Command
Accuracy Tests
Checklist
[FDConfig],[APIServer],[Engine],[Scheduler],[PD Disaggregation],[Executor],[Graph Optimization],[Speculative Decoding],[RL],[Models],[Quantization],[Loader],[OP],[KVCache],[DataProcessor],[BugFix],[Docs],[CI],[Optimization],[Feature],[Benchmark],[Others],[XPU],[HPU],[GCU],[DCU],[Iluvatar],[Metax]]pre-commitbefore commit.releasebranch, make sure the PR has been submitted to thedevelopbranch, then cherry-pick it to thereleasebranch with the[Cherry-Pick]PR tag.