Repository navigation
feat(dsv4): canonical BI sparse attention and sink VJPs (item 25) - #629
inaniloquentee wants to merge 24 commits into
Conversation
Signed-off-by: fangfangssj <1135470306@qq.com>
Signed-off-by: Ziyi <ziyizhao0723@gmail.com>
docs: sync blog sidebar navigation
ws1-gtest-gpu and ws1-chain-gpu fail at "Configure runpodctl" before any test runs. They install releases/latest (v2.14.0 since 2026-09-10), whose deprecated `config --apiKey` aborts with `Config File ".runpod.yaml" Not Found` when it is the first runpodctl call on a fresh HOME. It only succeeds once an earlier call has created ~/.runpod/config.toml, which is why gpu-ci, whose install step happens to run `runpodctl version` first, got past the same command. Drop the config step in all three workflows. runpodctl v2 reads RUNPOD_API_KEY from the environment, and every step that runs ci/run_gpu_ci.sh already exports it, including the cleanup trap that calls `pod remove`. Verified for v2.9.0 inside `unshare -rn`, from a fresh HOME after `runpodctl version`: without the variable `pod list` fails locally with no_credentials, with RUNPOD_API_KEY=dummy it gets past the credential check to a network_error, and config.toml keeps `apikey = ''` throughout. v2.14.0 behaves the same. Pin the download to v2.9.0, the version gpu-ci run 31455127310 exercised end to end (pod create with sold-out fallback, pod id parsing, pod get polling, SSH, cleanup), and verify it with sha256sum -c against the runpodctl-linux-amd64 line of checksums_2.9.0_sha256.txt (06e6f54957db79d5cd9f1909a7f1d365076826751ba2f5df65d75dde43a64148), which matches GitHub's asset digest. Signed-off-by: Wenbo Ji <36562829+fusheng-ji@users.noreply.github.com>
fix(ci): pin runpodctl v2.9.0 and drop the deprecated config step (RUNPOD_API_KEY auth)
Signed-off-by: maxiaosong1124 <maxiaosong7890@outlook.com>
Signed-off-by: maxiaosong1124 <maxiaosong7890@outlook.com>
Signed-off-by: maxiaosong1124 <maxiaosong7890@outlook.com>
Signed-off-by: maxiaosong1124 <maxiaosong7890@outlook.com>
ci: remove obsolete RunPod WS1-chain-GPU workflow
Signed-off-by: maxiaosong1124 <maxiaosong7890@outlook.com>
Signed-off-by: maxiaosong1124 <maxiaosong7890@outlook.com>
Signed-off-by: maxiaosong1124 <maxiaosong7890@outlook.com>
Signed-off-by: maxiaosong1124 <maxiaosong7890@outlook.com>
ci: add RL-Kernel-specific automatic OCR review
Signed-off-by: maxiaosong1124 <maxiaosong7890@outlook.com>
Signed-off-by: maxiaosong1124 <maxiaosong7890@outlook.com>
Signed-off-by: lamentropetion <3051000145@qq.com>
docs(rocm): make the vLLM companion base reproducible from upstream
[Ascend] Fix deterministic attention backward compilation
…ned Graph records
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the
 |
|
🔍 OpenCodeReview found 9 issue(s) in this PR.
📄
|
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
| @@ -43,15 +43,20 @@ jobs: | |||
| ref: ${{ github.event.pull_request.base.sha }} | |||
|
|
|||
| - name: Install runpodctl | |||
There was a problem hiding this comment.
The separate Configure runpodctl step (runpodctl config --apiKey "${{ secrets.RUNPOD_API_KEY }}") was removed, and the API key is now only exported as an env var in the later "Run GPU tests on RunPod" step. ci/scripts/run_gpu_ci.sh invokes runpodctl pod create/get/remove without any --api-key flag or runpodctl config call, so unless the pinned CLI happens to read RUNPOD_API_KEY from the environment, every runpodctl call will run unauthenticated and the job (including pod cleanup) will fail. Please either restore an explicit configure step (e.g. runpodctl config --apiKey "$RUNPOD_API_KEY") or confirm/encode the env-var mechanism the CLI uses.
| @@ -89,13 +89,19 @@ jobs: | |||
| persist-credentials: false | |||
|
|
|||
| - name: Install runpodctl | |||
There was a problem hiding this comment.
Same as gpu-ci.yml: the Configure runpodctl step that performed runpodctl config --apiKey "${{ secrets.RUNPOD_API_KEY }}" was dropped. ci/scripts/run_gpu_ci.sh calls runpodctl pod create/get/remove without passing --api-key, so without an explicit configure step the CLI is unauthenticated and the GPU job plus its cleanup trap will fail. Restore the configure step or confirm the pinned CLI reads the API key from the environment.
| "include": [ | ||
| "**/*.{cu,cuh,hip}", | ||
| "tests/**", | ||
| "rl_engine/validation/**" | ||
| ], |
There was a problem hiding this comment.
The rules list contains focused rules for {csrc,rl_engine/backends,rl_engine/kernels}/**, rl_engine/distributed/**, rl_engine/runtime/**, rl_engine/integrations/**, rl_engine/{ops,contracts,reference,models,layers}/** and {.github,ci,build_tools}/**, but include only lists **/*.{cu,cuh,hip}, tests/** and rl_engine/validation/**. If include acts as a selection filter (the docs describe it as overriding OCR's default extension/path filters so that .cu/.cuh/.hip are not dropped), then rl_engine/**/*.py files — e.g. rl_engine/integrations/orchestrators/..., rl_engine/models/deepseek_v4/*.py, rl_engine/backends/rocm/** — are never submitted for review and every Python-specific rule above becomes dead configuration. This PR is exactly such a case (DeepSeek-V4 model/backend/integration Python files are not matched by any include entry). Verify with ocr rules check rl_engine/models/deepseek_v4/joint_attention.py; if it is not selected, add rl_engine/**/*.py (and ci/**) to include.
| kvkeys = [[0, 0, i * 4 + 3] for i in range(topk)] | ||
| kvkeys.extend([0, 1, p] for p in range(start - window + 1, start + m)) |
There was a problem hiding this comment.
The kvkeys layout only ever uses request id 0, while qkeys spans m requests (e.g. m=7, m=65). JointMetadata.bind requires every active query's Top-K rows to satisfy kh[i][0] == request, so for m > 1 any selected index whose key has request == 0 is rejected with "cross-request, future or noncompressed Top-K row" — and since the selected tensor is the same arange(topk) for all m rows, only rows for request 0 can ever be valid. The script therefore cannot run for 3 of its 4 shapes (and the remaining benches an m=1 degenerate case). Emit per-request keys like benchmark_joint_attention.py does, plus matching per-request selected indices:
Suggestion:
| kvkeys = [[0, 0, i * 4 + 3] for i in range(topk)] | |
| kvkeys.extend([0, 1, p] for p in range(start - window + 1, start + m)) | |
| kvkeys, selected = [], [] | |
| for request in range(m): | |
| offset = len(kvkeys) | |
| kvkeys.extend([request, 0, i * 4 + 3] for i in range(topk)) | |
| kvkeys.extend([request, 1, p] for p in range(start - window + 1, start + m)) | |
| selected.append(list(range(offset, offset + topk))) |
| c, | ||
| torch.tensor(qkeys, device="cuda"), | ||
| torch.tensor(kvkeys, device="cuda"), | ||
| torch.arange(topk, dtype=torch.int32, device="cuda")[None].repeat(m, 1), |
There was a problem hiding this comment.
This still uses the old single-request selected tensor (arange(topk) repeated for all m rows) and the old (start - window + 1, start + m) window. With the fix above, the window must be built per request (it currently emits m window rows per request while bind only expects the window_size rows ending at that query's position), and the indices must be the corresponding per-request selected lists materialized as an int32 tensor of shape (m, topk).
Suggestion:
| torch.arange(topk, dtype=torch.int32, device="cuda")[None].repeat(m, 1), | |
| torch.tensor(selected, dtype=torch.int32, device="cuda"), |
|
@inaniloquentee The automated OCR review has completed. Please review and address all automated review bot findings, then push your updates. Maintainers will begin manual review after the bot findings have been addressed. |
Flink-ddd
added
platform: rocm
Roadmap item 25 ??sparse attention and sink backward
Target:
dsv4-rocm. Task-only commit:ce1c45d322191c8ff03e7dfe2a568524729bcf63.Prerequisite: item 24 / #628 (
159b0592be933b65aac76ff8b9fe411fa3f27456), itself on current main8588158650aff58c9c1f527674de94bad06c3d10(rechecked). Review task-only changes against159b0592; the target diff includes its unmerged forward prerequisite.Native reuse, not rewritten kernels
MI300X evidence
reports/joint-backward-mi300x-20261029.{md,json}.Explicit boundaries
The eager canonical autograd collector is not a captured whole training step. Fixed-record Graph does not claim dynamic CSR capture or framework checkpoint orchestration. CP/TP transport, quantized cache-byte gradients, higher-order gradients, full engines/models and cross-parallel-degree equality remain separate roadmap work. Native training VJPs use the upstream BF16 P/dP approximation; discrete rounding derivatives are not claimed. No baseline-engine speedup is inferred from the timings.