Repository navigation
feat(dsv4): BI grouped output projection with native aiter VJPs (item 23) - #627
inaniloquentee wants to merge 25 commits into
Conversation
Signed-off-by: fangfangssj <1135470306@qq.com>
Signed-off-by: Ziyi <ziyizhao0723@gmail.com>
docs: sync blog sidebar navigation
ws1-gtest-gpu and ws1-chain-gpu fail at "Configure runpodctl" before any test runs. They install releases/latest (v2.14.0 since 2026-09-10), whose deprecated `config --apiKey` aborts with `Config File ".runpod.yaml" Not Found` when it is the first runpodctl call on a fresh HOME. It only succeeds once an earlier call has created ~/.runpod/config.toml, which is why gpu-ci, whose install step happens to run `runpodctl version` first, got past the same command. Drop the config step in all three workflows. runpodctl v2 reads RUNPOD_API_KEY from the environment, and every step that runs ci/run_gpu_ci.sh already exports it, including the cleanup trap that calls `pod remove`. Verified for v2.9.0 inside `unshare -rn`, from a fresh HOME after `runpodctl version`: without the variable `pod list` fails locally with no_credentials, with RUNPOD_API_KEY=dummy it gets past the credential check to a network_error, and config.toml keeps `apikey = ''` throughout. v2.14.0 behaves the same. Pin the download to v2.9.0, the version gpu-ci run 31455127310 exercised end to end (pod create with sold-out fallback, pod id parsing, pod get polling, SSH, cleanup), and verify it with sha256sum -c against the runpodctl-linux-amd64 line of checksums_2.9.0_sha256.txt (06e6f54957db79d5cd9f1909a7f1d365076826751ba2f5df65d75dde43a64148), which matches GitHub's asset digest. Signed-off-by: Wenbo Ji <36562829+fusheng-ji@users.noreply.github.com>
fix(ci): pin runpodctl v2.9.0 and drop the deprecated config step (RUNPOD_API_KEY auth)
Signed-off-by: maxiaosong1124 <maxiaosong7890@outlook.com>
Signed-off-by: maxiaosong1124 <maxiaosong7890@outlook.com>
Signed-off-by: maxiaosong1124 <maxiaosong7890@outlook.com>
Signed-off-by: maxiaosong1124 <maxiaosong7890@outlook.com>
ci: remove obsolete RunPod WS1-chain-GPU workflow
Signed-off-by: maxiaosong1124 <maxiaosong7890@outlook.com>
Signed-off-by: maxiaosong1124 <maxiaosong7890@outlook.com>
Signed-off-by: maxiaosong1124 <maxiaosong7890@outlook.com>
Signed-off-by: maxiaosong1124 <maxiaosong7890@outlook.com>
ci: add RL-Kernel-specific automatic OCR review
Signed-off-by: maxiaosong1124 <maxiaosong7890@outlook.com>
Signed-off-by: maxiaosong1124 <maxiaosong7890@outlook.com>
Signed-off-by: lamentropetion <3051000145@qq.com>
docs(rocm): make the vLLM companion base reproducible from upstream
[Ascend] Fix deterministic attention backward compilation
…4-o-proj-grouped-bi
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the
 |
|
🔍 OpenCodeReview found 15 issue(s) in this PR.
📄
|
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
| - name: Configure runpodctl | ||
| run: runpodctl config --apiKey "${{ secrets.RUNPOD_API_KEY }}" | ||
|
|
||
| - name: Setup SSH key |
There was a problem hiding this comment.
The Configure runpodctl step (runpodctl config --apiKey "${{ secrets.RUNPOD_API_KEY }}") was removed, but ci/scripts/run_gpu_ci.sh still runs runpodctl pod create/get/remove (lines 36/50/67/98) and never reads RUNPOD_API_KEY. With no ~/.runpod/config.toml written, every runpodctl call is unauthenticated: pod create fails, POD_ID stays empty, and the cleanup trap silently skips teardown — the GPU CI job cannot provision pods (and any partially created pod leaks). Restore the configuration step (or configure from the env var before invoking the script).
| run: runpodctl config --apiKey "${{ secrets.RUNPOD_API_KEY }}" | ||
| runpodctl version | ||
|
|
||
| - name: Setup SSH key |
There was a problem hiding this comment.
Same regression as gpu-ci.yml: removing runpodctl config --apiKey leaves run_gpu_ci.sh's runpodctl pod create/get/remove calls unauthenticated (RUNPOD_API_KEY in the job env is never read by the script). The WS1 GPU gate will fail at pod creation with no cleanup. Restore the configuration step.
| def weight_gradient(self, rows, grads): | ||
| return self.linear( | ||
| grads.t().contiguous(), | ||
| rows.t(), | ||
| output_dtype=torch.bfloat16, | ||
| role="dw", | ||
| ) |
There was a problem hiding this comment.
rows.t() is passed as the weight operand to linear, which calls self.plan(x.shape[1], weight.shape[0], role). For DW the expected output is [K, N] = [rows.shape[1], grads.shape[1]]. Here x = grads.t().contiguous() has shape [N, M] so x.shape[1] == M, and weight.shape[0] == rows.shape[1] == K. The plan is therefore built for k=M, n=K (the reduction/shape pair is swapped) — the frozen tile/SPLITK config is chosen for the wrong geometry, and more importantly the underlying GEMM computes grads.t() @ rows.t() = [N, K] rather than the required [K, N] DW (or an [K, N] result requires rows.t() @ grads.t() order). Verify against the canonical_backward submit_linear contract: the operand order and plan dimensions here look transposed relative to a [K,N] weight-gradient output.
| if weight.requires_grad: | ||
| ctx.session = active_session() | ||
| require(ctx.session is not None, "trainable weights require canonical_backward_session") | ||
| ctx.slots = tuple( | ||
| _register_canonical(ctx.session, f"{prefix}.{g}", weight, logical) | ||
| for g in range(x.shape[0]) | ||
| ) | ||
| if packed: |
There was a problem hiding this comment.
_GroupedLinear.forward registers a canonical slot per group whenever weight.requires_grad, but the packed branch then never submits to those slots (it calls backend.linear_fp8 directly). With a_precision="block_fp8_ue8m0" plus a trainable BF16 master (the documented BF16-master-surrogate VJP profile) every registered slot stays pending: session.validate_complete() raises "incomplete canonical submissions", and if A also has requires_grad, dw is never produced. The forward must either take the canonical path (surrogate master GEMM) when slots were registered, or refuse registrations in the packed branch.
| elif weights.packed_a: | ||
| z = torch.stack( | ||
| [ | ||
| backend.linear_fp8(grouped[g], p, output_dtype=torch.bfloat16) | ||
| for g, p in enumerate(weights.packed_a) | ||
| ] | ||
| ) |
There was a problem hiding this comment.
In OProjection.__call__, when a_grad is False but weights.packed_a is non-empty (FP8 A checkpoint with frozen weights — the normal inference case), the in-place torch.stack([backend.linear_fp8(...)]) branch runs without any _GroupedLinear/canonical registration. That is fine for grad, but the same branch is also reachable when a_grad is True? No — a_grad requires the BF16 master and then _GroupedLinear.apply is used. However, the opposite combination is unchecked: a_precision == "block_fp8_ue8m0" with a trainable master goes through _GroupedLinear, whose forward takes the packed branch and never records the canonical slot in the actual computation, see the comment on _GroupedLinear.forward. Please make the two dispatch sites agree (and assert here that a trainable A always has packed_a == () or vice-versa) so grads cannot be silently dropped.
|
@inaniloquentee The automated OCR review has completed. Please review and address all automated review bot findings, then push your updates. Maintainers will begin manual review after the bot findings have been addressed. |
Roadmap item 23 ??grouped attention output projection
Target:
dsv4-rocm. Task-only commit:807e58b5434f302e4fd3d587a0e02660b6e6b2bc.Explicit prerequisites / review base
f9c8b27d) rather than duplicating its native BF16/FP8 GEMMs, vLLM quantization/checkpoint ABI, and canonical VJP plumbing.8588158650aff58c9c1f527674de94bad06c3d10, merged as1becef6d. The target branch predates that main. Review item-23 changes against1becef6d; the PR diff against dsv4-rocm also exposes those prerequisites.Implementation
Tests and evidence
MI300X/gfx942; Torch 2.11, HIP 7.2, Triton 3.6, aiter 0.1.16.post3.
reports/o-projection-mi300x-20261029.{md,json}.Scope boundaries (not hidden as passing tests)
Graph is explicitly rejected. Local TP assembly is not actual RCCL transport or invariance across parallel degrees. No claim of full-model vLLM/Megatron/verl/vime integration, shuffled checkpoint support, CP/DCP, LoRA, or FP8 throughput speedup. These remain independent roadmap work.