Repository navigation
[WS1][CUDA][MiniMax-H3] checkpoint mapper and the vLLM-Omni rollout adapter (RFC #420 stack 7/7) - #575
Open
fusheng-ji wants to merge 39 commits into
Open
[WS1][CUDA][MiniMax-H3] checkpoint mapper and the vLLM-Omni rollout adapter (RFC #420 stack 7/7)#575fusheng-ji wants to merge 39 commits into
fusheng-ji wants to merge 39 commits into
Conversation
fusheng-ji
requested review from
EthanZero2Hero,
Flink-ddd,
KJLdefeated,
bitborne,
inaniloquentee and
ryankert01
as code owners
October 22, 2026 16:58
|
Important Review skippedToo many files! This PR contains 173 files, which is 73 over the limit of 100. To get a review, reduce the PR to 100 files or fewer by splitting it into smaller PRs or changing its base branch. Upgrade to a paid plan to raise the limit. This review couldn't start because sufficient usage credits or metered capacity aren't available. Add credits or update usage-based reviews in the billing tab, then retry.
 |
…fixed-order Triton GEMM Signed-off-by: Wenbo Ji <36562829+fusheng-ji@users.noreply.github.com> (cherry picked from commit 650e2148c6f0e5dcdef136537e0e62f745020273) Signed-off-by: Wenbo Ji <36562829+fusheng-ji@users.noreply.github.com> (cherry picked from commit d099c0b)
Signed-off-by: Wenbo Ji <36562829+fusheng-ji@users.noreply.github.com> (cherry picked from commit b41f868)
…il-closed test instead of a skip Signed-off-by: Wenbo Ji <36562829+fusheng-ji@users.noreply.github.com> (cherry picked from commit 74e5b83)
…l keys Review finding (RL-Align#568): jointly permuting x and g changed dW/dbias because the token axis was reduced in call order. All projection ops (audio in/out, text context, Q/K/V) now take an optional logical_keys [T,2|3] (negative first column = padding); rows are reduced in ascending key order with padding dropped, active keys must be unique. Key format follows RL-Align#562. Adds the reviewer's repro plus permutation/padding tests for every row and Q/K/V. Co-authored-by: Yitao Wang <75578057+walker-ai@users.noreply.github.com> Signed-off-by: Wenbo Ji <36562829+fusheng-ji@users.noreply.github.com> (cherry picked from commit d2941f0)
…ready ordered rows Follows the canonical-row design of RL-Align#562: the Q/K/V node of the block graph supplies (batch row, position) keys so its dW is reduced in logical order, and rows that already arrive in canonical order are reduced in place (same bytes, no gather of x and g). Co-authored-by: Yitao Wang <75578057+walker-ai@users.noreply.github.com> Signed-off-by: Wenbo Ji <36562829+fusheng-ji@users.noreply.github.com> (cherry picked from commit 0da0583)
…block 0 node-by-node and 50-block/generation gates Squashes the one-block branch (its row-by-row history: timestep sinusoid, RMSNorm, AdaLN rows, sequence/tensor parallel norm rows, final AdaLN, one block, 50-block qualification) onto the current test-h3 so the pull request carries a reviewable, signed history. RFC RL-Align#420. Signed-off-by: Wenbo Ji <36562829+fusheng-ji@users.noreply.github.com>
…pull request Signed-off-by: Wenbo Ji <36562829+fusheng-ji@users.noreply.github.com>
Signed-off-by: Wenbo Ji <36562829+fusheng-ji@users.noreply.github.com> (cherry picked from commit eaba5efe3df971ea9d1a969f53e48f09e09ed176) Signed-off-by: Wenbo Ji <36562829+fusheng-ji@users.noreply.github.com> (cherry picked from commit 7da0574)
Signed-off-by: Wenbo Ji <36562829+fusheng-ji@users.noreply.github.com> (cherry picked from commit b45190d08fc033c7e63ad821d3918eb32dd454f6) Signed-off-by: Wenbo Ji <36562829+fusheng-ji@users.noreply.github.com> (cherry picked from commit b5536d6)
Signed-off-by: Wenbo Ji <36562829+fusheng-ji@users.noreply.github.com> (cherry picked from commit ad1925045b522772bf56f8468b23cca47f04156b) Signed-off-by: Wenbo Ji <36562829+fusheng-ji@users.noreply.github.com> (cherry picked from commit f9041e8)
Signed-off-by: Wenbo Ji <36562829+fusheng-ji@users.noreply.github.com> (cherry picked from commit b44540e)
…ody) Signed-off-by: Wenbo Ji <36562829+fusheng-ji@users.noreply.github.com> (cherry picked from commit eab901b)
…g provider, LoRA gate and CPS transition RFC RL-Align#420 rows h3_runtime_modes, h3_ablation_overlay, diffusers_runtime_adapter_h3, h3_training_provider and h3_lora_gradient_gate, plus the layout-invariant CPS transition the training provider uses. full_model_execution gains native_boundaries so a run can keep selected boundary modules on provider arithmetic (ablation). Signed-off-by: Wenbo Ji <36562829+fusheng-ji@users.noreply.github.com> (cherry picked from commit 65d7fff)
… weight loader, tests, doc Signed-off-by: Wenbo Ji <36562829+fusheng-ji@users.noreply.github.com> (cherry picked from commit d12a6bb26566fae58a309e76916af9aae767d487) Signed-off-by: Wenbo Ji <36562829+fusheng-ji@users.noreply.github.com> (cherry picked from commit b316869)
…ith H100 evidence Signed-off-by: Wenbo Ji <36562829+fusheng-ji@users.noreply.github.com> (cherry picked from commit 0734a32ee0c427790a244c7f83e826ba9fe66393) Signed-off-by: Wenbo Ji <36562829+fusheng-ji@users.noreply.github.com> (cherry picked from commit c2c2270)
JSON and figure: runtime modes, LoRA gradients against FP64, CPS transition against the scheduler formula and verl-omni, and the 33B model under off / strict / audit / ablations. Signed-off-by: Wenbo Ji <36562829+fusheng-ji@users.noreply.github.com> (cherry picked from commit 3f0bbe6)
Signed-off-by: Wenbo Ji <36562829+fusheng-ji@users.noreply.github.com> (cherry picked from commit 1e3a2ea)
… and lint Signed-off-by: Wenbo Ji <36562829+fusheng-ji@users.noreply.github.com> (cherry picked from commit d9b0c91)
Signed-off-by: Wenbo Ji <36562829+fusheng-ji@users.noreply.github.com> (cherry picked from commit 81515ae)
Signed-off-by: Wenbo Ji <36562829+fusheng-ji@users.noreply.github.com> (cherry picked from commit 67af796)
Signed-off-by: Wenbo Ji <36562829+fusheng-ji@users.noreply.github.com> (cherry picked from commit 2c08c25)
Signed-off-by: Wenbo Ji <36562829+fusheng-ji@users.noreply.github.com> (cherry picked from commit 58b724b)
…age import Signed-off-by: Wenbo Ji <36562829+fusheng-ji@users.noreply.github.com>
Signed-off-by: Wenbo Ji <36562829+fusheng-ji@users.noreply.github.com> (cherry picked from commit e452447)
….path Signed-off-by: Wenbo Ji <36562829+fusheng-ji@users.noreply.github.com>
…de collective, 2-node H100 evidence Signed-off-by: Wenbo Ji <36562829+fusheng-ji@users.noreply.github.com> (cherry picked from commit 7f1296a970efb0cfcebd9300ff92551de08cf311) Signed-off-by: Wenbo Ji <36562829+fusheng-ji@users.noreply.github.com> (cherry picked from commit d56c391)
Signed-off-by: Wenbo Ji <36562829+fusheng-ji@users.noreply.github.com> (cherry picked from commit 33cbcd6)
Signed-off-by: Wenbo Ji <36562829+fusheng-ji@users.noreply.github.com> (cherry picked from commit 5f4e34c)
Signed-off-by: Wenbo Ji <36562829+fusheng-ji@users.noreply.github.com> (cherry picked from commit d086b5c)
…hard leaves Signed-off-by: Wenbo Ji <36562829+fusheng-ji@users.noreply.github.com> (cherry picked from commit 5ae9f5e)
RFC RL-Align#420 rows h3_checkpoint_mapper (Diffusers <-> vLLM-Omni names and layouts, all 638 tensors, bitwise round trip, fail-closed on unmapped names) and vllm_runtime_adapter_h3 (strict fixed-order kernels behind qkv_proj / out_proj / fc1 / fc2, with read-back). Signed-off-by: Wenbo Ji <36562829+fusheng-ji@users.noreply.github.com> (cherry picked from commit 7b08b63)
…mni adapter Real checkpoint: 638 tensors map and round trip bitwise; fused qkv equals separate Q/K/V, fc1 halves, FP64 error and time per projection against torch. Signed-off-by: Wenbo Ji <36562829+fusheng-ji@users.noreply.github.com> (cherry picked from commit b517d13)
fusheng-ji
force-pushed
the
feat/h3-checkpoint-mapper-vllm
branch
from
October 22, 2026 21:04
40569fe to
a6c4f17
Compare
… block and time the full model CandidateOps attention=auto uses H3FullAttentionOp when the strict _H3_C extension is built (bounded workspace, same RL-Align#240 arithmetic) and the interim DeterministicAttentionOp otherwise; h3_full requires it, interim forces the old op. tools/validation/models/h3_full_speed.py times one full step. Signed-off-by: Wenbo Ji <36562829+fusheng-ji@users.noreply.github.com>
fusheng-ji
added a commit
to fusheng-ji/RL-Kernel
that referenced
this pull request
Oct 23, 2026
…#567 RL-Align#568 RL-Align#570 RL-Align#571 RL-Align#574 RL-Align#575 Test summaries (tests.json), runner outputs and figures measured on NVIDIA B200 (sm100), driver 580, CUDA 13.0, torch 2.13.0+cu130, MiniMax-H3@42ed227, clean trees at the PR heads eab901b, 3cf51bf, 6de320d, eb7840b, af3dcae and 4f7c5d1. Figures use B200 axis labels. No host names, paths or scheduler identifiers. Signed-off-by: Wenbo Ji <36562829+fusheng-ji@users.noreply.github.com>
fusheng-ji
added a commit
to fusheng-ji/RL-Kernel
that referenced
this pull request
Oct 23, 2026
…jectory diag summary - h3-fingerprint-trace-trajectory-h100/tests.json (77 passed, eab901b) - environment.json for RL-Align#574/RL-Align#575 H100 adapter reproducibility pins - dual_trajectory_diag_summary.json (H100/B200 diag strict 0/16 on seeds 11/22/33) Signed-off-by: Wenbo Ji <36562829+fusheng-ji@users.noreply.github.com>
fusheng-ji
added a commit
to fusheng-ji/RL-Kernel
that referenced
this pull request
Oct 23, 2026
…with the H100-pinned vllm-omni and verl-omni Same heads (af3dcae, 4f7c5d1), with vllm-omni 12e92800 and verl-omni efb9dc7f as recorded in the H100 environment files, replacing the first B200 run that used newer checkouts. Test summaries record the dependency revisions. Signed-off-by: Wenbo Ji <36562829+fusheng-ji@users.noreply.github.com>
Flink-ddd
added
platform: cuda
This was referenced Oct 25, 2026
Signed-off-by: Wenbo Ji <36562829+fusheng-ji@users.noreply.github.com>
Signed-off-by: Wenbo Ji <36562829+fusheng-ji@users.noreply.github.com>
Signed-off-by: Wenbo Ji <36562829+fusheng-ji@users.noreply.github.com>
Signed-off-by: Wenbo Ji <36562829+fusheng-ji@users.noreply.github.com>
Signed-off-by: Wenbo Ji <36562829+fusheng-ji@users.noreply.github.com>
…pper-vllm Signed-off-by: Wenbo Ji <36562829+fusheng-ji@users.noreply.github.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
h3_checkpoint_mapper: Diffusers <-> vLLM-Omni native names and layouts.transformer_blocks.N->blocks.N,token_refiner.refiner_blocks.N->token_refiner.blocks.N;to_q/k/v-> one fusedattn.qkv_proj([Q;K;V]rows);to_out.0->out_proj;ff.net.0.proj(value-first) ->mlp.fc1(gate-first, halves swapped);ff.net.2->mlp.fc2; model-level names from vLLM-Omni'sfasth3table. All 638 checkpoint tensors map and the round trip is bitwise; unmapped names and incomplete q/k/v sets fail closed.vllm_runtime_adapter_h3:install_h3_rollout(model, collective=None, tp=1)replaces the four fused linears' forwards (returning vLLM's(output, None)).qkv_proj/fc1useh3-projection-gemm-v1(a fused column equals the same column alone, tested);out_proj/fc2use the canonical row-parallel reduction ofh3-parallel-v1. Wrong dtype, non-CUDA weights or a model without those modules raise in strict mode. Attention, rotary, QK norm, SwiGLU and AdaLN stay vLLM-Omni's and are reportedexternal;handle.remove()restores.Prior art & reuse decision
fasth3name tablesvllm_omniis importableh3-projection-gemm-v1(#568),h3-parallel-v1(#569)Results (H100 / Hopper)
Full 33B step time (forward + backward, gradient checkpointing, one H100) — strict graph binds
h3_full_attention(#553) inCandidateOpswhen_H3_Cis built; provider is diffusers/cuBLAS. Median of 3 repeats; torch 2.13.0+cu130,MiniMax-H3@42ed227, runnertools/validation/models/h3_full_speed.py, measured at head4f7c5d11(#575 stack, includes #571).Strict uses the bounded-workspace #553 attention kernel (not the interim quadratic
DeterministicAttentionOp); 8349 tokens fit in ~65 GiB. This is still far above provider time (launch-bound strict path plus interim projection/GEMM rows in the stack); it does not meet the “R/R step time not worse than P/P” acceptance bar.Measured on NVIDIA H100 (93 GiB, sm90), CUDA 13.0, torch 2.13.0+cu130, with the real pinned
MiniMax-H3@42ed227checkpoint (14 shards), at head4f7c5d11. Runner:tools/validation/models/h3_mapper_vllm_report.py. Data and figure on the fork'spr_assetsbranch (report.json, figure.png, environment.json).Checkpoint mapper: all 638 checkpoint tensors map (534 native tensors after q/k/v fusion, all names unique), and
native_to_diffusers(diffusers_to_native(x))is bitwisexwith dtypes kept. One pass takes about 5.4 s forward and 5.5 s back (CPU, 53 groups).Adapter (blocks 0, 25, 49 of the real checkpoint, 70 and 925 tokens, BF16): the fused
qkv_projequals separate Q, K, V projections bitwise; the mappedfc1equals the Diffusersff.net.0.projoutput with halves swapped, bitwise; all four projections are bitwise repeatable. Relative error to FP64 is 1.66e-3 for every projection, within 1e-7 of torchF.linear(qkv_projandfc1are identical to torch's error).Speed against torch
F.linear(adapter time / torch time; this is slower, not faster):qkv_proj1.03x (70 tokens) and 1.47x (925),fc11.00x and 1.14 - 1.23x,out_proj10.3x and 4.9 - 5.1x,fc24.9x and 3.3 - 3.5x. The row-parallel reduction (out_proj,fc2) pays for its fixed 8-unit order; at 70 tokens this is launch overhead (about 0.44 ms vs 0.04 - 0.09 ms).TP invariance of the row-parallel reduction itself (TP 1/2/4/8) is measured in #569; it is not repeated here.
Results (B200 / Blackwell)
Measured on NVIDIA B200 (179 GiB, sm100), driver 580.126, CUDA 13.0, torch 2.13.0+cu130, pinned
MiniMax-H3@42ed227weights, clean tree at4f7c5d11. Optional dependencies pinned to the revisions recorded for the H100 run (environment.json): vLLM-Omni12e92800(vllm_omniimported from that checkout).Tests: 8 passed (8 collected), none skipped, including the vLLM-Omni name-table comparison;
tests.json. Report:report.json.native_to_diffusers(diffusers_to_native(x))is bitwisex. One pass takes 18.7 s forward and 8.9 s inverse.fc1equals diffusers' half swap; relative error against FP64 0.00166 to 0.00166, equal totorch.nn.functional.linear's (0.00166 to 0.00166).F.linear(the adapter uses the fixed-order kernels and the canonical row-parallel reduction):qkv_proj1.6x to 1.7x (median 1.6x),fc11.4x to 1.7x (median 1.5x),fc24.3x to 7.5x (median 5.9x),out_proj5.8x to 11.4x (median 8.5x).Tests
tests/models/minimax_h3/test_h3_checkpoint_mapper_vllm.py(8 tests): 638-tensor mapping and unique native names, bitwise round trip,fc1half swap and q/k/v order, fail-closed unmapped names, grouped layout, vLLM-Omni table comparison, adapter patch / restore, adapter strict rejection. 8 pass on H100 withvllm_omniinstalled; the table comparison is skipped when it is not importable. The #569 regression (test_h3_parallel.py,test_h3_ws2_processes.py) passes on this branch on H100 (74 tests, 4 GPUs).B200: 8 passed, with vLLM-Omni importable.
Scope & Known limitations
Not run with
tp > 1, and no single-node 8-GPU Hopper run. The adapter's row-parallel path takes a deterministic collective fortp = 2 / 4 / 8, but the tests and measurements here usetp = 1; the TP-invariance of the reduction is shown in [WS2][CUDA][MiniMax-H3] ws2 tensor / sequence / context parallel and the hierarchical cross-node collective (RFC #420 stack 3/5) #569, itself measured on 2 nodes x 4 H100 only. We lack a single-node 8-GPU Hopper machine; anyone who has one is welcome to run the adapter attp = 8and report back.Only the four fused linears are bound; attention, rotary and norms are not.
The pinned vLLM-Omni version is the one installed for these tests; it is not a CI dependency, so the name-table test is skipped there.