[sysuTobo]

Summary

Reference golden for the hyper-connection gated residual of Qwen3.8-Flash-Next (#1103), and the on-device gate that consumes it.

• tools/accuracy/dump_qwen38fn_hc_golden.py — records Qwen4ExpTextGatedResidual (transformers 5.16.1 qwen4_exp) at the real frozen-config geometry: for the combine block the grouped (1+w) normed stream, the mixed stream, the un-normed hyper echo and the injection weights; for the top-level readout the use_combine=False mixer output (the readout — there is no final-norm tensor).
• test_data/qwen38fn-hc-golden.safetensors — 2 cases (S=6, S=10), 19 tensors.
• Geometry: real frozen-config geometry — hidden_size 2560, hc_count 4 (10240-wide stream), hc_lowrank 320 — with no weights dropped: all four weights of the use_combine=True block (norm + down + up + inject) and all three of the use_combine=False mixer (norm + down + up; that variant builds no injection head) are stored, because the mix GEMMs are the component under test. The captured activations cover the grouped (1+w) RMSNorm (group size 2560 on a 10240-wide input), the silu->sigmoid mix gate, /hc_count on both the mix and inject paths, the mean (not sum) combine and the (0, 2) injection range.
• pegainfer-kernels/tests/qwen4_hc_block_parity.rs — the consumer: replays the block on device against every recorded tensor, plus four negative controls that guard the tolerance itself. See Consumer below.

Validation

The dump asserts nothing; it is a pure recorder. Everything it stores is the reference's own output (Qwen4ExpTextGatedResidual forwards) at the real geometry — nothing is re-derived or re-checked here. The consumer gate is what replays these tensors against the kernel and asserts agreement.

$ python tools/accuracy/dump_qwen38fn_hc_golden.py --out test_data/qwen38fn-hc-golden.safetensors
wrote test_data/qwen38fn-hc-golden.safetensors (25.90 MiB, 19 tensors)

The dump side is fixture + script only. The consumer commit adds three feature-gated kernels, their FFI and their ops wrappers behind a new qwen4 cargo feature; no default-feature code path changes (cargo check -p pegainfer-kernels compiles none of the new .cu) and no model crate reads the new ops yet.

Consumer

pegainfer-kernels/tests/qwen4_hc_block_parity.rs is the only reader of test_data/qwen38fn-hc-golden.safetensors. It replays Qwen4ExpTextGatedResidual.forward — modeling_qwen4_exp.py:959-968 in transformers 5.16.1 — on device through the shared GPU surface, at the real frozen geometry and both recorded sequence lengths, and compares every tensor the dump stored.

Replay semantics against modeling_qwen4_exp.py:959-968

Reference	Replay	Recorded tensor
:959 hc_norm(hyper_input) — grouped (1+w) RMSNorm, group_size = hidden_size 2560 over a 10240-wide stream	4x rms_norm_batch_offset_into, one per branch with that branch's own 2560-wide slice of the norm weight, then an on-device memcpy_dtod assembly into the row-major stream	normed
:960 F.silu(input_mix_weight_down(n) / hc_count)	gemm_into_checked (down, [320, 10240]) then hc_silu_scaled_into(scale = 1/4) — the division is folded into the scale and is exact, because hc_count is a power of two	intermediate
:961 torch.sigmoid(input_mix_weight_up(g))	gemm_into_checked (up, [10240, 320]) then hc_sigmoid_affine_into(scale = 1, gain = 1)	intermediate
:962-965 unflatten to [S, 4, 2560], gate * normed, .mean(dim=-2)	hc_mul_branch_combine_into(branches = 4, combine_scale = 1/4) — bf16 product per branch, f32 accumulate over branches, one cast out	mixed
:966-967 use_combine=False builds no injection head and returns mixed_input alone	the same replay against the mixer weights (mixer_norm / mixer_down / mixer_up), asserting inject is absent	mixed_only
:968 2 * torch.sigmoid(block_inject_weight(n) / hc_count)	gemm_into_checked ([4, 10240]) then hc_sigmoid_affine_into(scale = 1/4, gain = 2) — the divisor and the gain are both exact powers of two, so the sigmoid's cast is the only rounding	inject

The three elementwise steps had no entry point, so they are added under the new qwen4 cargo feature: hc_silu_scaled_into, hc_sigmoid_affine_into, hc_mul_branch_combine_into, over the kernels in pegainfer-kernels/csrc/qwen4/qwen4_hc_block.cu. Each lands its bf16 roundings where the reference lands them — the contract is documented on the kernel bodies. silu_f32 is the tree's x / (1 + expf(-x)), not x * sigmoid(x), which keeps it bitwise with the five other silus under csrc/ and with the default-feature two-launch equivalent (scale_bf16_in_place then silu_mul_batch_into against a buffer of ones) that this op fuses into one launch. All three wrappers validate extents with checked_extent before launching; all three launchers bound their element counts and form the grid in 64 bits.

Measured

RTX 5060 (sm_120), CUDA 12.9.1 (dev image pegainfer-dev:cu132) — test result: ok. 6 passed; 0 failed. Worst element per tensor over both cases:

Tensor	case 0 (S=6)	case 1 (S=10)	worst
normed	0.000e0 — 61440 / 61440 bit-identical	9.766e-4	9.77e-4
mixed	9.766e-3	7.812e-3	9.77e-3
inject	7.812e-3	7.812e-3	7.81e-3
mixed_only	7.812e-3	9.766e-3	9.77e-3

That is one to two bf16 ulps: one flipped rounding in a chain of seven (norm, down GEMM, SiLU, up GEMM, sigmoid, branch product, branch mean). Only two reductions can move it — the grouped norm's f32 sum of squares runs in FlashInfer's vector order where torch runs in its own, and cuBLAS accumulates the three projections in a different order than the torch 2.14.1+cpu bf16 matmul the fixture was dumped with.

Tolerance and its derivation. The bound is abs(got - want) <= ATOL + RTOL * abs(want) with ATOL = RTOL = 4.0e-2. Against the worst element any of the four tensors shows (9.77e-3): ATOL alone is 4.1x it (4.0e-2 / 9.77e-3), and at |want| = 1 the whole bound is 8.0e-2, i.e. 8.2x it. Below |want| = 1 the RTOL term shrinks towards ATOL, so the ratio never falls under 4.1; above it the ratio only grows. Four-to-eight times the disagreement actually observed leaves room for a machine whose cuBLAS picks a different accumulation order and flips once more at every stage — and, per the controls below, no room for a wrong block.

The 6 tests

Test	What it pins
fixture_provenance_is_the_frozen_config	fixture metadata + the four geometry values; needs no device, so a missing GPU cannot skip it
hyper_connection_block_matches_golden	all four recorded tensors, both cases, both modules
negative_sum_over_branches_fails	.mean(-2) is a mean, not a sum
negative_missing_hc_divisor_fails	/hc_count is present on the mix path and on the injection weights
negative_swapped_or_dropped_mix_activations_fail	SiLU on the down output and sigmoid on the up output — swapped, and each dropped on its own
negative_plain_weight_norm_fails	the norm is (1+w), not plain w

The four negative controls guard the tolerance itself, not the arithmetic. Each is checked against exactly the recorded tensors it can move — a control that also "failed" on a tensor it never touches would say nothing about whether the bound is tight, and one that failed on nothing would mean the control never reached the arithmetic — and a control that still passes is a test failure. Measured spread: the tightest, sigmoid where the reference puts the SiLU on the down output, moves mixed by 4.96e-1 at its worst and puts 5676 of 15360 elements (37%) outside the bound; the coarsest, plain w for (1+w), moves normed by 4.84 and puts 98934 of 102400 outside. Every control fails on a large fraction of its elements rather than on a marginal few, so the bound discriminates rather than absorbs.

Provenance

Asserted, not assumed, and it needs no device: format_version == 2, transformers_version == 5.16.1, the recorded config path, and the fixture's config_sha256 held against the bytes of pegainfer-qwen4/tests/frozen_config.json. hc_count, hidden_size, hc_lowrank and rms_norm_eps are then read back out of those same pinned bytes and held against the geometry the gate replays, so a config that moved under the fixture cannot pass on the hash alone. A fixture from another checkpoint or another semantics fails here instead of being skipped. The replay tests skip without a device; PEGAINFER_REQUIRE_GPU=1 turns that into a failure.

What remains ungated

• The scatter half of the block. modeling_qwen4_exp.py:1236-1237 — hyper_input + (sublayer * injection_weights).flatten(-2), the fold of a sublayer output back into the 4-branch stream — is not replayed. The golden records this block's own inputs and outputs, not a layer's, so it carries no sublayer tensor. inject is checked here, so the scatter's only per-branch coefficient is already held; the fold itself waits on the composition work in [Model] Qwen3.8-Flash-Next: gated residual #1108.

• No CI job enables the qwen4 feature yet. The clippy/test matrix carries "", gemma4, qwen35 and k3, so nothing in CI builds csrc/qwen4/ and the gate is manually run:

  ./docker/dev.sh run cargo test --release --locked -p pegainfer-kernels \
    --features qwen4 --test qwen4_hc_block_parity -- --nocapture
  

Merge notes

• Sibling-PR collisions are union-resolutions. qwen4 = [] in pegainfer-kernels/Cargo.toml, is_qwen4_source in build.rs, the csrc/qwen4/ directory and the src/ffi/qwen4.rs module slot are added verbatim by the in-flight QSA PRs too. Whichever branch lands second takes the union: the feature flag, the gating arm and the mod / pub use lines are identical, and the per-file .cu / .rs bodies do not overlap.
• feat(kernels): add Qwen4's grouped (1+w) RMSNorm #1158. This gate's grouped norm is four rms_norm_batch_offset_into launches because rms_norm_batched_offset_cuda pins stride_input to hidden_dim and cannot address a branch of a row-major stream. If/when feat(kernels): add Qwen4's grouped (1+w) RMSNorm #1158 lands rms_norm_grouped_offset_cuda, production switches to that single-launch grouped entry point and the on-device assembly disappears with it; this gate's four-call form is bit-identical on that degenerate path, so the numbers above do not move when the switch happens.

Refs #1103

[sysuTobo]

Code review

范围：2 文件、无运行时数据流（dump 脚本 + fixture）。逐文件认领：tools/accuracy/dump_qwen38fn_hc_golden.py（144 行，新增）、test_data/qwen38fn-hc-golden.safetensors（22 张量，BF16/I32/U8）。
正向路径：mix 门（silu→sigmoid 两级）、mean 合流、per-branch inject（2·sigmoid(·/hc_count)）的组合被真守卫——突变实测 mean→sum、sigmoid→silu 均令断言 RED（dump_qwen38fn_hc_golden.py:73-76,118-119）；缩几何保真（group 数=4 保留：真实 4×2560 → 缩为 4×512，分组语义有代表性）；header 22 张量名/shape/dtype 与脚本写入逐一一致；本机重跑与 committed fixture 逐字节相同（file sha256 d987100c…）。已独立验证 committed normed == 分组 (1+w) 重推导、!= 非分组，故 fixture 数据本身正确。

ID	级别	问题	证据
M1	M	分组 (1+w) RMSNorm（PR 自述「核心」）在 rederive 里直接委托 module.hc_norm，未独立重推导 = 同源比较。突变实测：把 norm 去分组（group_size→None）或改 eps，dump 期断言仍 GREEN（grouped 与 ungrouped 的 normed 实测 maxdiff 0.375，性质已变却不红）；而 mean→sum、sigmoid→silu 都 RED。即 mix/合流/inject 的组合被真守卫，唯独核心分组语义是空转。docstring「reference formulas are re-derived」因此名不副实。	dump_qwen38fn_hc_golden.py:72,15
M2	M	config_digest 规范化串漏 RMS_EPS——它是 config.rms_norm_eps、直接缩放分组 norm 输出。改 eps 而 config_sha256 不红 = 假 provenance，且 fixture 任何地方都没钉 eps。（姊妹 #1150 把影响输出的 rope_theta 纳入了 digest，此处不一致。）	dump_qwen38fn_hc_golden.py:60-62
L1	L	mixed_only（顶层 mixer / readout 变体，use_combine=False）被存下却无任何 dump 期断言：既未重推导、也未查 finite（断言只覆盖 combine 的 mixed/inject/hyper_out）。实测 finite 且与 mixed 可区分（maxdiff 0.99），但无守卫——readout 是模型出口，盲存风险高于普通中间量。	dump_qwen38fn_hc_golden.py:115,131,118-123
L2	L	与 #1150 最终态的约定分歧：config_digest 缺 sibling 那句 docstring（「sha256 over the frozen geometry…」），FORMAT_VERSION 缺「layout 变更时 bump、gate 加载即断言」注释。	dump_qwen38fn_hc_golden.py:46,60
L3	L	assert torch.equal(hyper_out, hyper) 近乎恒真：参考 forward 原样返回入参（identity），等于自比。它仍钉住「回显未改动流」契约（若参考改成返回 normed 会红），价值边际——建议保留、不改。	dump_qwen38fn_hc_golden.py:120

case 覆盖：两 case 仅在 token（batch）轴变化（S=6/10），确认逐 token 点处理、无跨 token 泄漏；该模块无 seq_len 分支，故无需 #1150 那种 6-case 截断矩阵。唯一未变的是几何（config 级、非 per-case），属单几何 scope 的已知边界，非缺陷。

最小加固：修 M1（在 rederive 里独立重推导分组 (1+w) norm、不再调用 module.hc_norm，使现有 mixed/inject 断言连同一条新增 norm 断言一并看守分组/eps）、M2（digest 纳入 eps）、L1（mixer 变体走同一独立重推导 + finite 断言）、L2（补 docstring/注释）；L3 保留为已知契约断言。M1/M2 会改动 config_sha256，fixture 需重生成（本机逐字节可复现）。

[sysuTobo]

修改措施

Pushed 05a02a00。

• M1 已修：rederive 不再委托 module.hc_norm，改为独立重推导分组 (1+w) RMSNorm（hc_count 组 × hidden，fp32 per-group rsqrt(mean+eps) → ×(1+w) → 回 dtype），并新增一条 mod_normed == normed 断言直接看守分组语义。突变复测：去分组（group_size→None）现令 norm/mix/inject 三条断言全部 RED（修复前 GREEN）；mean→sum 仍 RED。dump_qwen38fn_hc_golden.py:72-95,141,143
• M2 已修：config_digest 规范化串纳入 eps={RMS_EPS}；并在建模后断言 combine/mixer.hc_norm.eps == RMS_EPS，使 digest 声称的 eps 与模块实际所用一致（漂移即 abort，实测触发）。dump_qwen38fn_hc_golden.py:62-65,114-117
• L1 已修：mixer（use_combine=False）readout mixed_only 现走同一独立重推导并断言 mixed_only == r_mixed_only，另加 isfinite(mixed_only)。dump_qwen38fn_hc_golden.py:90-94,140,146,149
• L2 已修：补齐 config_digest docstring 与 FORMAT_VERSION 注释，与 feat(qwen4): add Qwen3.8-Flash-Next QSA indexer golden #1150 最终态对齐；docstring 改口径为「含分组 norm 的独立重推导」，不再名不副实。dump_qwen38fn_hc_golden.py:15-17,47,63
• L3 不修（保留）：assert torch.equal(hyper_out, hyper) 虽是 identity 自比，但钉住「回显未改动流」契约（参考若改返回 normed 会红），价值边际但无害，按评审结论保留。dump_qwen38fn_hc_golden.py:147

复验：fixture 重生成（22 张量，torch 2.14.1+cpu / transformers 5.16.1 / CPU），provenance config_sha256=7a8402e4…、ref_transformers=5.16.1、format_version=1；2 case（S=6/10）重推导逐位自洽（rederivation_ok=True，inject_mean 0.9887 / 0.9619）。与修复前 committed 相比仅 config_sha256 变（e3f5d32e → 7a8402e4），normed/mixed/inject/mixed_only/权重全部逐字节相同（独立分组重推导 == 模块输出，故 golden 数据未变、只加固了守卫）。file sha256 e7a996f1…，714626 bytes（尺寸不变）。

[sysuTobo]

Code review

对本 PR 消费端提交（e5fe036b，即 HC golden 的 device 侧 parity gate）做了一轮自审。证据行号中，「原状态」指修复前的位置，「现状」指提交后的位置。

ID	级别	一句话	证据(文件:行)
F1	M	三个新增 launch wrapper 跳过了仓库强制的 checked_extent 发射前校验，逻辑尺寸超出 backing allocation 时会越界读写	原状态 pegainfer-kernels/src/ops/qwen38fn_hc.rs:67-70,109-112,172-174；契约在 src/tensor.rs:580-593，惯用写法在 src/ops/elementwise.rs:1188-1189，同类先例 src/ops/norm.rs:207-209,270-272
F2	M	silu_f32 写成 x * sigmoid(x)，与 torch 及仓库其余五处 silu 的 x / (1 + expf(-x)) 不同形式，f32 末位可差 1 ulp 并翻转 bf16 cast	原状态 csrc/qwen38fn/qwen38fn_hc_block.cu:53；对照 csrc/shared/elementwise.cu:304、csrc/shared/norm.cu:69、csrc/shared/fused_proj.cu:59、csrc/qwen35/conv1d.cu:59,119、csrc/kimi_k2/kimi_marlin_wna16.cu:114,142
F3	M	hc_silu_scaled_into 与默认 feature 已有的 scale_bf16_in_place + silu_mul_batch_into(x, ones) 组合等价（bitwise），而模块文档声称「standalone SiLU 不存在」	原状态 src/ops/qwen38fn_hc.rs:5-7、csrc/qwen38fn/qwen38fn_hc_block.cu:14-17；已有实现在 src/ops/elementwise.rs:1184（scale）与 :773（silu_mul），kernel 体 csrc/shared/elementwise.cu:392-401,295-307
F4	L	两个 elementwise launcher 的 grid_for(n) 里 n + 255 没有上界，而 combine launcher 有 dims_fit_i32，两处防护不对称	原状态 csrc/qwen38fn/qwen38fn_hc_block.cu:101,117-119,132-134 vs :106,155-160
F5	L	hc_silu_scaled_into / hc_sigmoid_affine_into 平铺 re-export 进 ops::，前缀 hc 无处解释	原状态 src/ops.rs:239-246
F6	L	ATOL 文档写「factor of six over the worst observed」，但它自己引用的数字只支持 4.1（0.04 / 9.77e-3），margin 不可复算	原状态 tests/qwen38fn_hc_block_parity.rs:118-120
F7	L	dims_fit_i32 的注释说「every dimension is bounded」，实际只约束乘积	原状态 csrc/qwen38fn/qwen38fn_hc_block.cu:103-106
F8	L	combine kernel 内层的 bf16 rounding（__float2bfloat16(product)）没有独立 pin，只由 mixed 的端到端比对间接覆盖	csrc/qwen38fn/qwen38fn_hc_block.cu:96（现状 :107）
F9	L	三个新 op 的拒绝路径（ensure_finite、形状不符、checked_extent、checked_i32）没有 #[cfg(test)] 单测	src/ops/qwen38fn_hc.rs:30-44 及各 wrapper 的 ensure! 分支
F10	L	qwen4 = [] / is_qwen38fn_source / csrc/qwen38fn/ / src/ffi/qwen38fn*.rs 与在飞的 QSA 兄弟 PR 逐字撞车	pegainfer-kernels/Cargo.toml:40、build.rs:773,2296、csrc/qwen38fn/、src/ffi/qwen38fn_hc.rs
F11	L	两处引用未合并的 #1158（rms_norm_grouped_offset_cuda）是前向引用，措辞像是既成事实	原状态 tests/qwen38fn_hc_block_parity.rs:35-38,421-422
F12	info	工作区里 .qoder/ 未被 gitignore，git add -A 会把它带进提交	git status 输出 ?? .qoder/

修改措施

F1 已修。 三个 wrapper 都加了 checked_extent：silu 查 x 与 out（src/ops/qwen38fn_hc.rs:77-78），sigmoid 同样查两个（:118-119），combine 按要求查 a、b、out（:184-186）。elementwise 两个 op 把 out 也一并查了——形状 ensure! 只保证逻辑尺寸相等，backing 长度可以不等，写出边界是真实风险；这与 src/ops/norm.rs:207-209 的既有写法一致。

F2 已修。 silu_f32 改成 x / (1.0f + expf(-x))（csrc/qwen38fn/qwen38fn_hc_block.cu:64），并在注释里写明为什么必须用这个形式（与树内五处 silu bitwise 一致，也是 F3 那个两发射等价成立的前提）。按要求重跑了 gate，并刷新了所有实测数字：六个数字一个都没动（normed 9.77e-4 / mixed 9.77e-3 / inject 7.81e-3 / mixed_only 9.77e-3，case0 normed 仍是 61440/61440 bit-identical；四个 negative control 的 4.96e-1 / 5676 of 15360、4.84 / 98934 of 102400 也不变）。这一点本身写进了 ATOL 文档（tests/...:114-120），作为「形式改了、语义没漂」的证据。

F3 已修（选定方案：保留单发射 kernel，把文档改成实话）。 模块文档（src/ops/qwen38fn_hc.rs:3-16）和 kernel 头注释（csrc/...:12-22）现在都写明：默认 feature 上确实存在等价的两发射组合（scale_bf16_in_place 再 silu_mul_batch_into 对全 1 buffer），本 op 的价值是把 scale+silu 融进一次发射；两者 bitwise 相同的原因是参考实现的 scale 是 2 的幂，两发射形式多出来的那次中间 bf16 rounding 因此是精确的。文档同时点明 silu 体不是需要手工同步的第二份实现——它用的就是树内那一份。

F4 已修。 两处一起收口：grid_for 改成 64 位计算并返回 unsigned（csrc/...:118-120），这样 0x7ffffffe + 255 溢出 int 的隐患对三个 launcher 同时消失（combine 原来也有）；另外给两个 elementwise launcher 补上和 combine 同一条 dims_fit_i32 上界（:145、:164），防护现在对称。

F5 已修（选定方案：不改名，只加一行说明）。 src/ops.rs:239-240 在 re-export 处加了注释：hc 即 Qwen4 hyper-connection block（Qwen4ExpTextGatedResidual），这三个 op 是它的 elementwise 尾部，细节见 ops/qwen38fn_hc.rs。

F6 已修。 ATOL 文档改成可从引用数字复算的表述（tests/...:122-130）：四张表里最差的元素是 9.77e-3，ATOL 单独是它的 4.1 倍（4.0e-2 / 9.77e-3），|want| = 1 处整条界 8.0e-2 是 8.2 倍；|want| < 1 时 RTOL 项向 ATOL 收缩，所以倍数不会低于 4.1，|want| > 1 时只会更大。「factor of six」这句删掉了。

F7 已修。 dims_fit_i32 的注释（csrc/...:122-127）现在说的是它真正做的事：约束的是维度的乘积，不逐个约束任何单一维度，也不需要——真正要活下来的是 launcher 传给 grid_for 的计数和 combine kernel 构造的 i32 源下标 row * branches * hidden + j * hidden + c，两者都被 rows * branches * hidden 这个乘积盖住。

F8 / F9 明确留给 #1108（本次不实现）。 F8：combine 内层那次 bf16 rounding 目前只被 mixed 的端到端比对间接覆盖，需要一个独立 pin（把内层 cast 去掉做 negative control，或直接对 gate * normed 的逐元素积建 golden）；这属于把整块拼进 layer 时要一起定的口径，留给 #1108。F9：三个新 op 的拒绝路径缺 #[cfg(test)] 单测（非有限 scale/gain、形状不符、backing 不足、i32 溢出），同样留给 #1108 在 ops 被真实模型消费时补——那时才有第二个调用方来证明这些拒绝分支不是只服务于 gate 的摆设。两项的接手方都是 #1108 那个 composition PR，本条评审意见即为它们的 deferred receipt：本 PR 不实现，也不假装完成。

F10 已写进 PR body。 见 PR body 的 ## Consumer → ### Merge notes：qwen4 = []、is_qwen38fn_source、csrc/qwen38fn/、src/ffi/qwen38fn*.rs 与在飞的 QSA PR 逐字撞车，后落地的一方按 union 解——feature flag、gating 分支、mod / pub use 三处内容相同，各自的 .cu / .rs 文件体不重叠。

F11 已修。 两处 #1158 的措辞都改成条件式（tests/...:35-39、:429-430）：若 #1158 落地 rms_norm_grouped_offset_cuda，生产切到那个单发射 grouped 入口、on-device 拼装随之消失；本 gate 的四发射形式在那条退化路径上与之 bit-identical，所以上面的实测数字不会因为切换而变动。PR body 的 Merge notes 里同样按这个口径写。

F12 遵守。 全程逐文件 git add（九个路径一个一个加），.qoder/ 保持 untracked，提交里 9 files changed, 1352 insertions(+)，没有它。

本次落地的验证

• gate 重跑：./docker/dev.sh run cargo test --release --locked -p pegainfer-kernels --features qwen4 --test qwen38fn_hc_block_parity -- --nocapture → 6 passed; 0 failed（F2 之后重跑，数字见上）。
• ./docker/dev.sh run cargo fmt --all -- --check → 干净。
• ./docker/dev.sh run cargo check --locked -p pegainfer-kernels → 默认 feature 绿，且构建日志显示 csrc/qwen38fn/ 的新 .cu 一个都没编（qwen4 gating 生效）。
• 新增行与新文件的红线扫描（内部标识、私有网段、本机绝对路径、私有目录等）：零命中。pegainfer-kernels/build.rs 里 8 处命中全部落在既有未改动的第三方 include 路径上下文行上，不属于本次 diff。

仍未验证的：CI 上没有 job 开 qwen4，所以这个 gate 只有手跑证据；scatter 半块（modeling_qwen4_exp.py:1236-1237）无 golden 覆盖，等 #1108。

[github-actions]

This pull request has merge conflicts with main. Please rebase it so it can be reviewed and merged.

[sysuTobo]

Rebase

已 rebase 到 main @ 1f17ddb3（含 #1160 QSA indexer、#1176 FlashInfer v0.7.2），并统一到 #1160 合入后的 kernels 布局：

• 核文件 csrc/qwen38fn/qwen38fn_hc_block.cu → csrc/qwen4/qwen4_hc_block.cu，C 入口改为 qwen4_hc_*_cuda
• FFI/ops 追加进 main 已有的 src/ffi/qwen4.rs / src/ops/qwen4.rs，不再各开模块；is_qwen38fn_source 与 feature 行删除，复用 main 的 is_qwen4_source
• gate 改名 tests/qwen4_hc_block_parity.rs；fixture 名不变

源文件里的“兄弟 PR 取并集”段落已按现状改写：qwen4 feature、is_qwen4_source 与 csrc/qwen4/ 已在 main，本 PR 及在飞的 QSA-attn / MoE / PLE 都只是在同一模块的不同函数处追加。

重跑：gate 6 passed（0.27s），默认 feature 的 cargo check --all-targets 绿，cargo fmt --check 干净。