HIP: the prompt GEMM pays a group-32 scale and bias fold every K stage #1600

On AMD Ryzen AI MAX+ 395, Radeon 8060S gfx1151 (40 CUs, 128 GB unified memory, measured 225.6 GB/s read), Ubuntu 26.04, archive HIP 7.1 (clang 21), TensorFold 7c068895, TensorFold/Qwen3.8-Flash-Next-MLX-4bit-MTP.

With the WMMA sparse attention above, a 7.9K-token prompt spends 60% of its time (8.2 s) in affine_wmma_gemm<4> (zig/kernels/hip/tiles/matrix_gemm.hpp) at about 14 TFLOP/s, a quarter of the chip's fp16 WMMA rate. With MLX group 32 equal to the tile's K step (per == 1), every 32-wide stage zeroes a second accumulator set, runs two WMMAs, then folds the group's scale and bias and the activation row sums into the main accumulator.

Proposal

For group sizes equal to the K step, stage dequantized weights (code x scale + bias) as the bf16 B operand so WMMA accumulates straight into the output tile, dropping the per-stage fold, the row sums and the second accumulator set (freeing registers for a second resident block). This changes the sums (the dequantized weight is rounded to bf16 before the product), so it needs the agreed arithmetic gate; the existing path stays selectable.

Gate to agree

FP64 reference per projection class, worst-column error no worse than today's path, teacher-forced top-token differences, kernel contract and limits, cold prefill at 8K/32K before and after.