perf(kimi3): tune small-batch latent projection

[panditsa]

Summary

• tune the gfx950 3584→7168 latent projection selector for graph-captured M=2/4
• use the fused RMSNorm + projection + add path through M=2 and the faster composed path through M=16

Performance

Exact parent c5a50e8f → head 6406405a, 4K/1K TP8/EP8 CUDA graphs, median of three:

M	Parent tok/s	Head tok/s	Change	Parent TPOT	Head TPOT
2	81.94	84.77	+3.45%	23.686 ms	22.864 ms
4	149.18	152.23	+2.04%	25.501 ms	24.954 ms

Isolated graph-captured normalized MoE output tail:

• M=2: 49.02 → 14.48 µs (-70.5%)
• M=4: 49.37 → 30.43 µs (-38.4%)
• M=8: 42.12 → 21.43 µs (-49.1%)
• M=16: 44.74 → 22.29 µs (-50.2%)

The standalone registered M=2/4 projection improves from 30.58/30.54 µs to 22.78/23.19 µs under graph replay.

Tests

• targeted Kimi projection suite: 37 passed, 18 deselected
• all 18 measured 4K/1K requests completed with valid 4096-token prompts and 1024-token outputs
• CUDA-graph capture/replay covered by both the targeted tests and end-to-end runs

Stack

• depends on perf(gemm): extend Kimi AttnRes projection to M4 #1134

[jhinpan]

Exact #1132 M=8 result on current main (9196fa27), testing only the M=3..16 kimi3_latent_projection_add3 change under forced acceptance=1 at 131072/1024 TP8/EP8:

• baseline: 36.80 ms TPOT, 19.7647 tok/s
• candidate: 36.85 ms TPOT, 19.7423 tok/s

The isolated M=8 tail win does not transfer to full-round wall time here (about -0.11% throughput, effectively flat/slightly regressed). I do not plan to duplicate or extract this as a #1132 fix.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 78a97e64da

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[chatgpt-codex-connector]

Keep the one-row fused kernel restricted to M=1

When norm_weight is supplied for a two-row CDNA4 projection, this condition now dispatches to gluon_rmsnorm_linear_add_gfx950, but that kernel launches only over output columns and has no row index or row strides: it reads and writes only the first row. Consequently, the second row of the returned [2, 7168] tensor remains uninitialized. Either retain m == 1 here or extend the fused kernel to process both rows before enabling M=2.

Useful? React with 👍 / 👎.