Skip to content

(WIP) perf(kimi3): join TP MoE reductions - #1144

Draft
panditsa wants to merge 1 commit into
lightseekorg:sanket/wip-k3-iris-attnresfrom
panditsa:sanket/wip-k3-tp-joint-moe
Draft

(WIP) perf(kimi3): join TP MoE reductions#1144
panditsa wants to merge 1 commit into
lightseekorg:sanket/wip-k3-iris-attnresfrom
panditsa:sanket/wip-k3-tp-joint-moe

Conversation

@panditsa

@panditsa panditsa commented Aug 19, 2026

Copy link
Copy Markdown
Contributor

Summary

  • enable joint routed/shared reduction for native TP as well as EP layouts
  • acquire producer-direct outputs from the TP-EP reduction group
  • pass activated shared inputs through the tuned M<=4 A8W4 expert kernel

Performance

Exact parent a1a0f8e7 → head 30cac366, 4K/1K TP8/EP1 CUDA graphs, median of three:

M Parent tok/s Head tok/s Change Parent TPOT Head TPOT
2 97.17 103.64 +6.66% 19.939 ms 18.621 ms
4 175.34 185.64 +5.88% 21.693 ms 20.380 ms

Each run had one first-repetition TTFT disturbance; the two steady repetitions agreed closely and the reported median rejects that startup effect.

Tests

  • TP8 execution policy and reduction-group coverage pass
  • joint projection/output reuse is covered for M=1/2/3/4/8
  • targeted suite: 6 passed, 28 deselected
  • all 18 measured requests per boundary completed with valid 4096-token prompts and 1024-token outputs

Stack

Use the TP-EP group for joint routed/shared reductions and pass activated shared inputs through the tuned M<=4 expert kernel.

Signed-off-by: Sanket Pandit <sanket.pandit@amd.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant