Skip to content

(WIP) feat(amd): add gfx950 A8W4 SiTU core - #1138

Draft
panditsa wants to merge 1 commit into
lightseekorg:sanket/wip-k3-situ-foundationfrom
panditsa:sanket/wip-k3-a8w4-core
Draft

(WIP) feat(amd): add gfx950 A8W4 SiTU core#1138
panditsa wants to merge 1 commit into
lightseekorg:sanket/wip-k3-situ-foundationfrom
panditsa:sanket/wip-k3-a8w4-core

Conversation

@panditsa

@panditsa panditsa commented Aug 19, 2026

Copy link
Copy Markdown
Contributor

Summary

  • add dynamic FP8 activation quantization for the TP-sharded A8W4 path
  • add route-direct SiTU W13/W2 decode with optional shared-down fusion
  • use the package-prefill path above the small-M regime

Performance

Direct TP8/EP1 K3-shaped CUDA-graph microbenchmark (E=896, H=3584, I/TP=384, topk=16):

M Existing grouped A16W4 New A8W4 core Reduction
1 84.80 µs 26.51 µs 68.7%
2 85.53 µs 27.32 µs 68.1%
4 85.12 µs 30.26 µs 64.4%
8 89.84 µs 39.20 µs 56.4%
16 157.37 µs 65.03 µs 58.7%

This PR exposes the core but does not register it with the model-facing selector, so there is intentionally no standalone end-to-end claim here. Registration and E2E selection land in #1139.

Tests

Stack

Add dynamic FP8 quantization and route-direct A8W4 SiTU decode, including optional shared-down projection and package-prefill fallback.

Signed-off-by: Sanket Pandit <sanket.pandit@amd.com>
@panditsa
panditsa force-pushed the sanket/wip-k3-a8w4-core branch from 49adbc9 to 07fded4 Compare August 19, 2026 20:21
@panditsa
panditsa force-pushed the sanket/wip-k3-situ-foundation branch from 3785c40 to 2c32487 Compare August 19, 2026 20:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant