feat(attention): add Gluon forward attention backend - #469
Conversation
Vendor and integrate the gfx950 Gluon forward kernel, add explicit dense forward-only dispatch, benchmarks, correctness coverage, and the upstream 05b349b synchronization fixes.
| def backward(ctx, dout, *args): | ||
| if ctx.backend == BackendType.GLUON: | ||
| raise AssertionError( | ||
| "internal contract violation: gluon flash-attn is forward-only and must not reach backward" |
There was a problem hiding this comment.
Is forward FA kernel only need for client ?
There was a problem hiding this comment.
This PR adds forward first. Gluon backward is the next focus and will follow separately.
|
|
||
|
|
||
| @pytest.mark.skipif( | ||
| os.environ.get("PRIMUS_TURBO_TEST_LONG") != "1", |
There was a problem hiding this comment.
Please remove debug env.
There was a problem hiding this comment.
Please merge this file into test_attention.py.
| _GFX950_DEVICE_CACHE: dict[torch.device, bool] = {} | ||
|
|
||
|
|
||
| def _is_gfx950_device(device: torch.device) -> bool: |
There was a problem hiding this comment.
Please use this func.
Primus-Turbo/primus_turbo/pytorch/core/utils.py
Lines 24 to 25 in e29a9b3
There was a problem hiding this comment.
What's the purpose of this unit test file ?
There was a problem hiding this comment.
This was a self-added vendoring-policy check rather than existing CI coverage. I removed this test and the corresponding checker to keep the PR focused on the Gluon backend.
c4689f6 to
6196e77
Compare
Description
Adds an opt-in Gluon backend for dense Flash Attention forward on gfx950.
Gluon is not part of the default selection or cross-backend autotuning. It is forward-only, and a pinned unsupported call fails before launch instead of falling back. The port does not load an LLVM pass plugin, pin LLVM/Triton, or select behavior from a compiler version.
Fixes # (N/A)
Type of change
Changes
BackendType.GLUONin the dense attention dispatcher.torch.compile(fullgraph=True)support.Supported envelope
Dense rank-4 FP16/BF16 attention on gfx950; BSHD or BHSD storage; MHA, GQA, and MQA; head dimension up to 256; causal and noncausal. Causal requires
Sq <= Skv.Performance
MI350X (
gfx950), PyTorch2.13.0+rocm7.1, Triton3.8.0+fb, FP16, BSHD, D128, warm cache. The 30-shape set covers causal/noncausal MHA, GQA, and MQA from S128 through S16384. Each row must pass the FP32-reference check at greater than 40 dB output SNR before it is timed. Backends are pinned; no fallback result is included. Gluon uses a fresh autotune cache: 26 general shape keys use the 36-config autotuner; four short-causal rows use the fixed specialization. Tuning and correctness complete before 20 warmups and 100 measured launches. TFLOPS use the attended FLOP count, including the causal fraction.The geometric mean of the 30 per-shape Gluon/AITER throughput ratios is
0.830x; the long, high-throughput rows dominate the arithmetic mean. The result is shape-dependent. Selected causal D128 rows:Checklist