Repository navigation
activation: Add XPU backend to kernels-community/activation - #1071
Conversation
0854a71 to
8b437ac
Compare
Implement the eleven activation operators in SYCL and register them under the XPU dispatch key, so existing callers work unchanged on Intel GPUs. Previously the kernel declared only cuda, rocm, metal and cpu, and could not be loaded on XPU at all. Gated operators map one work-group to one token; element-wise operators use a flat grid-stride loop. Both use 128-bit vectorized accesses with a runtime alignment check and a scalar fallback, which is what keeps the element-wise operators on par with eager PyTorch, whose TensorIterator already vectorizes. Parameterize tests/kernels/test_activation.py over the available accelerator rather than hard-coding CUDA, so the existing suite runs unmodified on XPU. It passes in full on Intel Arc Pro B60 with torch 2.13.0+xpu, including the gated operators that are compared against eager PyTorch with zero tolerance. Add two cases to the same file for shapes and alignments that force the scalar path, and for empty inputs. Requires huggingface/kernels#752, which makes kernel-builder request IEEE correctly rounded fp32 division for SYCL kernels; without it silu_and_mul and mul_and_silu differ from eager PyTorch by one ULP.
|
huggingface/kernels#752 merged. This PR is ready to be reviewed. |
# Conflicts: # activation/tests/kernels/test_activation.py
|
/kernel-bot security-and-build activation |
|
Build request processed. Command: Dispatched (3):
Dispatched (1):
|
🔒 Security Audit — findings detectedAn automated, AI-generated security review of this PR flagged the following. These may include false positives — review each item before acting. I've reviewed the diff, which adds an Intel XPU (SYCL) backend to the existing Assessment:
This is a clean, well-bounded backend port with no attacker-exploitable behavior. NO_FINDINGS Audited head SHA |
|
/kernel-bot merge-and-upload activation |
|
Build request processed. Command: Merge result: PR is already merged. Continuing with build/upload. Dispatched (3):
Hub uploads: |
2 participants
Summary
Adds a SYCL implementation of the
activationkernels so the component can beused on Intel GPUs. All eleven operators exposed by the CUDA backend are
implemented and registered under the
XPUdispatch key, so existing callerswork unchanged on XPU devices.
Previously
activationdeclared onlycuda,rocm,metalandcpu. On XPUthe kernel could not be loaded at all, forcing every consumer to carry a
separate eager fallback path.
Operators:
silu_and_mul,mul_and_silu,gelu_and_mul,gelu_tanh_and_mul,fatrelu_and_mul,gelu_new,gelu_fast,gelu_quick,silu,gelu,gelu_tanh. Supported dtypes:float32,float16,bfloat16.New sources live in
activation_xpu/, following the layout of the existing CPUbackend.
tests/kernels/test_activation.pyis parameterized over the availableaccelerator instead of hard-coding CUDA, so the existing suite runs unmodified
on XPU.
Depends on huggingface/kernels#752, which makes
kernel-builderrequest IEEEcorrectly rounded fp32 division for SYCL kernels.
Correctness
Validated on Intel Arc Pro B60 with
torch 2.13.0+xpuand oneAPI 2026.0.The existing
tests/kernels/test_activation.pypasses in full on XPU (420cases), including the gated operators that are compared against eager PyTorch
bit-for-bit with zero tolerance. Two cases were added to that file, covering
shapes and alignments that force a vectorized backend onto its scalar path, and
empty inputs. Both are device-parameterized like the rest of the suite.
Performance
Measured on one Intel Arc Pro B60. Speedup is eager time divided by kernel time;
values above 1.00 favour the kernel. Gated operators use a
16384 x 16384input, element-wise operators
16384 x 8192; the4096columns are the samemeasurement at
4096 x 8192and4096 x 4096.silu_and_mulmul_and_silugelu_and_mulgelu_tanh_and_mulfatrelu_and_mulgelu_newgelu_fastgelu_quicksilugelugelu_tanhThree regimes are visible:
[n, d]intermediates and cut memory trafficfrom
6ndto3nd.fatrelu_and_mulgains more because its eager formadditionally materializes a boolean mask.
eight or more element-wise launches in eager mode, so fusing them gives the
largest speedups.
corresponding PyTorch operator and are therefore on par with it. They exist
for interface completeness, so callers need no device-dependent branch.
Memory
Peak memory allocated while producing the result, via
torch.xpu.max_memory_allocated. The output tensor is included and the inputexcluded, so both paths are charged for what they produce. The ratio is
independent of dtype and shape; absolute figures are
float16at16384 x 16384(gated) and16384 x 8192(element-wise).silu_and_mul,mul_and_silu,gelu_and_mul,gelu_tanh_and_mul,fatrelu_and_mulgelu_quickgelu_newgelu_fastsilu,gelu,gelu_tanhThe fused kernels never allocate beyond the output tensor. Eager
gelu_fastpeaks at four times that because three intermediates are live simultaneously.
Build
Both XPU variants offered by
kernel-builderbuild cleanly and pass themanylinux_2_28and Pythonabi3compatibility checks:Limitations