You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Backport arch/CC compatibility check to 0.16 as a stopgap #769
Users hit confusing failures when a kernel has no image for the device's compute capability: get_kernel succeeds, and the failure only shows up at launch time — sometimes as a hard process exit (fprintf + exit(1) in the kernel's own error checking) that can't be caught from Python. This affects both too-new CCs (e.g. FA3 on B200, sm_100) and too-old ones (e.g. T4, sm_75).
The full fix depends on the kernel-dependency work that rewrites the loading code, so it can only land on main. In the meantime we'd like a stopgap patch release on 0.16 so there's something usable in the current stable line.
Proposed scope for the 0.16 patch:
Use the architectures declared in metadata.json (backend.archs) to check the current device's compute capability at load time.
Add check_arch: bool = True to get_kernel so it hard-fails early with a clear error instead of dying at launch; check_arch=False opts out.
Expose the same check without loading/hard-failing — either a new is_kernel_supported(...) helper or by extending has_kernel(...) — so callers (e.g. transformers) can branch before selecting a kernel.
An opt-out is required because metadata.json archs are the union across all kernel components, and a kernel may legitimately support more capabilities than the metadata declares (e.g. by falling back to Triton). Note also the pre-existing, better path stays unchanged: layer registrations that declare supported CUDA capabilities let kernels avoid loading an incompatible kernel at all (https://huggingface.co/docs/kernels/main/en/layers#registering-kernels-for-specific-cuda-capabilities).
Activity
sayakpaul commentedon Aug 25, 2026
#774