Skip to content

Backport arch/CC compatibility check to 0.16 as a stopgap #769

Closed

Description

@moon-bot-app

Follow-up to #707 (Slack discussion).

Users hit confusing failures when a kernel has no image for the device's compute capability: get_kernel succeeds, and the failure only shows up at launch time — sometimes as a hard process exit (fprintf + exit(1) in the kernel's own error checking) that can't be caught from Python. This affects both too-new CCs (e.g. FA3 on B200, sm_100) and too-old ones (e.g. T4, sm_75).

The full fix depends on the kernel-dependency work that rewrites the loading code, so it can only land on main. In the meantime we'd like a stopgap patch release on 0.16 so there's something usable in the current stable line.

Proposed scope for the 0.16 patch:

  • Use the architectures declared in metadata.json (backend.archs) to check the current device's compute capability at load time.
  • Add check_arch: bool = True to get_kernel so it hard-fails early with a clear error instead of dying at launch; check_arch=False opts out.
  • Expose the same check without loading/hard-failing — either a new is_kernel_supported(...) helper or by extending has_kernel(...) — so callers (e.g. transformers) can branch before selecting a kernel.

An opt-out is required because metadata.json archs are the union across all kernel components, and a kernel may legitimately support more capabilities than the metadata declares (e.g. by falling back to Triton). Note also the pre-existing, better path stays unchanged: layer registrations that declare supported CUDA capabilities let kernels avoid loading an incompatible kernel at all (https://huggingface.co/docs/kernels/main/en/layers#registering-kernels-for-specific-cuda-capabilities).


Requested by Daniël de Kok - Slack thread - Agent trace

Activity

  1. self-assigned this
    on Aug 18, 2026
  2. sayakpaul commented on Aug 25, 2026

    @sayakpaul
    Member
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions