Hi 馃憢. I am one of the core team members of the Hugging Face Kernels project: https://huggingface.co/docs/kernels/ The goal of the kernels project is to make it easy for kernel developers to write and distribute kernels and for kernel users to use kernels in their favorite models. The project consists of several parts:
-
kernel-builder: builds kernels for multiple systems (OS + arch), backends (CUDA, XPU, ROCm, Metal and multiple versions of each), and framework versions (e.g. multiple Torch versions).
-
Kernel Hub: which can be used for distribution of kernels. There are quite some kernels already available from us and from third-parties: https://huggingface.co/kernels
-
The kernels Python client. Using the Python client, users can download and load kernels using simply:
import kernels
fa3 = kernels.get_kernel("kernels-community/flash-attn3", version=1)
kernels will take care of download and loading the right variant for a user's system.
So far, we have focused primarily on Torch support, but I think TVM FFI fits in well with our goal of making it easy to build and load kernels, especially because it would make Hub kernels available for frameworks beyond Torch. A while ago, we implemented initial support for building and loading TVM FFI kernels. There is a minimal example at: https://github.com/huggingface/kernels/tree/main/examples/kernels/relu-tvm-ffi , with the built kernel at: https://huggingface.co/kernels/kernels-test/relu-tvm-ffi
We would love to collaborate more if you are interested - we have a bunch of questions around encoding ABI versions, etc. If you are interested, we could set up a shared Slack channel or otherwise communicate through this issue.
Hi 馃憢. I am one of the core team members of the Hugging Face Kernels project: https://huggingface.co/docs/kernels/ The goal of the kernels project is to make it easy for kernel developers to write and distribute kernels and for kernel users to use kernels in their favorite models. The project consists of several parts:
kernel-builder: builds kernels for multiple systems (OS + arch), backends (CUDA, XPU, ROCm, Metal and multiple versions of each), and framework versions (e.g. multiple Torch versions).Kernel Hub: which can be used for distribution of kernels. There are quite some kernels already available from us and from third-parties: https://huggingface.co/kernels
The
kernelsPython client. Using the Python client, users can download and load kernels using simply:kernelswill take care of download and loading the right variant for a user's system.So far, we have focused primarily on Torch support, but I think TVM FFI fits in well with our goal of making it easy to build and load kernels, especially because it would make Hub kernels available for frameworks beyond Torch. A while ago, we implemented initial support for building and loading TVM FFI kernels. There is a minimal example at: https://github.com/huggingface/kernels/tree/main/examples/kernels/relu-tvm-ffi , with the built kernel at: https://huggingface.co/kernels/kernels-test/relu-tvm-ffi
We would love to collaborate more if you are interested - we have a bunch of questions around encoding ABI versions, etc. If you are interested, we could set up a shared Slack channel or otherwise communicate through this issue.