Skip to content

feat: support rust gpu - #907

Open
drbh wants to merge 1 commit into
mainfrom
rust-gpu-support
Open

drbh wants to merge 1 commit into
mainfrom
rust-gpu-support

Conversation

@drbh

@drbh drbh commented Oct 15, 2026 •

Copy link
Copy Markdown
Collaborator

this pr builds on the recent rust cpu support and extends support to rust gpu

related prior work

@HuggingFaceDocBuilderDev

Copy link
Copy Markdown

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@github-actions

Copy link
Copy Markdown

Coverage report — kernels/

Measured on: Python 3.10 / Torch 2.13.0.
Other CI configurations are not included in this number.
Hardware-gated code paths (ROCm/XPU/NPU/Darwin/Windows) are excluded or unreachable on the Linux+CUDA runner.

Total coverage: 87.8% — threshold: 80% — ✅

Per-file breakdown
Name Stmts Miss Cover Missing
src/kernels/__init__.py 14 0 100%
src/kernels/_system.py 6 1 83% 10
src/kernels/_versions.py 130 14 89% 53, 59-60, 63-64, 102, 165-170, 199, 219
src/kernels/archs.py 56 1 98% 94
src/kernels/backends.py 215 63 71% 42, 46, 50-53, 70, 92, 110, 119, 123, 127-129, 150, 159, 163, 167-169, 190, 201, 203, 210-213, 222, 226, 230-255, 263, 286-306
src/kernels/compat.py 9 1 89% 5
src/kernels/deps.py 70 1 99% 56
src/kernels/hf_hub.py 63 2 97% 21, 23
src/kernels/importer.py 44 5 89% 80, 84, 87, 101-102
src/kernels/install.py 21 7 67% 76-100
src/kernels/layer/__init__.py 6 0 100%
src/kernels/layer/_interval_tree.py 103 4 96% 23, 52, 147, 150
src/kernels/layer/device.py 48 14 71% 42, 47-49, 91, 96-98, 101, 149, 152, 155-157
src/kernels/layer/func.py 85 6 93% 90, 115, 191, 311, 338, 368
src/kernels/layer/globals.py 5 0 100%
src/kernels/layer/kernelize.py 82 8 90% 259, 297, 305-306, 312, 316, 332-334
src/kernels/layer/layer.py 309 15 95% 211, 258, 285, 416, 556-557, 578, 586, 597, 637, 641, 654, 708, 738, 813
src/kernels/layer/mode.py 14 0 100%
src/kernels/layer/repos.py 144 42 71% 27, 33, 36-43, 63-64, 70, 73-76, 90, 94, 103-104, 110, 113-116, 123-124, 130, 133-136, 143-144, 150, 153-156, 163-164, 170, 173-176, 257
src/kernels/load.py 71 2 97% 338, 378
src/kernels/locking.py 89 64 28% 35-83, 91-98, 102-125, 137, 152-159, 165-175, 179-186
src/kernels/python_deps.py 58 6 90% 59-60, 64-65, 101, 104
src/kernels/resolver.py 162 1 99% 248
src/kernels/status.py 50 2 96% 25, 79
src/kernels/validate.py 88 5 94% 9, 100, 167, 190-191
src/kernels/variants.py 278 17 94% 65, 96, 117, 147, 256-257, 299-302, 304, 388-394, 400-406, 455-461
src/kernels/verify.py 127 6 95% 46, 202-204, 318-319
TOTAL 2347 287 88%

Updated by the Test kernels workflow on commit c440ae7f92fbb6fb142376c16a55bed8f56a16fe.

@danieldk
danieldk self-requested a review October 19, 2026 08:29

@danieldk danieldk left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! Added a bunch of comments.

depends: Vec<Dependency>,
src: Vec<String>,
},

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: superfluous newline?

RustCuda {
cargo_manifest: String,
device_manifest: Option<String>,
ptx_dir: Option<String>,

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we want to make this user-configurable?


RustCuda {
cargo_manifest: String,
device_manifest: Option<String>,

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I looked at some of the example code in cuda-oxide and I couldn't find separate device manifests? Do you have an example? (The PR doesn't seem to have an example.)

Comment on lines +771 to +772
let serialized = toml::to_string(&v6::Build::from(build)).unwrap();
let reparsed: v6::Build = toml::from_str(&serialized).unwrap();

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know we have more of this, but I'm not sure what we are actually testing here? Aren't we testing the serde/toml crates here?

cargoLock ? null,

# Compile CUDA device crates with cuda-oxide and its matching rustc.
rustCudaDevice ? false,

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the difference between this and rustSupport && cudaSupport?

I wonder of the option is very descriptive? Should this rather be cudaOxide or something along those lines?

Comment on lines +51 to +57
CUDA_HOME = cudaHome;
CUDA_TOOLKIT_PATH = cudaHome;
LIBRARY_PATH = "${lib.getOutput "stubs" cudaPackages.cuda_cudart}/lib/stubs";
LIBCLANG_PATH = "${lib.getLib pkgs.libclang}/lib";
BINDGEN_EXTRA_CLANG_ARGS = lib.concatStringsSep " " [
"-isystem ${lib.getLib pkgs.libclang}/lib/clang/${lib.versions.major pkgs.libclang.version}/include"
"-isystem ${lib.getDev stdenv.cc.libc}/include"

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not very keen on propagating all these variables to the build environment if not required. Which of these variables are needed by what? BINDGEN_EXTRA_CLANG_ARGS is clear, but I'm not clear on the rest?

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants