Cookbook: Using Layers

[ariG23498]

CC: @stevhliu @danieldk @sayakpaul

[github-actions]

Hi @ariG23498, thanks for your interest in contributing!

This project requires that pull request authors are vouched, and you are not in the list of vouched users.

This PR will be closed automatically. See https://github.com/huggingface/kernels/blob/main/CONTRIBUTING.md for more details.

[github-actions]

Coverage report — kernels/

Measured on: Python 3.10 / Torch 2.13.0.
Other CI configurations are not included in this number.
Hardware-gated code paths (ROCm/XPU/NPU/Darwin/Windows) are excluded or unreachable on the Linux+CUDA runner.

Total coverage: 87.5% — threshold: 80% — ✅

Per-file breakdown

Name	Stmts	Miss	Cover	Missing
src/kernels/__init__.py	14	0	100%
src/kernels/_system.py	6	1	83%	10
src/kernels/_versions.py	130	14	89%	53, 59-60, 63-64, 102, 165-170, 199, 219
src/kernels/archs.py	56	1	98%	94
src/kernels/backends.py	214	62	71%	42, 46, 50-53, 70, 92, 110, 119, 123, 127-129, 150, 159, 163, 167-169, 190, 201, 203, 210-213, 222, 226, 230-250, 258, 281-301
src/kernels/compat.py	9	1	89%	5
src/kernels/deps.py	70	1	99%	56
src/kernels/hf_hub.py	63	2	97%	21, 23
src/kernels/importer.py	44	5	89%	80, 84, 87, 101-102
src/kernels/install.py	21	7	67%	76-100
src/kernels/layer/__init__.py	6	0	100%
src/kernels/layer/_interval_tree.py	103	4	96%	23, 52, 147, 150
src/kernels/layer/device.py	48	14	71%	42, 47-49, 91, 96-98, 101, 149, 152, 155-157
src/kernels/layer/func.py	85	6	93%	90, 115, 191, 311, 338, 368
src/kernels/layer/globals.py	5	0	100%
src/kernels/layer/kernelize.py	82	8	90%	259, 297, 305-306, 312, 316, 332-334
src/kernels/layer/layer.py	309	22	93%	211, 258, 285, 416, 437, 444-447, 453-454, 556-557, 578, 586, 597, 637, 641, 654, 708, 738, 813
src/kernels/layer/mode.py	14	0	100%
src/kernels/layer/repos.py	144	42	71%	27, 33, 36-43, 63-64, 70, 73-76, 90, 94, 103-104, 110, 113-116, 123-124, 130, 133-136, 143-144, 150, 153-156, 163-164, 170, 173-176, 257
src/kernels/load.py	71	2	97%	338, 378
src/kernels/locking.py	89	64	28%	35-83, 91-98, 102-125, 137, 152-159, 165-175, 179-186
src/kernels/python_deps.py	58	6	90%	59-60, 64-65, 101, 104
src/kernels/resolver.py	158	1	99%	231
src/kernels/status.py	50	2	96%	25, 79
src/kernels/validate.py	88	5	94%	9, 100, 167, 190-191
src/kernels/variants.py	278	17	94%	65, 96, 117, 147, 256-257, 299-302, 304, 388-394, 400-406, 455-461
src/kernels/verify.py	127	6	95%	46, 202-204, 318-319
TOTAL	2342	293	87%

Updated by the Test kernels workflow on commit ae4a3bef2c4c1043bb55816e9b29fdfb8c9cb7d5.

[sayakpaul]

Since both @danieldk and I have extensively reviewed the content, requesting a review from @stevhliu and also to determine how to make it visible from our docs :)

[stevhliu]

sorry for the delay! i think you could place it under Use kernels > Cookbook in the docs right now alongside the Quickstart and Use layers since that section already groups practical usage there. you could also link to the cookbook from the Use layers guide. as (and if) the cookbook grows, it'd make more sense to eventually put it in the Resources section.

[stevhliu]

i couldn't comment directly on the ipynb, so leaving my comments here:

• in Using kernel layers with functions, should @use_kernelized_func(Gelu) be @use_kernelized_func(gelu) instead as that's whats called in forward?
• in Kernel Mapping, the key is Gelu but the registered kernel is SiluAndMul
• where it says replace_kernel_forward_from_hub replaces the layer definition with the kernel layer, it may be clearer that replace_kernel_forward_from_hub marks it for lookup and its kernelize() that does the actual replacing
• may be nice to add a sentence in the Are kernels useful section about what conditions are being benchmarked. would be be nice to include how to download or clone the step that gets the linked Gist before running the Jobs command

[ariG23498]

Thanks for the pointers @stevhliu!

in Using kernel layers with functions, should @use_kernelized_func(Gelu) be @use_kernelized_func(gelu) instead as that's whats called in forward?

As pointed out by @danieldk and @sayakpaul, the registration should be in CamelCase while the function name should be in snake_case. This is the reason why we see the function name being gelu() while the registration was Gelu.

in Kernel Mapping, the key is Gelu but the registered kernel is SiluAndMul

Great catch! Updated.

where it says replace_kernel_forward_from_hub replaces the layer definition with the kernel layer, it may be clearer that replace_kernel_forward_from_hub marks it for lookup and its kernelize() that does the actual replacing

Updated!

may be nice to add a sentence in the Are kernels useful section about what conditions are being benchmarked. would be be nice to include how to download or clone the step that gets the linked Gist before running the Jobs command

Updated!

[ariG23498]

Also @stevhliu about the placement of this cookbook

i think you could place it under Use kernels > Cookbook in the docs right now alongside the Quickstart and Use layers since that section already groups practical usage there

Can we do that with notebooks? I see that the Quickstart or Use Layers are markdown while this is not. How do you think we should handle the situation.

you could also link to the cookbook from the Use layers guide

That I can definitely do! After this gets merged I can get another PR rolling to link this notebook from the Use Layers guide. 🤗

[sayakpaul]

I am going to merge this PR to not block it. Let's think about where we should hyperlink it from / move it in another PR.

[sayakpaul]

Beautiful!

[stevhliu]

Can we do that with notebooks? I see that the Quickstart or Use Layers are markdown while this is not. How do you think we should handle the situation.

yeah, i believe you can do doc-builder notebook-to-mdx path/to/notebook.ipynb to convert it to markdown. then in the markdown doc, add [[open-in-colab]] to the top to generate a notebook from it