Relocate cached CUDA kernels at load time

[maleadt]

Adapts CUDA to JuliaGPU/GPUCompiler.jl#878 using the :patch relocation strategy.

Compiled cubins retain a relocation manifest and expose each host reference as a named writable global. After CuModule loading, CUDA resolves the current-session value, finds the slot with cuModuleGetGlobal, and patches it with cuMemcpyHtoD before retrieving the kernel.

Cubins, entry names, and relocation metadata can therefore survive package precompilation on supported Julia versions. Module and function handles remain context-local and are never serialized. The precompile workload explicitly enrolls its GPU CodeInstance so the cached cubin is present in a fresh session.

Tests execute kernels containing runtime-library type tags, ordinary Julia values, and boxed interior relocations.

[github-actions]

CUDA.jl Benchmarks

Details

Benchmark suite	Current: c67e8c0	Previous: 12db784	Ratio
array/accumulate/Float32/1d	98074 ns	98294 ns	1.00
array/accumulate/Float32/dims=1	71945 ns	72213 ns	1.00
array/accumulate/Float32/dims=1L	1600184 ns	1599386 ns	1.00
array/accumulate/Float32/dims=2	137450 ns	137131 ns	1.00
array/accumulate/Float32/dims=2L	660544 ns	659676 ns	1.00
array/accumulate/Int64/1d	118265 ns	118271 ns	1.00
array/accumulate/Int64/dims=1	76193 ns	76038 ns	1.00
array/accumulate/Int64/dims=1L	1714887 ns	1716033 ns	1.00
array/accumulate/Int64/dims=2	149070 ns	149554 ns	1.00
array/accumulate/Int64/dims=2L	987326 ns	987949 ns	1.00
array/broadcast	17852 ns	17708 ns	1.01
array/broadcast launch	9242 ns	9121 ns	1.01
array/construct	867.8070175438596 ns	870.9649122807018 ns	1.00
array/copy	16243 ns	16047 ns	1.01
array/copyto!/cpu_to_gpu	208711 ns	208420 ns	1.00
array/copyto!/gpu_to_cpu	241583 ns	240966 ns	1.00
array/copyto!/gpu_to_gpu	8795.666666666666 ns	8885 ns	0.99
array/iteration/findall/bool	132887 ns	132809 ns	1.00
array/iteration/findall/int	146933 ns	147034 ns	1.00
array/iteration/findfirst/bool	69963 ns	69294 ns	1.01
array/iteration/findfirst/int	70717 ns	70371 ns	1.00
array/iteration/findmin/1d	67551 ns	65194 ns	1.04
array/iteration/findmin/2d	100395 ns	99776 ns	1.01
array/iteration/logical	191303 ns	189891 ns	1.01
array/iteration/scalar	63232 ns	62869 ns	1.01
array/permutedims/2d	48814 ns	48914 ns	1.00
array/permutedims/3d	48947 ns	49999 ns	0.98
array/permutedims/4d	50042 ns	49890 ns	1.00
array/random/rand/Float32	11165 ns	11620 ns	0.96
array/random/rand/Int64	23148 ns	22933 ns	1.01
array/random/rand!/Float32	7758.25 ns	7662.75 ns	1.01
array/random/rand!/Int64	20279 ns	20359 ns	1.00
array/random/randn/Float32	33259 ns	33171 ns	1.00
array/random/randn!/Float32	24441 ns	23304 ns	1.05
array/reductions/mapreduce/Float32/1d	33546 ns	33403 ns	1.00
array/reductions/mapreduce/Float32/dims=1	37804 ns	37787 ns	1.00
array/reductions/mapreduce/Float32/dims=1L	50656 ns	50842 ns	1.00
array/reductions/mapreduce/Float32/dims=2	55141 ns	55083 ns	1.00
array/reductions/mapreduce/Float32/dims=2L	66915 ns	66987 ns	1.00
array/reductions/mapreduce/Int64/1d	40084 ns	40587 ns	0.99
array/reductions/mapreduce/Int64/dims=1	40348 ns	40532 ns	1.00
array/reductions/mapreduce/Int64/dims=1L	88458 ns	88708 ns	1.00
array/reductions/mapreduce/Int64/dims=2	57529 ns	57418 ns	1.00
array/reductions/mapreduce/Int64/dims=2L	83514 ns	83444 ns	1.00
array/reductions/reduce/Float32/1d	33109 ns	33242 ns	1.00
array/reductions/reduce/Float32/dims=1	37933 ns	37706 ns	1.01
array/reductions/reduce/Float32/dims=1L	50776 ns	50707 ns	1.00
array/reductions/reduce/Float32/dims=2	55184 ns	54900 ns	1.01
array/reductions/reduce/Float32/dims=2L	68723 ns	68728 ns	1.00
array/reductions/reduce/Int64/1d	40527 ns	40891 ns	0.99
array/reductions/reduce/Int64/dims=1	40571 ns	40506 ns	1.00
array/reductions/reduce/Int64/dims=1L	88557 ns	88810 ns	1.00
array/reductions/reduce/Int64/dims=2	57633 ns	57070 ns	1.01
array/reductions/reduce/Int64/dims=2L	83417 ns	83370 ns	1.00
array/reverse/1d	17064 ns	16706 ns	1.02
array/reverse/1dL	69825 ns	69822 ns	1.00
array/reverse/1dL_inplace	67563 ns	67424 ns	1.00
array/reverse/1d_inplace	10398 ns	8537.666666666666 ns	1.22
array/reverse/2d	20417 ns	20193 ns	1.01
array/reverse/2dL	73856 ns	73620 ns	1.00
array/reverse/2dL_inplace	67100 ns	67193 ns	1.00
array/reverse/2d_inplace	9955 ns	9918 ns	1.00
array/sorting/1d	2640733 ns	2657107 ns	0.99
array/sorting/2d	1028562 ns	1028466 ns	1.00
array/sorting/by	3184768 ns	3181859 ns	1.00
cuda/synchronization/context/auto	1057.3 ns	1009.9 ns	1.05
cuda/synchronization/context/blocking	793.8659793814433 ns	785.1460674157304 ns	1.01
cuda/synchronization/context/nonblocking	5812.666666666667 ns	5750.833333333333 ns	1.01
cuda/synchronization/stream/auto	871.3265306122449 ns	867.4807692307693 ns	1.00
cuda/synchronization/stream/blocking	673.6556291390729 ns	671.6711409395973 ns	1.00
cuda/synchronization/stream/nonblocking	5718.857142857143 ns	5594.714285714285 ns	1.02
integration/byval/reference	147829 ns	147934 ns	1.00
integration/byval/slices=1	148804 ns	149023 ns	1.00
integration/byval/slices=2	292025 ns	291837 ns	1.00
integration/byval/slices=3	435126 ns	434978 ns	1.00
integration/cudadevrt	105035 ns	104908 ns	1.00
integration/volumerhs	9149464 ns	9145910 ns	1.00
kernel/indexing	12970 ns	12970 ns	1
kernel/indexing_checked	13930 ns	13675 ns	1.02
kernel/launch	2092.4444444444443 ns	2095.3333333333335 ns	1.00
kernel/occupancy	708.0143884892086 ns	725.2794117647059 ns	0.98
kernel/rand	13979 ns	14027 ns	1.00
latency/import	4104136858 ns	4101411871 ns	1.00
latency/precompile	4872008839 ns	4876779323 ns	1.00
latency/ttfp	4874549868 ns	4829821898 ns	1.01

This comment was automatically generated by workflow using github-action-benchmark.