Make generated code relocatable across Julia sessions - #878
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #878 +/- ##
==========================================
+ Coverage 81.79% 83.95% +2.16%
==========================================
Files 26 27 +1
Lines 4893 5365 +472
==========================================
+ Hits 4002 4504 +502
+ Misses 891 861 -30 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
Okay, this is starting to look good. IR should be fully relocatable after this, at least for back-ends supporting run-time relocations. |
| # The pass builder resolves this name to the registered instance, which | ||
| # captures the Relocations object owned by optimize!. | ||
| add!(mpm, "GPULinkRuntime") | ||
| add!(mpm, GPULinkLibrariesPass(job)) | ||
| add!(mpm, GPUFinishRuntimeIntrinsicsPass(job)) |
There was a problem hiding this comment.
Should we also do this for the job capturing?
There was a problem hiding this comment.
You mean make them all by-name references?
| function resolve_relocation_target(target::JuliaValueRef) | ||
| box = Any[target.value] | ||
| GC.@preserve box begin | ||
| return unsafe_load(Base.unsafe_convert(Ptr{UInt}, pointer(box))) | ||
| end | ||
| end |
There was a problem hiding this comment.
I think the previous version of this uses jl_value_ptr directly?
Why the 1-element array instead of a Ref?
We had something similar in Enzyme, but we restricted it to only types. https://github.com/EnzymeAD/Enzyme.jl/blob/4c8b1c0a04689cce931bd4ad8aa02f6a0919cecc/src/utils.jl#L1-L52
I think for all other cases one can use pointer_from_objref.
There was a problem hiding this comment.
Why the 1-element array instead of a
Ref?
Fable is convinced that jl_value_ptr on 1.14 can allocate temporary stack boxes and return its address, while Any[] forces the contents to be heap-allocated too.
@noinline jvp(x) = UInt(ccall(:jl_value_ptr, Ptr{Cvoid}, (Any,), x))
@noinline slot(x) = (s = Any[x]; GC.@preserve s unsafe_load(Base.unsafe_convert(Ptr{UInt}, pointer(s))))
tag(a) = unsafe_load(Ptr{UInt}(a - 8)) & ~UInt(15)
const F64_TAG = jvp(Float64) # heap-interned DataType: fine either way
@noinline clobber(n) = n == 0 ? UInt(0) : clobber(n - 1) + hash(n)
function demo()
for (name, a) in ("jl_value_ptr" => jvp(1.25), "Any[] slot" => slot(1.25))
clobber(200) # reuse the stack region the callee frame occupied
println(rpad(name, 13), ": contents = ", rpad(unsafe_load(Ptr{Float64}(a)), 23),
" valid type tag = ", tag(a) == F64_TAG)
end
end
demo()❯ jl +nightly mwe.jl
jl_value_ptr : contents = 6.92391611475455e-310 valid type tag = false
Any[] slot : contents = 1.25 valid type tag = true
vchuravy
left a comment
vchuravy
left a comment
There was a problem hiding this comment.
One thought is around GC safety. Julia currently throws everything that is reachable from a native compilation into a global roots list (this has changed over time, I think 1.10 still had per CodeInstance rooting) so the thing I am wondering about is: Could we lose a root by going through this?
|
Another suggestion by @vchuravy: Try this out on AllocCheck.jl |
|
AllocCheck.jl integration: JuliaLang/AllocCheck.jl#113 @vchuravy I think this is ready for another look. |
16fed03 to
44c1e41
Compare
I don't think so. EDIT: although I guess we could just make sure to perma-root, and then get rid of the |
|
Turns out Metal.jl doesn't support patching relocations like we do in CUDA (no support for something CuGlobal like), so I'm investigating a new strategy where we have a GOT-like table that's passed as part of the KernelState. |
29e310a to
8b4a7dd
Compare
maleadt
changed the title
8b4a7dd to
abc93a8
Compare
Represent Julia values and runtime globals as a deterministic relocation manifest instead of embedding host addresses in cached IR. Back-ends can bake the manifest for compatibility, expose writable globals for loader patching, or read a table supplied at execution time. Only persist compilation results when both Julia and the selected lowering make them session-independent. Keep runtime-library linking symbolic until final emission, permanently root resolved Julia values, and treat zero-sized singleton values as addressable relocation targets. Exercise the three lowering strategies, linking and validation failures, cache persistence, PTX, SPIR-V, Metal-style table delivery, boxed constants, empty type objects, and interior relocation sites.
2 participants
Keep Julia values and libjulia globals symbolic until final backend lowering, instead of embedding session-specific host addresses in cached IR.
Relocation records are deterministic and remain attached while modules are linked and optimized. A backend selects one of three delivery strategies:
:bakeresolves addresses during final emission. This preserves compatibility but keeps results session-local.:patchemits named writable globals for a loader such as CUDA or ORC to patch after loading.:tablerewrites references through a runtime table for loaders such as Metal that cannot look up data symbols.Cached backend results persist only when Julia exposes relocatable global metadata and the backend selects
:patchor:table. Resolved Julia values are permanently rooted, runtime bitcode remains symbolic until it reaches the final module, and manifests are frozen after lowering so metadata cannot diverge from emitted code.The implementation is exercised by CUDA.jl#3200, Metal.jl#916, and JuliaLang/AllocCheck.jl#113. Backends that have not opted in retain the eager
:bakebehavior; this was validated with OpenCL.jl on both its SPIR-V and OpenCL C paths.Supersedes #125 and #348.