BTF relocations#3966
Conversation
BTF, the BPF Type Format, encodes type information for both the running
Linux kernel and compiled eBPF programs. An eBPF object can carry
relocation records that describe field and aggregate accesses in terms
of BTF types instead of fixed offsets; at load time, the loader compares
the program's BTF with the kernel's BTF and rewrites those accesses to
the correct offsets for the target kernel. This mechanism is often
referred to as "CO-RE relocations" or "BTF relocations".
`offset_of` always folds to a plain layout constant and does not
preserve enough information for BTF CO-RE relocation emission. As a
result, it is not suitable for relocatable field queries on BPF targets.
Add three explicit intrinsics for BTF field metadata queries:
* `btf_field_byte_offset`
* `btf_field_byte_size`
* `btf_field_exists`
The user-facing BTF relocatable type support remains behind the
`btf_relocations` feature gate, so use of this experimental CO-RE
surface requires an explicit nightly opt-in.
These intrinsics provide a frontend surface for CO-RE relocations.
Unlike `offset_of`, they remain visible to backend codegen and can lower
to relocatable field-info queries instead of immediate layout constants.
For LLVM, lower these intrinsics through `@llvm.bpf.preserve.field.info`
with the corresponding query kind. The necessary
`@llvm.preserve.{struct,array,union}.access.index` chain is constructed
internally during lowering, but it is not exposed as part of the
user-facing API.
On targets or backends without BTF relocation support, fall back to the
ordinary layout-computed result: the field offset for
`btf_field_byte_offset`, the field size for `btf_field_byte_size`, and
`true` for `btf_field_exists`.
The language-level design for this feature is proposed in
rust-lang/rfcs#3966.
BTF, the BPF Type Format, encodes type information for both the running
Linux kernel and compiled eBPF programs. An eBPF object can carry
relocation records that describe field and aggregate accesses in terms
of BTF types instead of fixed offsets; at load time, the loader compares
the program's BTF with the kernel's BTF and rewrites those accesses to
the correct offsets for the target kernel. This mechanism is often
referred to as "CO-RE relocations" or "BTF relocations".
`offset_of` always folds to a plain layout constant and does not
preserve enough information for BTF CO-RE relocation emission. As a
result, it is not suitable for relocatable field queries on BPF targets.
Add three intrinsics for BTF field metadata queries:
* `btf_field_byte_offset`
* `btf_field_byte_size`
* `btf_field_exists`
Their availability is hidden behind the `btf_relocations` feature gate.
Unlike `offset_of`, they remain visible to backend codegen and can lower
to relocatable field-info queries instead of immediate layout constants.
For LLVM, lower these intrinsics through `@llvm.bpf.preserve.field.info`
with the corresponding query kind. The necessary
`@llvm.preserve.{struct,array,union}.access.index` chain is constructed
internally during lowering, but it is not exposed as part of the
user-facing API.
On targets or backends without BTF relocation support, fall back to:
* The field offset for `btf_field_byte_offset`.
* The field size for `btf_field_byte_size`.
* `true` for `btf_field_exists`.
The language-level design for this feature is proposed in
rust-lang/rfcs#3966.
0a822d7 to
2ec6577
Compare
2ec6577 to
54a1461
Compare
| `#[repr(Btf)]` is intentionally not just a layout hint. It marks a type as one | ||
| whose fields should not be accessed through ordinary Rust field projection for | ||
| accesses that are meant to be relocatable. For such types, direct field access | ||
| is rejected: | ||
|
|
||
| ```rust | ||
| #![feature(btf_relocations)] | ||
|
|
||
| #[repr(Btf)] | ||
| pub struct task_struct { | ||
| pub pid: i32, | ||
| } | ||
|
|
||
| fn pid(task: &task_struct) -> i32 { | ||
| task.pid | ||
| // error: cannot access fields of a `#[repr(Btf)]` type directly |
There was a problem hiding this comment.
I would rather we do not use another repr that has the same "feature" as repr(packed), for which we seem to still be encountering new issues: rust-lang/rust#157011
I would prefer that we find some new way to express it syntactically that makes it harder for us to make more such mistakes in the future. Essentially, because these are sort of an Alien Being relative to normal structs, maybe they should not use the syntax of normal structs at all?
But I also wish to acknowledge that I do not have any objection to the fundamental behavior proposed, so this should not block any initial experimentation. It is more of a yet-to-be-resolved-concern type of deal. Unanswered question? "Can anyone think of something better?"
There was a problem hiding this comment.
I'm fine with replacing repr with something else, although the only (not sophisticated at all) idea I have now is an attribute (e.g. #[relocatable], #[btf_relocatable]. Your other comment #3966 (comment) is an another good argument against repr, and in general it should be possible to make a repr(C), a repr(Rust) or repr(align) struct BTF-relocatable.
There was a problem hiding this comment.
To me, it almost feels like it's some kind of exotic extern struct, so I would start thinking in that direction and ping-pong it a little with whoever I asked on T-lang to sponsor things.
|
|
||
| These restrictions avoid silently producing non-relocatable code for operations | ||
| that appear to query a relocatable type. Code that genuinely wants a normal | ||
| non-relocatable Rust type should not use `#[repr(Btf)]`. |
There was a problem hiding this comment.
It isn't clear to me that this achieves its intended goal? Or I don't understand the goal?
The current proposal includes a "compile-time fallback" to normal layout behavior, several lines back.
On targets or backends without BTF
relocation support, they fall back to the current compilation unit's ordinary
layout information.This is a fallback path that is sometimes permitted based on some variable that can differ based on the circumstances of the compilation. So if we do not trust the compiler to correctly handle the matter, it can always choose a "wrong" path and silently produce directly non-relocatable accesses despite using all these special macros. According to my understanding, we have gone to some length to supposedly prevent silent failure via introducing syntactic noise, and then allowed for silent failure?
Thus we must trust the compiler anyways, right? So what are we trying to guard against?
There was a problem hiding this comment.
The reason I proposed the fallback mechanism is that most of Linux BPF projects consist of two parts - the actual BPF program running in the kernel, and the user-space application. The user-space part grabs the data produced by the BPF program from a BPF map (usually a ring buffer, or a hash map). BPF maps, at least in Linux, can store any type with C representation. It's not uncommon to store a BTF-relocatable kernel struct there.
But now you made me realize that this can be solved with #[cfg_attr(target_arch = "bpf", ...)], so I agree with you, we could carry on without a fallback and emit a compilation error if BTF relocations are requested on non-BPF target. That absolutely makes sense from the "trust the compiler" perspective.
There was a problem hiding this comment.
It's not uncommon to store a BTF-relocatable kernel struct there.
Actually, after giving it more thought, this should be avoided, and it's good that we will make it difficult with a compile error. BTF-relocatable struct has an unknown layout. Anything stored in BPF maps should have deterministic layout across kernel versions, so the user-space application behaves the same regardless of the kernel version. A good pattern here is always to copy specific fields from a relocatable type to a non-relocatable type before storing it in a map.
| #[inline] | ||
| pub fn pid(&self) -> Option<&i32> { | ||
| self.has_pid().then(|| { | ||
| let offset = core::btf::field_byte_offset!(task_struct, pid); | ||
| let ptr = self as *const task_struct as *const u8; | ||
|
|
||
| // SAFETY: the BTF relocation says that `se.vruntime` exists in the | ||
| // target layout, and the returned offset is relative to `task_struct`. | ||
| Some(unsafe { &*(ptr.add(offset) as *const i32) }) | ||
| }) |
There was a problem hiding this comment.
So a typo in a field name can result in simply turning this into a None and never taking the "active" code path, right? Since the relocation is functionally a set of runtime checks, and we would have to assume we take the field name for granted, and the entire reason to use these relocations is the struct definition per se is actually external to the program...?
There was a problem hiding this comment.
Yes, that's right. Although instead of saying "is actually external", I would say that the struct definition is a reference point for the runtime checks.
To be fair, I proposed a field projection (like in clang) in my pre-RFC, but given the complexity, I'm not pursuing it for now.
| A type that should participate in BTF relocation is written with | ||
| `#[repr(Btf)]`: |
There was a problem hiding this comment.
Something to consider with this as repr: Is this meant to be exclusive with other repr hints?
- align?
- packed?
There was a problem hiding this comment.
Not really, actually I can't think of anything preventing BTF-relocatable types be align or packed, in theory. Seems like that's an another argument against repr.
There was a problem hiding this comment.
Part of the reason I'm not super-happy with repr in general (just... in Rust, and to be clear I'm not asking you to fix all of Rust) is indeed the sort of arbitrary composition rules, where sometimes a repr is exclusive (repr(C) and repr(transparent), for instance) and sometimes it isn't (repr(align(N)), unless it's repr(transparent)...) so I don't think "another repr has kinda weird/arbitrary composition rules" is disqualifying, but it does mean we always have to think about the interactions.
There was a problem hiding this comment.
I don't think repr(C) is exclusive? You can definitely have a repr(C, align(...)) struct for example (there's even an example in the reference: https://doc.rust-lang.org/nightly/reference/type-layout.html#r-layout.repr.align-packed).
I do think that the RFC should spell out what should be allowed / disallowed.
There was a problem hiding this comment.
Yes, the problem with reprs in general is basically there's no obvious way to tell what is an "exclusive" repr attribute like C, transparent, and Rust, vs. the ones that are "non-exclusive" like align. Except even that general principle is not enough because transparent is exclusive with align, for instance. You just have to memorize the pile of rules, which gets more complex every time we add a new one.
There was a problem hiding this comment.
I agree that repr is not ideal for the reasons mentioned.
| A `#[repr(Btf)]` type uses C-compatible field layout. In compiler terms, | ||
| `repr(Btf)` implies the layout constraints of `repr(C)` and also marks the type | ||
| as BTF-relocatable. This gives the backend stable field ordering and offsets for | ||
| the compile-time fallback while preserving a distinct marker for type checking |
There was a problem hiding this comment.
What do we mean here, or what do we achieve, by having a "compile-time fallback"?
There's an implicit alternative here: No fallback. What are we giving up if we choose that instead?
There was a problem hiding this comment.
As agreed above, I'm going to remove the "compile-time fallback" from the RFC.
| relocations. This is attractive ergonomically, but it requires a more intrusive | ||
| change to MIR and the design of a proper abstract machine with operational | ||
| semantics. It also has some similarities to the [`Sized` hierarchy | ||
| RFC][sized-hierarchy], which has not yet been accepted. |
There was a problem hiding this comment.
This statement seems to be wearing its shirt backwards, so to speak. We do have an abstract machine, but rather we would need to fit these magic accesses into that semantics, which seems like a lot.
| relocations. This is attractive ergonomically, but it requires a more intrusive | |
| change to MIR and the design of a proper abstract machine with operational | |
| semantics. It also has some similarities to the [`Sized` hierarchy | |
| RFC][sized-hierarchy], which has not yet been accepted. | |
| relocations. This is attractive ergonomically, but it requires a | |
| more intrusive change to MIR and the design of a proper | |
| operational semantics for these relocatable accesses. | |
| It also has some similarities to the [`Sized` hierarchy | |
| RFC][sized-hierarchy], which has not yet been accepted. |
| If the target, backend, or codegen mode cannot emit BTF field relocations, the | ||
| field-info queries fall back to ordinary layout-computed values: | ||
|
|
||
| * `field_byte_offset!` returns the complete field-path offset from the | ||
| current compilation layout. | ||
| * `field_byte_size!` returns the field size from the current compilation | ||
| layout. | ||
| * `field_exists!` returns `true` for a field path present in the current | ||
| compilation layout. |
There was a problem hiding this comment.
This would just become a name resolution question, right? It doesn't have anything to actually do with layout.
|
Syntax questions and my nits aside, this will want a sponsor from T-lang for proceeding as a nightly experiment (which will later come back to re-informing this RFC). I have nominated it so that if you do not find a sponsor in your own time they eventually see it appear on their agenda and one of them can either take up the gauntlet or they can decline it. And this comment as explanation for why I nominated it, of course. This may be a very eventual eventually, so I do suggest finding a sponsor in your own time. |
| On BPF targets with BTF-capable backend support and debug info enabled, these | ||
| queries lower to CO-RE relocations. On targets or backends without BTF | ||
| relocation support, they fall back to the current compilation unit's ordinary | ||
| layout information. |
There was a problem hiding this comment.
Generally speaking, most Rust features that are not very obviously codegen backend specific (e.g., -Cllvm-args), we try to avoid "backend support" as a caveat here. Is there a reason to caveat this in this way? Do we have expected use cases for this being a no-op when not supported?
Maybe this wants to say that if you're compiling repr(Btf) types for a non-bpf target it's a no-op in that case, but on bpf targets we mandate it does what's expected?
(What is the failure mode as a user of this feature if I accidentally don't have BTF relocation support when I expected it? Is it reliably a BPF verifier error?)
There was a problem hiding this comment.
Generally speaking, most Rust features that are not very obviously codegen backend specific (e.g.,
-Cllvm-args), we try to avoid "backend support" as a caveat here. Is there a reason to caveat this in this way? Do we have expected use cases for this being a no-op when not supported?
As already agreed with @workingjubilee in #3966 (comment), I'm going to ditch the idea of a no-op fallback in favor of a compiler error, when BTF relocations are requested, but can't be emitted.
That said, BTF relocations have to be explicitly supported by compiler backends. Both LLVM and GCC support them. But then Cranelift doesn't support BPF at all. For now there is no compiler backend that supports BPF target without supporting BTF relocations, but in theory such combination could exist, and GCC used to be in such state in between version 10 (when BPF targets were introduced) and 12 (when relocations were introduced). If BPF support ever appears in Cranelift, I would also expect its features to be implemented gradually, not all at once. Therefore I don't think we can entirely avoid considering backend support.
Maybe this wants to say that if you're compiling repr(Btf) types for a non-bpf target it's a no-op in that case, but on bpf targets we mandate it does what's expected?
I think what would make sense is emitting an error if repr(Btf) appears during compilation:
- To non-BPF target.
- To BPF target, but with a backend that does not support relocations. Again, no such backend exists now, but GCC used to be like that, and there is non-zero chance Cranelift could fall into this territory in future.
(What is the failure mode as a user of this feature if I accidentally don't have BTF relocation support when I expected it? Is it reliably a BPF verifier error?)
BTF relocations are done by user-space libraries that load the BPF programs (e.g. Aya, cilium/ebpf, libbpf), and they are supposed to perform them before the program is loaded into the kernel. I don't think there is any library that doesn't support BTF relocations, but in theoretical scenario where such library exists, and it loads a BPF program with a BTF-relocatable struct, where the layout carried by the program differs from the kernel, then the original program's bytecode would remain unchanged, the field access would be done according to the old offset, and the verifier would likely detect an invalid field access and reject the program.
There was a problem hiding this comment.
Yeah, saying "the compiler must produce an error if this code generation feature is not supported" seems fine. Especially if failing to handle BTF relocations means the program is rejected as ill-formed by the BPF interpreter anyways? Then there doesn't seem to be much benefit to permitting it.
"The codegen backend is allowed to give you a useless program" is usually something we consider an occasional and avoidable side effect of things which have significant benefits for all programs, but require such permission. Here, there's nothing we're trading that off for.
And assuming we do use an attribute of some kind to indicate these semantics, the cfg_attr case does handle any fallback someone might deliberately want.
|
We talked about this in the lang call today. We were interested to hear what the people working on @rust-lang/rust-for-linux think about this, as well as what those working on reflection (@oli-obk, @scottmcm, rust-lang/rust-project-goals#406) and projection (@BennoLossin, @dingxiangfei2009, @tmandry, rust-lang/rust-project-goals#390) think. If this generally makes sense to people, I'll be happy to champion a lang experiment here. cc @lqd @Nadrieril |
|
Also needs input from sized hierarchy side. cc @davidtwco |
| This overlaps with the accepted [Field Projections project goal][field-projections], | ||
| which is exploring virtual places as a general mechanism for custom field | ||
| projection. This RFC deliberately does not depend on that work: it provides the | ||
| low-level BTF field metadata queries needed for CO-RE today, while leaving a | ||
| future field-projection-based ergonomic surface open. | ||
|
|
||
| Providing the field-info queries proposed in this RFC does not rule out | ||
| exploring this alternative in the future. On the contrary, Clang provides both | ||
| explicit field-info builtins and field projection. It makes sense to treat these | ||
| as separate RFCs. |
There was a problem hiding this comment.
I think this RFC could be written to make it more clear that we expect field access and offset_of! to work in the future. For example, the offset_of! section is phrased as if we could never support offset_of! with btf relocations.
There was a problem hiding this comment.
Good point, I'm going to rephrase this (while still making it clear it's out of scope of this RFC).
|
|
||
| `offset_of!` is intentionally a constant layout query. It does not preserve the | ||
| field identity needed to emit a BTF relocation. Reusing it would either silently | ||
| produce non-relocatable code or require changing the meaning of an existing |
There was a problem hiding this comment.
How is the new macro with the same API change anything? Why isn't that also silently producing wrong things when e.g. a field doesn't exist at all in the relocation table?
There was a problem hiding this comment.
How is the new macro with the same API change anything?
When I submitted the pre-RFC, where I initially proposed emission of BTF relocations in regular field projections and offset_of! , the main feedback was that it goes against the principle that Sized types should have layout known at compile time, which is not the case with BTF-relocatable types.
Then the discussion went towards the hierarchy of Sized types RFC #3729 which would eliminate this problem. But then we also agreed treating BTF-relocatable types as !Sized and allowing access to them only with dedicated macros/intrinsics could be a good way to keep the feature rolling without hard dependency on #3729.
Why isn't that also silently producing wrong things when e.g. a field doesn't exist at all in the relocation table?
Not sure if I got this question right - are you asking about what field_byte_offset! and field_byte_size! macros produce when a field does not exist? That's a great question and actually made me realize that they should probably return an Option:
core::btf::field_byte_offset!(Carrier, field.path) -> Option<usize>
core::btf::field_byte_size!(Carrier, field.path) -> Option<usize>
There was a problem hiding this comment.
Jup, that makes sense now, thx
Thank you, @traviscross, for the discussion and for offering to champion this. I appreciate the lang team's interest. I'm going to modify the RFC according to the comments later today. |
The RFC proposes the `btf_relocations` feature gate, which provides a `#[repr(Btf)]` representation and `core::btf` field-info macros that emit BTF (BPF Type Format)[0] CO-RE (Compile Once, Run Everywhere)[1] relocations. It also documents the relationship to `offset_of!`, ordinary field projection, and LLVM BPF lowering. This feature was originally proposed as a pre-RFC[2]. [0] https://docs.kernel.org/bpf/btf.html [1] https://nakryiko.com/posts/bpf-portability-and-co-re/ [2] https://internals.rust-lang.org/t/pre-rfc-btf-relocations/24161/25
It's better to emit a compile error.
54a1461 to
6292c33
Compare
|
This PR was rebased onto a different master commit. Here's a range-diff highlighting what actually changed. Rebasing is a normal part of keeping PRs up to date, so no action is needed—this note is just to help reviewers. |
View all comments
The RFC proposes the
btf_relocatable_typesfeature gate, that provides a#[repr(Btf)]representation and low-level field-info intrinsics that emit BTF (BPF TypeFormat) CO-RE (Compile Once, Run Everywhere) relocations.It also documents the relationship to
offset_of!, ordinary field projection and LLVM BPF lowering.This feature was originally proposed as pre-RFC.
Rendered