Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions docs/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,20 @@ recorded is indistinguishable from law that was never agreed.

### Added

- `docs/rfcs/0004-ipc.md` — **proposed**, the second half of the kernel's core and the paper "IPC is
the product" is argued on. Endpoints are RFC-0003 capability objects: send/receive rights on the
same endpoint give a client/server split for free, and there is no global endpoint registry the
kernel arbitrates, so IPC is the enforcement surface rather than a hole beside it. The base primitive
is a synchronous unbuffered rendezvous (L4) — no kernel message buffer, which is what discharges O-7
(nothing to flood, nothing to size) and avoids the multi-copy-IPC grave (Mach). A register fast path
carries small messages with no allocation and no copy; a bounded slow path copies now and can map
(zero-copy) once the MMU exists, behind an ABI that hides the choice. Messages move capabilities as
RFC-0003 moves them — atomically, `TRANSFER`-gated, fail-closed. `call` with single-use reply
capabilities gives RPC without ambient "who called me" state (O-4 preserved), and bounded
notifications give async signalling without payload or flood. Open questions named rather than
solved: slow-path copy-vs-map (joins the MMU RFC), multi-core rendezvous (the hardest, joins the
scheduler), timeouts, and the exact register budget (joins the syscall ABI RFC).

- `docs/rfcs/0003-capability-table.md` — **accepted** (2026-07-30; selective revocation's final
verdict expressly deferred to RFC-0003a), the first Phase 1 design RFC and the first to
cite the threat model's obligations by number, discharging O-1, O-2 and O-4 and part of O-3. Argues
Expand Down
227 changes: 227 additions & 0 deletions docs/rfcs/0004-ipc.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,227 @@
<!-- SPDX-License-Identifier: GPL-3.0-or-later -->

# RFC-0004 — Inter-process communication

| Field | Value |
|-------|-------|
| Status | **Proposed** — the paper the design is argued on, awaiting the maintainer's verdict |
| Author | Drafted by Claude Code as sparring partner; verdict the maintainer's |
| Date | 2026-07-30 |
| Affects | Constitution §3 (kernel doctrine, "IPC is the product"); the syscall ABI; the scheduler; every userspace service |
| Depends on | RFC-0003 (capability table) — endpoints are objects, transfer moves capabilities |
| Discharges | Contributes to O-5 (argument validation), O-7 (no unprivileged exhaustion), O-9 (no ambient side channel); the substrate for O-10 (broker) and O-17 (drivers) |

## 1. The question

**How does one process send a message to another — small ones fastest of all — such that the transfer
is safe, carries capabilities correctly, and never lets a sender starve or stall the kernel or the
receiver?**

§3 is unusually emphatic here: *"IPC is the product."* Every service, every driver, every broker call
is an IPC. If IPC is slow, the whole microkernel premise fails; if IPC is unsafe, the capability
system leaks. This RFC designs the mechanism. It does **not** design any particular protocol spoken
over IPC — that is userspace's business — only the kernel primitive they all rest on.

## 2. Which pillar

**The kernel doctrine directly (§3), and pillar 2 by consequence.** Capabilities are the only
authority (RFC-0003), and IPC is how authority *moves* between processes: an endpoint is a capability,
and a message may carry capabilities. IPC is therefore not a feature beside the capability system — it
is the capability system in motion. The two RFCs are halves of one mechanism.

## 3. Endpoints are objects, invoked by capability

An **endpoint** is a kernel object (RFC-0003 §3): a rendezvous point threads send to and receive from.
It is named, like everything, by a capability:

- A capability to an endpoint with `WRITE` (`SEND`) may send to it.
- A capability to an endpoint with `READ` (`RECV`) may receive from it.
- The same endpoint, handed out with different rights, gives a natural client/server split: the server
holds the receive capability; clients hold send capabilities derived from it. A client cannot
receive on the server's endpoint because it was never given `READ` — O-4 and O-2, for free, from the
capability model already accepted.

There is no global name for an endpoint, no port number, no registry the kernel arbitrates. A process
can send to exactly the endpoints it holds send-capabilities for, and no others. This is what makes
IPC the enforcement surface rather than a hole beside it.

## 4. Synchronous rendezvous as the base primitive

The base operation is a **synchronous, unbuffered rendezvous** — the L4 lesson, and the same choice
the earlier C++ blueprint reached for:

- A sender blocks until a receiver is ready; a receiver blocks until a sender is ready. The message
transfers directly, sender to receiver, in one operation. There is **no kernel buffer** holding
messages in flight.
- This is the direct sender→receiver switch §3 names: on a completed rendezvous the kernel can hand
the CPU straight to the receiver (§6), because the receiver is exactly the thread that should run
next and its data is already in place.

Why synchronous-first, when asynchronous notification is also wanted (§8)? Because the synchronous
rendezvous is the primitive that is both fastest *and* safest: no buffer means no buffer to size, no
buffer to account against a quota, no buffer for a sender to flood (O-7), and no copy into and out of
kernel storage (the multi-copy IPC grave — Mach — §11). Asynchronous patterns are built *on* this
base as an explicit, bounded mechanism, not baked into it.

The cost is real and stated: pure synchronous IPC couples sender and receiver scheduling, and naïve
use invites deadlock (two processes each waiting to send to the other). §8's `call` pattern and the
non-blocking variants exist to manage that, and the broker/driver designs must respect send/receive
ordering. This is the tax L4 pays for its speed, and Setonix pays it deliberately.

## 5. The message: a register fast path and a bounded slow path

A message is small metadata plus a payload. Two payload paths, chosen by size:

### Fast path — registers only

Small messages travel in **CPU registers**, never touching memory. On AArch64 the argument registers
`x0`–`x7` carry a message tag plus up to a handful of payload words; the kernel transfers them
sender-to-receiver as part of the context switch it was going to do anyway. §3's "register-based fast
path for small messages" and the blueprint's "~100 cycle" target live here. The exact register budget
is an ABI question (§10) settled with the syscall RFC, but the shape is: **the common case — a short
request, a small reply — allocates nothing and copies nothing beyond the registers.**

### Slow path — bounded, capability-named buffers

Larger payloads use a memory buffer the *sender already holds a capability to* — not a kernel buffer.
Two sub-options, and this is a genuine open choice (§9):

- **Copy** the buffer sender→receiver through the kernel (one copy, bounded by a maximum message
size). Simple, always safe, but it is a copy.
- **Map** (zero-copy): the kernel remaps the sender's pages into the receiver read-only for the
duration, per §3's "zero-copy page transfer for large ones". Faster for big transfers, but it
entangles IPC with the MMU and address-space lifetime, which do not exist yet.

**Proposed:** ship the copy path first (bounded, simple, needs no MMU), and add the map path when the
MMU lands, behind the same syscall — the caller says "here is a buffer capability", and whether the
kernel copies or maps is an implementation choice invisible to the ABI. This keeps IPC buildable now
and defers the MMU entanglement to when the MMU is real. Recorded as the recommendation; the alternative
(hold IPC's large path until the MMU exists) is rejected because it would block the first userspace
service on the MMU.

## 6. Capability transfer in a message (RFC-0003 §6, cashed out)

A message may carry capabilities as well as data. This is how a client hands a server a reply
endpoint, how the broker hands an app a granted resource, how authority propagates at all.

- The sender names capabilities to transfer by handle; the kernel **moves** them (RFC-0003 §6) from
the sender's table into the receiver's, assigning fresh handles in the receiver. `TRANSFER` right is
required; a capability without it cannot be sent, and the send fails closed.
- Transfer is atomic with the message: either the whole message and all its capabilities arrive, or
none of it does. There is no state where the data arrived but a capability was lost, or a capability
moved but the message did not.
- Because transfer is a move, the kernel-side accounting is exactly RFC-0003's: no `Clone`, the
capability leaves one table and enters another, and the borrow checker already models the invariant.
IPC is where that move actually happens at runtime.

## 7. Safety: what the kernel validates (O-5)

Every IPC syscall is an attacker-controlled crossing of B1 (kernel/userspace). The kernel validates,
before it acts:

- **The endpoint handle** resolves to a capability the caller holds, carrying the right for the
operation (`SEND` to send, `RECV` to receive). Generation-checked (RFC-0003 §8); a stale handle
fails closed.
- **Every transferred capability handle** resolves and carries `TRANSFER`. One bad handle fails the
whole send; nothing partial.
- **The payload length** is within the fixed maximum; the buffer capability (slow path) grants the
claimed range. No caller-supplied length is trusted against a smaller buffer.
- **No kernel pointer is ever supplied by userspace.** The caller names registers and handles; the
kernel supplies every address. This is O-5 for the IPC surface, and it is the single most
security-critical validation in the kernel because IPC is the surface most exercised.

## 8. Beyond the base: call, and asynchronous notification

Two derived operations, built on the rendezvous, not beside it:

- **`call` (send-and-wait-reply).** The overwhelmingly common RPC shape: a client sends a request and
blocks for the reply, atomically. The kernel hands the client a single-use **reply capability** to
give the server, so the server can reply exactly once to exactly this caller without holding a
standing capability to it. This is the L4/seL4 `Call`, and it is also what makes the direct switch
(§6) pay off: `call` → server runs → server replies → client runs, with the CPU handed along the
chain and no scheduler round-trip. Reply capabilities are the mechanism that keeps `call` from
needing ambient "who called me" state — O-4 preserved.
- **Notification (asynchronous, bounded).** Some patterns — an interrupt arriving, a driver signalling
readiness — must not block the sender. A **notification** is a bounded, non-blocking signal: it sets
a bit (or a small counter) on a notification object the receiver holds, and wakes it if waiting. It
carries no payload and no capability — it is a doorbell, not a message. Bounded by construction (a
fixed-width word of pending bits), so it cannot be used to flood (O-7). This is seL4's notification /
Zircon's signal, kept deliberately minimal: the moment a notification wants to carry data, the answer
is "then send a message", not "grow the notification".

## 9. Open questions — the next design work

1. **Slow-path copy vs map** (§5): settled jointly with the MMU RFC. The ABI is designed to make the
choice invisible to callers, so this is an implementation decision deferred, not a design fork that
blocks.
2. **Multi-core rendezvous** (RFC-0003 §11's open question, inherited here): a sender on one core and a
receiver on another. The synchronous switch is trivial same-core; cross-core needs a defined story
(IPI wakeups, per-core run queues) that couples to the scheduler. This is the hardest open question
and likely its own RFC once SMP is on the table.
3. **Timeouts.** Should a blocking send/receive take a timeout, or is that policy a userspace timer
capability imposes? L4 has gone both ways across versions. Proposed: no kernel timeout in the base
primitive (keep it minimal); a process that wants one holds a timer capability and composes it —
but flagged as genuinely arguable.
4. **The register budget** (§5): exactly how many payload words the fast path carries, settled with the
syscall ABI RFC. Too few and common messages spill to the slow path; too many and every context
switch saves registers it need not.

## 10. The syscall surface this implies

RFC-0003 §9 constrained the syscall ABI to be capability-indexed; this RFC names the IPC syscalls that
constraint produces. A minimal set, each taking handles and registers only:

| Syscall | Meaning |
|---------|---------|
| `send(ep, msg)` | block until a receiver takes the message on endpoint `ep` |
| `recv(ep) -> msg` | block until a sender delivers on `ep` |
| `call(ep, msg) -> reply` | send, then block for a reply via a kernel-minted single-use reply capability |
| `reply(reply_cap, msg)` | reply once to a `call`er |
| `notify(notif_cap)` | non-blocking: signal a notification object |
| `recv` with a notification bound | receive a message *or* wake on a notification |

Non-blocking `send`/`recv` variants (fail rather than block) are likely needed for drivers and are
flagged for the syscall RFC. The exact encoding — one syscall with a mode, or several — is that RFC's
to settle; this RFC fixes the *semantics*, not the numbers.

## 11. Graves checked (§3)

- **Multi-copy IPC (Mach) — the grave this RFC most directly avoids.** The fast path copies nothing
beyond registers; the slow path copies at most once (or maps, zero-copy). There is no kernel message
buffer, no double-copy in and out of kernel storage. This is the single most important thing to get
right, and §4–§5 are built around it.
- **Policy in the kernel.** The kernel provides send/recv/call/notify. *Which* endpoints exist, *who*
may talk to whom, *what protocol* is spoken — all userspace. The kernel arbitrates no names and
enforces no protocol.
- **Baroque capability hierarchies.** Endpoints and reply capabilities use RFC-0003's flat model
unchanged; nothing here adds capability structure.
- **Bolted-on multicore.** Named as the hardest open question (§9.2) and coupled to the scheduler now,
rather than assumed away and retrofitted.

## 12. Costs — what this makes harder

- **Synchronous-first couples scheduling and invites deadlock.** Real, and paid deliberately (§4);
`call` and non-blocking variants are the management, and userspace must respect ordering.
- **The fast path constrains the ABI forever.** Once services depend on N payload registers, changing
N is a breaking change. §9.4 is where that number gets chosen carefully.
- **No kernel buffering means no "fire and forget" for messages.** A sender that wants to not block
must use notification (no payload) or a userspace buffering service. This is a real ergonomic cost,
and it is the cost that buys O-7 and the no-Mach-grave guarantee.

## 13. Obligations discharged

| Obligation | This RFC | Status after implementation |
|-----------|----------|-----------------------------|
| O-5 argument validation | every IPC syscall validates handles, rights, lengths; no userspace pointers (§7) | **discharged for the IPC surface** |
| O-7 no unprivileged exhaustion | no kernel message buffer; notifications bounded; synchronous rendezvous allocates nothing per-message (§4, §8) | **discharged for IPC** |
| O-9 no ambient side channel | processes reach only endpoints they hold capabilities for; no global registry (§3) | **contributes** — IPC adds no ambient channel |
| O-10 confused-deputy / broker | reply capabilities and capability-carrying messages are the broker's substrate (§6, §8) | **enables** — the broker RFC builds on this |
| O-17 contained drivers | drivers are processes speaking IPC; notification carries interrupts (§8) | **enables** — the driver framework builds on this |

## 14. What this unblocks

With IPC and capabilities both settled on paper, the two halves of the kernel's core are designed. The
next papers, roughly in dependency order: the **MMU / address-space RFC** (which §5 and §9.1 wait on),
the **scheduler RFC** (which §6 and §9.2 wait on), and then the **broker RFC** (which §6 and §8 are
built to serve). The first *code* that could land now is the capability table itself (RFC-0003) or the
frame allocator; IPC's own implementation waits on threads existing to rendezvous between.