Skip to content

fdguard macOS: adopt POSIX_SPAWN_CLOEXEC_DEFAULT to close the descriptor race in-kernel (follow-up to #352) #358

Description

@hartsock

Follow-up referenced by the fdguard crate docs and the assurance register after #352. #352 deliberately kept the parent-computed FD_CLOEXEC sweep and rejected this primitive for that PR; this issue records why, and what would have to change to adopt it.

The rejection was not doubt about the primitive. On the evidence below it is strictly better than the sweep. It was rejected because adopting it is an architectural slice, not a mechanism swap.

Native evidence (real Apple silicon, macOS 26.5.2 / build 25F84, Darwin 25.5.0, xnu-12377.121.10, arm64)

POSIX_SPAWN_CLOEXEC_DEFAULT is available (<spawn.h>, 0x4000) and posix_spawnattr_setflags accepts it. Probe method: parent opens descriptors with open("/dev/null", O_RDONLY) and asserts FD_CLOEXEC is clear (genuinely inheritable), then spawns a child that walks fd 0..65535 with fcntl(F_GETFD) and reports what it can see.

case setup child sees
A no flag, parent holds inheritable 3,4,5,6,7 0 1 2 3 4 5 6 7 — baseline leak
B flag + dup2(0,0)/(1,1)/(2,2) file actions 0 1 2 — all five closed
C flag + explicit adddup2(3 -> 9) 0 1 2 9 — file actions still win; the flag is a default, not an override
D flag + a second thread calling open() 64x during the spawn 0 1 2 (and 0..71 without the flag)
E/F parent dup2s to 61439, one below kern.maxfilesperproc no flag → 0 1 2 3 61439; with flag → 0 1 2

Two properties the sweep cannot have:

  1. The race is closed in the kernel, between fork and exec inside the spawn implementation — there is no parent-side interval for another thread to open or dup an inheritable descriptor into the child (case D). This is strictly stronger than serializing a critical region.
  2. It is range-independent (case E/F). A numeric sweep is only ever as good as its bound; this primitive does not care where the descriptor sits. Adopting it would make the whole getrlimit-vs-kern.maxfilesperproc bound question stop being load-bearing for the inheritance property.

Why #352 did not adopt it

std::process::Command exposes no posix_spawnattr seam at all. The flag cannot be requested through std even when std does take its posix_spawn fast path. (Separately, requesting pre_exec forces the fork/exec path outright, so a sweep-shaped implementation and this primitive are mutually exclusive by construction.)

Therefore adoption requires:

  • calling libc::posix_spawn directly (or via a crate exposing attrs) and giving up std::process::Child, at all three confined-spawn sites;
  • explicit stdio file actions. With the flag and no file actions, stdin/stdout/stderr are closed too — a probed child ran and exited 0 while producing no output because fd 1 was gone. A confined child that silently loses stderr is a vicious failure mode;
  • a separate exec-failure path from the Linux leg. Under posix_spawn an exec failure surfaces as a nonzero return from posix_spawn itself rather than through std's CLOEXEC status pipe. That is arguably a cleaner reporting story, but it is a different one, so the two legs would share neither spawn code nor error-path code.

Acceptance criteria

  • All three confined-spawn sites migrated, or an explicit decision recorded for any that are not.
  • stdio survival asserted natively, not assumed — including that a failed exec is still reported as an error rather than a spawned-then-died child. (Prior art: the Linux leg's close_range(flags=0) bug closed std's CLOEXEC exec-status pipe and turned Err(NotFound) into Ok + SIGABRT.)
  • Case D reproduced as a native regression test — a thread opening descriptors concurrently with the spawn, asserting the child sees only its intended stdio.
  • Case E/F reproduced: a descriptor near kern.maxfilesperproc is closed without the test relying on any derived bound.
  • The fail-closed refuse path retained for platforms/configurations where the mechanism is unavailable.
  • Assurance register updated: ASM-POSIX-DESCRIPTOR's macOS leg re-premised on the in-kernel mechanism rather than a bounded sweep, and the macos-rlimit-raise-race residual named in fdguard: macOS close-on-spawn leg — bounded FD_CLOEXEC sweep (#319) #352 re-evaluated — it should be dissolved rather than mitigated, since the primitive does not consult a bound.

Do not adopt mechanically

State the theorem before changing the mechanism. The sweep is not wrong; it is bound-dependent and has a parent-side window. If the migration cannot deliver stdio file actions and honest exec-failure reporting at every confined-spawn site, keeping the sweep and documenting the residual is the better outcome.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions