Skip to content

lab: container jail on Linux; runtime design (RUNTIME.md) - #19

Merged
AdvayMonga merged 2 commits into
mainfrom
design/controller-worker
Oct 10, 2026
Merged

AdvayMonga merged 2 commits into
mainfrom
design/controller-worker

Conversation

@AdvayMonga

Copy link
Copy Markdown
Owner

Why

srt can't jail GPU code on Linux:

  • it mounts a fresh /dev with no device passthrough, so jailed code sees no GPU (found on the H200, Profiler tools: fixes from the H200 run #18);
  • each srt jail gets its own network namespace, so a jailed engine can't be reached by the measurement clients (allowLocalBinding only works on macOS). Reproduced in Docker on Linux.

Together these mean test, profile/trace/kernel/hostprof, bench, equiv and submit can't run jailed on a GPU VM.

What

  • lab/safety/container.py: on Linux, one container per jailed command. NVIDIA runtime (--gpus when nvidia-smi exists), --network none, read-only root, non-root user (uid 10001 when the lab runs as root), all capabilities dropped, no-new-privileges, a process limit, and a cpuset that leaves the first 4 cores to the measurement clients.
    • A served engine keeps binding 127.0.0.1. A socat inside the container serves that port on a unix socket, and a host socat serves the socket on the host's 127.0.0.1.
    • srt stays on macOS. LAB_JAIL picks the jail; LAB_NO_JAIL=1 behaves as before.
  • lab/safety/jail.Dockerfile (lab-jail image) and python -m lab.safety.container build.
  • vm-setup.sh installs Docker and the NVIDIA Container Toolkit, builds the image, and prints jail: gpu ok or jail: no GPU in the container.
  • lab/RUNTIME.md: the controller/worker design this starts (trust zones, rules, job interface).

Tested

  • Docker-in-Docker on Ubuntu 24.04: no network, no capabilities, non-root, host files invisible, the served port reaches the host and no other jail, nothing left behind after a timeout or teardown. Lab suite: 223 passed. The 2 failures and 4 errors are from the box: its old apt pytest, and a missing git SHA in the copy.
  • macOS: the affected test files pass, ruff is clean.
  • Not tested: --gpus and the profilers inside the container. That needs a GPU run (vm-setup.sh checks it).

Not in this PR

  • The agent's CLI still runs under srt, so on Linux its shell has no GPU yet.
  • The controller/worker split.
  • GPU exclusivity during measurements comes next, in its own PR.

🤖 Generated with Claude Code

srt cannot jail GPU code on Linux: no device passthrough, and each jail gets its own
network namespace, so a served engine is unreachable from the clients. On Linux,
agent code (tests, profilers, served engines) now runs in one container per command:
NVIDIA runtime, --network none, read-only root, non-root user, no capabilities,
cpuset leaving cores for the clients. A served port reaches the host through a socat
bridge over a unix socket. srt stays on macOS; LAB_JAIL overrides.

vm-setup.sh installs docker + NVIDIA Container Toolkit, builds lab-jail, and checks the
GPU inside it. lab/RUNTIME.md is the controller/worker design this starts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@AdvayMonga
AdvayMonga merged commit faabaa8 into main Oct 10, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant