Repository navigation
lab: the agent on a remote worker (CLI over SSH, workspace mirror, proxy tunnel) - #25
Merged
Merged
Conversation
srt cannot jail GPU code on Linux: no device passthrough, and each jail gets its own network namespace, so a served engine is unreachable from the clients. On Linux, agent code (tests, profilers, served engines) now runs in one container per command: NVIDIA runtime, --network none, read-only root, non-root user, no capabilities, cpuset leaving cores for the clients. A served port reaches the host through a socat bridge over a unix socket. srt stays on macOS; LAB_JAIL overrides. vm-setup.sh installs docker + NVIDIA Container Toolkit, builds lab-jail, and checks the GPU inside it. lab/RUNTIME.md is the controller/worker design this starts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A leftover process on the GPU could slow a bench, equiv or submit or stop the engine from starting, and the ledger recorded that as the snapshot's result; a contaminated base measurement was cached and skewed every later submit. serve.Served now refuses to launch while any process holds the GPU (GpuBusy, nothing launched) and polls the GPU's processes every second while serving; anything outside the engine's own process tree (descendants, or the container's processes) makes exclusive() raise Contaminated. bench/equiv/submit/baseline record verdict refused or contaminated with gpu_processes as facts; a contaminated run is never evidence, never cached as the base, and is caught before the holdout guard spends a query. lab.validate's noise bands get the same check. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The GPU work behind test, profile/trace/kernel/hostprof, bench, equiv, submit and the base measurement now runs as worker jobs (lab/worker.py: test, measure, equiv, profile). A job takes input trees and JSON args, writes an out dir and returns JSON; the tools keep the policy, the ledger and the budget and call self.worker. LAB_WORKER=local (default) runs jobs in process, as before. LAB_WORKER=ssh runs them on the lab.vm VM: each tree goes by content hash at most once (has/put), the worker re-hashes a fresh copy before every job and refuses a mismatch, and the out dir comes back as a tar. GpuBusy, Contaminated, NotReady and ValueError are raised again on the controller. equiv ships the snapshot's earlier answers with the job, so a second equiv never re-rolls the gate (it already did not, in process). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…10-09) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…he dependency build step (approved 2026-10-09) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…and the API-key proxy API proxy (lab/apiproxy.py): the agent's CLI holds a placeholder key and talks to a per-session proxy that swaps in the lab's key or OAuth token, forwards only /v1/messages, count_tokens and /v1/models, prices each response's usage (JSON or SSE) at first-party rates, and refuses once the session cap is spent (and any unpriced model or fast mode). The session is charged what the proxy priced. Before, the key sat in the CLI's environment, readable from /proc by the agent's shell. Workbench (Linux): the CLI runs in a per-session container with the GPU, the internet, root inside, writable system dirs; the engine venv, weights and CLI binary read-only so an install never reaches a measurement. vm-setup.sh blocks the cloud metadata service for containers. Pause: measure, equiv and profile jobs kill the workbench's GPU holders, pause it, and thaw it after; what was killed is recorded as workbench_killed and returned as a fact. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…oxy tunnel) With LAB_WORKER=ssh the agent works on the VM while the loop, ledger, referee, budget and API proxy stay on the controller. The workspace's live copy is on the VM, mirrored by rsync by checksum: pushed before each session and after restore or a bundle copy-in, pulled before every lab tool's audit and after the session (links stay links; the audit refuses them). The VM builds its own workbench `docker run` with the Linux CLI of the controller's Agent SDK version; cli.sh is `ssh -T -R <sock>:127.0.0.1:<proxy> vm ...`, so the CLI's stdio is the SDK's and lab tools stay in process here. The proxy reaches the workbench through a unix socket and a socat on the container's own loopback; it listens on no interface of the VM. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts: # CLAUDE.md # lab/LEDGER.md # lab/README.md # lab/RUNTIME.md # lab/agent.py # lab/evaltools.py # lab/proftools.py # lab/safety/container.py # lab/tools.py # lab/vm-setup.sh # lab/worker.py # tests/test_lab_container.py # tests/test_lab_evaltools.py
# Conflicts: # tests/test_lab_evaltools.py
# Conflicts: # CLAUDE.md # lab/evaltools.py # lab/proftools.py # lab/tools.py # lab/worker.py
# Conflicts: # lab/session.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #24. With
LAB_WORKER=sshthe agent works on the VM. The loop, ledger, referee, budget and API proxy stay on the controller.--delete);restoreor a bundle copy-in;python -m lab.worker workbenchbuilds the VM's owndocker run, with the Linux CLI of the controller's Agent SDK version installed once with uv;cli.shisssh -T -R <sock>:127.0.0.1:<proxy> vm '<docker run…>' <SDK args>. The CLI's stdio is the SDK's, so the lab tools stay in process on the controller.workbenchcommand;sshstand-in that prints the remote command;🤖 Generated with Claude Code