lab: worker job interface (local and SSH transports)

[AdvayMonga]

Stacked on #20, which is stacked on #19. Retarget once those merge.

What

lab/worker.py: the job interface from lab/RUNTIME.md, built for the measured jobs.

Job	Inputs	Returns	Out dir
test	tree	{lint, tests}
measure	tree	{results, ready_s}	engine log; device/telemetry/client rows when passive
equiv	tree, reference, prior	the gate's result	equiv.json, equiv.outputs.jsonl, passive data
profile	tree	{returncode, output, …}	the bundle or instrument files

• bench, submit and the base measurement share measure. profile, trace, kernel and hostprof share profile; the instrument definitions moved into a pure proftools.instrument(kind, args).
• The tools keep the policy, the ledger and the budget, and call self.worker. Ledger records and the agent-facing text are unchanged.
• LAB_WORKER=local (default) runs jobs in process.
• LAB_WORKER=ssh runs them on the VM lab.vm names:
  • each input tree goes by content hash, at most once per worker (has, put);
  • the worker re-hashes a fresh copy right before each job and refuses a mismatch, so what is measured is what the controller audited;
  • the out dir comes back as a tar;
  • GpuBusy, Contaminated, NotReady and ValueError are raised again on the controller.

Kept on purpose

Calling equiv again on the same snapshot reuses its earlier answers; it never re-rolls the statistical gate. In process this came from the answers file living in the run dir. With a fresh out dir per job it would have been lost, so equiv now ships those answers as the prior input. A new test marks the stored answers and checks the mark survives a second equiv; it fails without the fix.

Tested

• Remote end to end through a local shell running the same python -m lab.worker CLI that SSH would run:
  • a job runs on the shipped tree, and the tree ships once;
  • GpuBusy, with its facts, crosses from a fake nvidia-smi;
  • NotReady crosses, and the engine log still comes back;
  • a cached tree that changed after put is refused;
  • a put whose content doesn't match its hash is refused.
• All existing lab tests pass on the local transport. macOS: 235 passed. Linux (Docker-in-Docker, container jail live): the worker tests pass inside the jail; the remaining failures are the test box's (old pytest, no git SHA).
• Not tested: LAB_WORKER=ssh against a real VM.

Not in this PR

The agent's side on the worker: the agent container with freeze-before-measure, the API-key proxy, and the agent/exec/snapshot/restore/facts jobs.

🤖 Generated with Claude Code