Problem
The cloud-init UserData script in `internal/workspace/bootstrap.go` only installs harnessd if the `HARNESS_DOWNLOAD_URL` environment variable is set:
```bash
if [ -n "${HARNESS_DOWNLOAD_URL}" ]; then
curl -fsSL "${HARNESS_DOWNLOAD_URL}" -o /usr/local/bin/harnessd
chmod +x /usr/local/bin/harnessd
fi
```
But nothing in the codebase ever sets that variable. The runner calls `harnessBootstrapScript()` and passes its return value as `UserData`, with no env injection. So:
- VM boots successfully on Hetzner.
- cloud-init runs apt-get + creates the workspace dir.
- The if-branch is skipped (URL is empty).
- The fallback expects harnessd at `/usr/local/bin/harnessd` but stock `ubuntu-24.04` has no such binary.
- Systemd starts the harnessd unit, which fails immediately with "executable not found".
Net effect: `workspace_type=vm` provisions a real VM, the agent does its work on the harness host (not on the VM — see #564), and the VM sits idle until `Destroy` removes it. Useless and billable.
Verified live this session: a Hetzner cax11 booted at 178.105.38.116, the run completed without ever talking to the VM, and the agent's file write landed at `/tmp/harness-vm-test/VM_PROOF.txt` (host) rather than on the VM.
Acceptance criteria
- `workspace_type=vm` provisioning produces a VM where `curl http://:8080/healthz` returns `{"status":"ok"}` within 90 seconds of `workspace.provisioned` firing.
- No external dependency on a manually-set env var to make this work — defaults are sane.
- Cleanly handle ARM vs x86 (Hetzner cax* is ARM, cpx* is x86). The default server type after "fix(workspace): make Hetzner VM mode work" is `cax11` (ARM).
Proposed approaches
Pick whichever is simplest given current build infra:
Option A — Pull the harnessd Docker image (recommended).
Build a multi-arch `go-agent-harness:latest` image (already exists for amd64; need to add linux/arm64). Bootstrap installs Docker, pulls the image, and runs it as a systemd unit:
```bash
curl -fsSL https://get.docker.com | sh
docker pull go-agent-harness:latest
docker run -d --name harnessd --restart=on-failure -p 8080:8080 -v /workspace:/workspace go-agent-harness:latest
```
Reuses the same image as container mode. Works for any provider that supports Docker (Hetzner, AWS, GCP, etc.).
Option B — Download a binary from GitHub Releases.
Set `HARNESS_DOWNLOAD_URL` automatically in the bootstrap script to a pinned release URL. Requires CI to publish multi-arch binaries on each release.
Option C — Pre-baked Hetzner snapshot.
Build a snapshot once with harnessd pre-installed at `/usr/local/bin/harnessd`. Set `opts.ImageName` to that snapshot ID. Cheapest at runtime; least flexible (locks us to Hetzner).
Recommendation: Option A. Same image as container mode, no separate publish pipeline, works on any Docker-capable cloud.
Related
Validation
- Add an integration test gated on `HETZNER_API_KEY`: provision a vm-mode workspace, poll `:8080/healthz`, expect 200 within 90s, destroy.
- Manual: `curl http://:8080/v1/runs -d '{"prompt":"hi","max_steps":2}'` should return a run-id from the VM's harnessd, not the host's.
Problem
The cloud-init UserData script in `internal/workspace/bootstrap.go` only installs harnessd if the `HARNESS_DOWNLOAD_URL` environment variable is set:
```bash
if [ -n "${HARNESS_DOWNLOAD_URL}" ]; then
curl -fsSL "${HARNESS_DOWNLOAD_URL}" -o /usr/local/bin/harnessd
chmod +x /usr/local/bin/harnessd
fi
```
But nothing in the codebase ever sets that variable. The runner calls `harnessBootstrapScript()` and passes its return value as `UserData`, with no env injection. So:
Net effect: `workspace_type=vm` provisions a real VM, the agent does its work on the harness host (not on the VM — see #564), and the VM sits idle until `Destroy` removes it. Useless and billable.
Verified live this session: a Hetzner cax11 booted at 178.105.38.116, the run completed without ever talking to the VM, and the agent's file write landed at `/tmp/harness-vm-test/VM_PROOF.txt` (host) rather than on the VM.
Acceptance criteria
Proposed approaches
Pick whichever is simplest given current build infra:
Option A — Pull the harnessd Docker image (recommended).
Build a multi-arch `go-agent-harness:latest` image (already exists for amd64; need to add linux/arm64). Bootstrap installs Docker, pulls the image, and runs it as a systemd unit:
```bash
curl -fsSL https://get.docker.com | sh
docker pull go-agent-harness:latest
docker run -d --name harnessd --restart=on-failure -p 8080:8080 -v /workspace:/workspace go-agent-harness:latest
```
Reuses the same image as container mode. Works for any provider that supports Docker (Hetzner, AWS, GCP, etc.).
Option B — Download a binary from GitHub Releases.
Set `HARNESS_DOWNLOAD_URL` automatically in the bootstrap script to a pinned release URL. Requires CI to publish multi-arch binaries on each release.
Option C — Pre-baked Hetzner snapshot.
Build a snapshot once with harnessd pre-installed at `/usr/local/bin/harnessd`. Set `opts.ImageName` to that snapshot ID. Cheapest at runtime; least flexible (locks us to Hetzner).
Recommendation: Option A. Same image as container mode, no separate publish pipeline, works on any Docker-capable cloud.
Related
Validation