relay-worker-miniflare tests hang on unbounded mf.ready under suite load #176

relay-worker-miniflare.test.ts intermittently fails with a 30s timeout, locally and in CI. The failing test varies between runs; the accompanying output is always workerd/server/server.c++: error: disconnected: ... Broken pipe and a killed 1 dangling process.

The hang is in Miniflare startup, not in the websocket exchange the test is named for.

Mechanism

Every timeout inside the harness is bounded — socket open at 5s, waitForControl at 5s. The observed failure is 30s, which is bun's default test timeout, so none of those fired. The only unbounded awaits in the file are await mf.ready, in startWorker and again in connectWorkerSocket.

Under full-suite load, workerd startup can be starved badly enough that mf.ready never settles. Four tests each start their own worker, so whichever one loses the race hangs — which is why the failing test name moves around.

Reproduction

Race mf.ready against an 8s timeout in startWorker:

await Promise.race([
  mf.ready,
  new Promise((_, rej) =>
    setTimeout(() => rej(new Error('MFREADY_TIMEOUT_startWorker')), 8000),
  ),
])

Then run the full suite (bun run test, not the file alone):

error: MFREADY_TIMEOUT_startWorker
(fail) relay Worker under Miniflare > websocket hash mismatch returns 409 before upstream fetch [8001.51ms]
Ran 1359 tests across 45 files. [59.71s]

The instrumented timeout fires on exactly the test that otherwise sits for 30s. A second full run passed, and the file alone passes 4/4 repeatedly — it needs the concurrent load to reproduce.

Measured at 99de290, but it reproduces on a near-upstream tree with none of my changes.

Suggested fix

Bound mf.ready and retry startWorker once on timeout. Startup starvation is transient, so a retry after the spike should succeed, and a bounded failure reports the real cause instead of a bare 30s timeout on an unrelated assertion. Happy to open a PR if you want it.

Correction to my own earlier notes

I characterised this failure three times on PRs #174 and #175 and was wrong each time — "always the last of the four tests", "deterministic, not a flake", and "CI passes it consistently". The first two came from measuring without controlling machine load; the third has since been contradicted by a CI failure. All three were about the test's subject rather than its setup, which is where the fault actually is. Anywhere those claims are quoted, this is the accurate version.