fix(sdk): don't SIGKILL a database dump copy after 30 seconds

[helix-nine]

Fixes #3636.

The bug

SubContainer.exec defaults timeoutMs to 30 000 and enforces it with SIGKILL. execFail declares timeoutMs with no default and forwards it, so any call site that omits the argument silently inherits 30 s.

336eda75 (1.5.2) split "pg_dump writes onto the backup target" into "pg_dump writes to /tmp, then cp through the FUSE". The explicit null stayed on pg_dump; the new cp — the step whose duration is set by the size of the dump and the speed of the target — got none. Once a dump grew past a thirty-second copy, every backup failed:

Failed: Unknown Error: Error: cp terminated with signal SIGKILL:

Restore stages the dump off the target through the same kind of copy under the same cap, so a database whose dump took longer than 30 s to copy could not be restored at all — discovered during recovery, when the backup is all the user has.

What changed

Every step of a dump or restore whose duration follows the data now opts out, through a named NO_TIMEOUT (which also replaces the file's existing bare nulls, so the opt-out is greppable): both copies, pg_ctl start/stop, the recursive chowns over the data directory, initdb, mysql_install_db / mysqld --initialize-insecure, and the foreground mysqld MariaDB runs for the length of the dump. The steps left at 30 s touch a fixed, tiny number of inodes (touch, mkdir -p, rm -f, single-file chown, createdb, psql ALTER USER) or are already bounded by their own poll loop (pg_isready, mysqladmin ping).

readyTimeout now reaches pg_ctl. pg_ctl bounds itself — -w is its default and it gives up after -t seconds, 60 by default — so lifting the SDK cap alone would only have moved the ceiling 30 s → 60 s, out of reach of the one knob packages are told to raise for slow clusters. readyTimeout now supplies -t. Its 60 000 ms default is pg_ctl's own default, so behaviour is unchanged until someone raises it.

A timeout kill now says it timed out. The error was built from the signal alone, so the SDK's own timer was indistinguishable from an OOM kill, a cgroup kill, or an operator's kill -9 — and cp writes nothing to stderr when killed, leaving the bare cp terminated with signal SIGKILL: above as the entire notification. exec's result now carries timedOutAfterMs and the message names the limit:

cp timed out after 30000ms and was killed with SIGKILL:

The flag is decided in the exit handler, not the timer: the deadline can elapse after something else has already killed the process (the 'exit' event is delivered in the poll phase, after timers), and setting it from the timer reported an external SIGTERM as an SDK timeout — the same misattribution inverted. timedOutAfterMs is set only when our timer fired and the process died of SIGKILL.

Docs. The packaging guide documented exec/execFail without ever mentioning the 30 s default — the omission this issue is made of, and its example was a git clone. It now documents the third argument. Two TSDoc claims that 1.5.2 falsified are corrected: the dump is staged in /tmp rather than "written directly to the backup target — no data duplication on disk", and withMysqlDump referred to a dumpVolume option that does not exist in MysqlDumpConfig.

While in the file, the four hand-rolled structural sub: parameter types in Backups.ts are replaced with the real SubContainer<M> — one of them had to be widened by hand just to accept the new third argument.

Verification

• make check, make test (87), make check-fmt, repo-root prettier --check, make bundle — all clean. New exitError.test.ts covers the three message branches.
• pg_ctl's self-bounding was measured, not assumed: in postgres:16-alpine, a smart stop held open by a live client and PGCTLTIMEOUT=5 returned exit 1 at exactly 5 s. So dropping the SDK cap on pg_ctl cannot hang a backup — it converts a SIGKILL into a real pg_ctl error.
• Worth knowing, and not addressed here: the SDK's SIGKILL lands on the host-side start-container subcontainer exec wrapper. SIGKILL cannot be in that wrapper's forwarded-signal set (FWD_SIGNALS), and it setnses into the subcontainer's PID namespace before spawning the real command, so the cp itself is not signalled — it is orphaned until the subcontainer is torn down. That is why the timeout note in the guide describes a timeout as "the SDK stopped waiting", not "the work stopped".

Not verified end-to-end on a server with a multi-gigabyte dump; the failing path is a wall-clock timer, which is impractical to exercise here.

[dr-bonez]

can we make this part of the named opts instead?

[helix-nine]

Done — 64127f292.

timeoutMs and abort both move into the options object, so this site is now:

await sub.execFail(
  ['pg_ctl', 'start', '-D', pgdata, '-t', pgCtlTimeout, '-o', pgStartOpts],
  { user: 'postgres', timeoutMs: null },
)

That also retires the NO_TIMEOUT constant I'd added — it only existed to make the bare positional legible, and timeoutMs: null says it directly.

I moved abort along with it because dropping only timeoutMs would have left abort sliding into third position, where a caller passing the old third argument would hit a type mismatch rather than a clear error. Passing either positionally is now a compile error, so any straggler says so at build time. In-repo the only positional callers were DockerProcedureContainer's two subcontainer.exec calls, updated here; its own wrapper signatures are unchanged.

One thing that fell out while wiring it up: exec forwards the remainder of its options straight to cp.spawn, so the two fields have to come back off. The existing code does that by delete-ing user/cwd from the caller's own object — with timeoutMs on it, a reused options object would have silently reverted to the 30 s default on its second use, which is exactly the failure this PR is about. It's a destructure now, so the deletes hit a copy.

Verified: make check, make test (87), make check-fmt, make bundle, root prettier --check, and container-runtime's tsc --noEmit against the rebuilt dist/.

Flagging one judgement call for you: this is a breaking signature change on a public SDK API landing in a patch release. It's undocumented today — the third argument was never mentioned in the packaging guide, which is half of why #3636 happened — so I'd expect approximately zero external callers, and any that exist fail loudly at compile. Say the word if you'd rather I keep the positional form working as a deprecated fallback instead.