Repository navigation
fix(core): resolve a child's exit from /proc when Bun never reports it - #40
Merged
Merged
Conversation
|
Thanks for your contribution! This PR doesn't have a linked issue. All PRs must reference an existing issue. Please:
See CONTRIBUTING.md for details. |
|
The following comment was made by an LLM, it may be inaccurate: |
Bun 1.3.14 can drop a child's exit notification and leave the process a zombie. The spawner's exit Deferred then never resolves, so the tool call, its timeout, and the whole turn wedge forever. - poll /proc/<pid>/stat once a second while the exit is unknown and resolve the Deferred from field 52 (the waitpid status) when the child is a zombie - bound the wait after SIGKILL so a release finalizer can never pin a fiber - one terminate() helper replaces the duplicated kill sequence in the scope release and handle.kill Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
PierrotAWB
force-pushed
the
lost-child-exit-watchdog
branch
from
September 3, 2026 21:11
1754d55 to
26ec53e
Compare
There was a problem hiding this comment.
1 issue found and verified against the latest diff
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="packages/core/src/cross-spawn-spawner.ts">
<violation number="1" location="packages/core/src/cross-spawn-spawner.ts:382">
P1: When a scoped command omits `forceKillAfter` and ignores SIGTERM, this branch waits for `signal` forever during scope finalization. Keep finalizer cleanup non-blocking without a bound, or give that path its own timeout.</violation>
</file>
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
| const send = (s: NodeJS.Signals) => Effect.catch(killGroup(command, proc, s), () => killOne(command, proc, s)) | ||
| const attempt = send(opts?.killSignal ?? "SIGTERM").pipe(Effect.andThen(Deferred.await(signal)), Effect.asVoid) | ||
| const bound = opts?.forceKillAfter | ||
| if (Predicate.isUndefined(bound)) return attempt |
There was a problem hiding this comment.
P1: When a scoped command omits forceKillAfter and ignores SIGTERM, this branch waits for signal forever during scope finalization. Keep finalizer cleanup non-blocking without a bound, or give that path its own timeout.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/core/src/cross-spawn-spawner.ts, line 382:
<comment>When a scoped command omits `forceKillAfter` and ignores SIGTERM, this branch waits for `signal` forever during scope finalization. Keep finalizer cleanup non-blocking without a bound, or give that path its own timeout.</comment>
<file context>
@@ -342,6 +370,24 @@ export const make = Effect.gen(function* () {
+ const send = (s: NodeJS.Signals) => Effect.catch(killGroup(command, proc, s), () => killOne(command, proc, s))
+ const attempt = send(opts?.killSignal ?? "SIGTERM").pipe(Effect.andThen(Deferred.await(signal)), Effect.asVoid)
+ const bound = opts?.forceKillAfter
+ if (Predicate.isUndefined(bound)) return attempt
+ // A child whose exit is never reported must not pin this fiber after SIGKILL.
+ return Effect.timeoutOrElse(attempt, {
</file context>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A tool call whose child process has already exited can no longer wedge a turn forever. When Bun never reports the exit, the spawner now reads it from
/procwithin a second, and the kill path can no longer wait unboundedly after SIGKILL.Context
The
existing-site-lifecyclecanary has failed its "agent turn completes" step in ~15% of runs since Aug 26, every one of them a bash tool call that ran a browser command (replo-browser attach … && playwright-cli snapshot) and never came back. Caught live on the canary sandbox (opencode pid 686, Bun v1.3.14, kernel 6.17): the bash tool's shell had exited and sat as aZzombie under opencode for 50+ minutes; opencode held pidfds for its other live children but none for the zombie and never calledwaitpid; nothing else held its pipes; the part stayedrunning,session/statusreported idle, and opencode logged nothing further for that session. Isolated repros (48 direct shell runs, 40 throughPOST /session/:id/shell, 5 real bash-tool turns) never hung, so it needs the canary turn's context, most likely capture-html's in-process Chromium exiting a few seconds earlier. Bun's own 1.3.14 notes describe this class (lost pidfd exit when multiple children exit close together) as fixed forstdio: "ignore"; it evidently is not for piped, detached children.The exit
Deferredincross-spawn-spawner.tsresolves only from Node'scloseevent. When that never fires,handle.exitCodenever resolves, and the bash tool's 120s timeout fires into an acquire/release finalizer that sends SIGTERM, awaits the same Deferred, escalates to SIGKILL after 3s, then awaits it again with no bound. Finalizers are uninterruptible, so the timeout is swallowed silently.Why it surfaced now: the canary user's model moved from sonnet-5 to gpt-5.6-terra via the
replo-agent-modelflag on the same days. Sonnet never ran a browser command on that sandbox; Terra runs one in most turns. All 15 hung runs sit inside Terra windows, zero in Sonnet windows.What changed
getExit(stat)decodes/proc/<pid>/stat: returns thewaitpid-shaped[code, signal]from field 52 when the state isZ, otherwiseundefined. Verified against the live zombie's stat line.spawnstarts a Linux-only, unref'd 1s poller alongside thecloselistener; whichever observes the exit first completes the Deferred and clears the poller. A reaped-and-gone child yieldsundefinedand the poller keeps waiting for Bun's event, which owns that case.terminate()replaces the identical SIGTERM → wait → SIGKILL → wait sequence in the scope release andhandle.kill; the wait after SIGKILL is now bounded byforceKillAfterviaEffect.timeoutOption.Net: one source file, +76/−39 including the dedupe.
What to scrutinize
The poller resolves only on state
Z, never on ENOENT, so it cannot race Bun's own reaping into a wrong status. The status decode assumes the Linuxwaitpidlayout (status & 0x7fsignal,status >> 8code); that is what/procdocuments for field 52. On macOS and Windows the poller is not created. Zombies stay unreaped until opencode restarts, which on sandboxes is every boot.Testing done
packages/corebun test test/effect/cross-spawn-spawner.test.ts test/process/process.test.ts: 49 pass, including the newgetExitdecode test (exit code, signal, running, empty).bun run typecheckinpackages/core: clean.bun linton the changed files: only the four pre-existing warnings remain.existing-site-lifecycle"agent turn completes" failures on Terra days should go to zero; sandbox-side,Tool execute before tool=bashwith no matchingaftershould stop appearing in{job="sandbox", service_name="opencode"}.🤖 Generated with Claude Code
Summary by cubic
Fixes the
existing-site-lifecyclecanary's "agent turn completes" failures by stopping a bash tool call from wedging a turn forever when Bun never reports the child's exit. The spawner now reads the exit from/proc/<pid>/statwithin a second, and the kill path can no longer wait unboundedly after SIGKILL.What changed
Deferredresolves from Node'scloseevent or a Linux-only 1s poller that decodes field 52 of/proc/<pid>/statfor zombie children.terminate()replaces the duplicated SIGTERM → wait → SIGKILL → wait sequence; the wait after SIGKILL is bounded byforceKillAfter.getExitis exported with tests covering exit codes, signals, running processes, and empty stat lines.Testing
bun testpasses 49 tests acrosscross-spawn-spawnerandprocess, including the newgetExitdecode test.Written for commit 26ec53e. Summary will update on new commits.