Skip to content

feat: subagent control — per-spawn budgets, idle watchdog, per-call tool timeouts - #131

Open
Emasoft wants to merge 3 commits into
mlhher:mainfrom
Emasoft:feat/subagent-control
Open

Emasoft wants to merge 3 commits into
mlhher:mainfrom
Emasoft:feat/subagent-control

Conversation

@Emasoft

@Emasoft Emasoft commented Sep 21, 2026

Copy link
Copy Markdown

Description of Changes

Four features giving the user and the orchestrator explicit control over subagent lifetime and tool call duration:

Subagent time budgets (default 24h)

  • Every subagent run is bounded by a wall-clock budget: --subagent-timeout (default 24h; 0 = unlimited), also settable persistently as subagent_timeout in config.json. Precedence is flag > config > default; invalid config values warn and fall back to the 24h default.

Per-spawn budget on spawn_subagent

  • The orchestrator can budget a single run via the new optional timeout argument of spawn_subagent (e.g. "45m", "2h"; "0" = unlimited; omitted = the global value). Invalid values return an error result instead of silently changing behavior.

Idle watchdog

  • Per-subagent activity tracking (stream progress, in-flight tools, nested spawns) drives a watchdog: --subagent-idle-timeout (default 15m; 0 = off) fires a SubagentIdleEvent including a transcript probe of the subagent's last activity, surfaced in the TUI. --subagent-idle-kill-after (default 0 = notify only) escalates true idleness to a two-stage kill: the in-flight tool is cancelled first, then the agent run.

Per-call tool timeouts + shell hardening

  • The bash tool accepts a per-call timeout argument (absent = the 10m global default; "0" = unlimited; invalid = error result). Shells run in their own process group (Unix Setpgid + group SIGKILL; Windows taskkill /T /F) with WaitDelay=5s, so grandchild processes that inherit the output pipes can no longer hang Wait forever, and cancellation always kills the whole tree. A session kill hook makes in-flight tool calls cancellable mid-wait, which is what lets the idle watchdog (and the user) stop a stuck tool.

Independence

This PR is independent of #127/#129/#130 — based on main; textually overlaps with #130 (shell hardening, runner budget): merge one, rebase the other.

Behavior changes

  • Subagents now carry a default 24h wall-clock budget (0 disables); runs exceeding it are cancelled.
  • Idle subagents notify the orchestrator/user after 15m of true idleness (stream progress, in-flight tools, and nested spawns all count as activity).
  • Idle-kill is opt-in (--subagent-idle-kill-after); by default idle subagents are only notified about, never killed.
  • bash invocations default to a 10m per-call bound; cancellation kills the whole process group.

Test plan

  • go test ./... -race -count=1 full suite
  • go build ./...; GOOS=windows go build ./internal/tool/; go vet ./...; gofmt -l clean on all changed files
  • New tests: internal/tool/shell_timeout_test.go (per-call timeout, "0" unlimited, invalid timeout, global timeout expiry and group kill), internal/tool/subagent_test.go (per-spawn timeout parsing), internal/config/config_test.go (subagent_timeout resolution precedence, invalid values, JSON round-trip), cmd/late/main_test.go (explicit --subagent-timeout flag reporting), internal/orchestrator/base_idle_test.go (idle watchdog transitions, idle event, escalation to kill), internal/executor/inflight_test.go (in-flight tool kill hook)

Known limitations

  • The internal/plugin package has known environmental failures on main (3 HOME/XDG-dependent plugin tests, plus a flaky TestRunHook_ProcessGroupKillsChildrenOnCancel) — reproduced on this branch unchanged and unrelated to these features.
  • Approval-waiting tool calls count as active (not idle) — a subagent waiting on user approval will not trigger the idle watchdog.
  • The global bash timeout (10m) is currently not user-configurable; only the per-call argument overrides it.

Contributor License Agreement (CLA)

To accept your code, we legally need you to agree to our CLA so we can maintain the project's Business Source License (BSL) and future open-source transitions.

  • By checking this box, I confirm that I have read and agree to the terms of the CLA.md in this repository. (To check the box, put an x between the brackets like this: [x])

…meouts

Long-running shell commands could hang forever: exec.CommandContext only
signals the direct shell child, so a grandchild that inherited the output
pipes (e.g. a spawned daemon or pipe) survived the shell and blocked Wait
indefinitely. There was also no way to bound a single invocation.

- Unix: run the shell in its own process group (Setpgid) and SIGKILL the
  whole group on cancellation; set WaitDelay=5s so Wait always returns.
- Windows: cancel via taskkill /T /F to walk the PowerShell process tree,
  with the same WaitDelay cap.
- bash tool: add a per-call "timeout" argument. Absent = the 10m global
  default, "0" = unlimited, invalid values return an error result.
- Note: gofmt -w applied to internal/tool/implementations.go to remove a
  pre-existing trailing-whitespace line inside the SYSTEM DIRECTIVE block
  (whitespace-only, flagged during Step 1).

Tests: internal/tool/shell_timeout_test.go covers per-call timeout,
"0" = unlimited, invalid timeout, global timeout expiry, and kill.
Subagents could run forever and stuck tools could block the session with
no way to bound or stop them.

- Subagent time budget: default 24h (--subagent-timeout), also settable
  via config.json "subagent_timeout"; precedence flag > config >
  default, invalid config values warn and fall back to 24h. "0" =
  unlimited.
- Per-spawn budget: spawn_subagent gains an optional "timeout"
  argument overriding the global value for that run ("0" = unlimited).
- Idle watchdog: activity tracking (stream progress, in-flight tools,
  nested spawns) per subagent; --subagent-idle-timeout (15m, 0 = off)
  emits SubagentIdleEvent with a transcript probe of the last activity;
  --subagent-idle-kill-after (0 = notify only) escalates to a two-stage
  kill: cancel the in-flight tool first, then cancel the agent run.
- Killable in-flight tools: a session kill hook lets a blocked shell call
  be cancelled mid-wait instead of hanging the cancel path.
- TUI surfaces idle notifications for the affected subagent.

Tests: config resolution and round-trip, spawn timeout parsing, idle
watchdog transitions and escalation, executor in-flight kill hook.
Add a compact "Subagent control" section to the quickstart covering
--subagent-timeout (24h default, 0 = unlimited, config.json
subagent_timeout), --subagent-idle-timeout (15m, 0 = off),
--subagent-idle-kill-after (0 = notify only), the per-spawn timeout
argument of spawn_subagent, and the per-call timeout argument of bash.
@mlhher

mlhher commented Sep 21, 2026

Copy link
Copy Markdown
Owner

I like the thought behind it, seems valuable.

Note that things that directly affect the LLM performance need to be evaluated with care as thus I will likely proceed with #130 first. Also please make sure to note the latest comment in #127, or I can push a commit removing that part for now and then merge. As noted there we can always come back to it.

Emasoft added a commit to Emasoft/late-cli that referenced this pull request Sep 21, 2026
Union of PR mlhher#131 (feat/subagent-control @ 2d96f26, main-based) into the
local/full stack (mlhher#127 resilient streaming + mlhher#129 OTP gate + mlhher#130 bash
timeouts). Both feature sets are kept and both test suites pass.

Per-file union decisions:

- cmd/late/main.go: adopted incoming's budget resolution stack (24h
  DefaultSubagentTimeout, config.json subagent_timeout precedence via
  flag.Visit + ResolveSubagentTimeout, per-spawn timeout override via
  effectiveSubagentBudget) and the nested-spawn busy tracking
  (BeginNestedSpawn/EndNestedSpawn + parent heartbeat around
  child.Execute, SetContext(runCtx), child SetIdlePolicy). Kept
  local/full's transcript writer + cause classification: the
  classification now uses the EFFECTIVE budget and gained an idle-kill
  cause (child IdleKillReason() non-empty) that renders as
  "idle: killed by the harness idle watchdog (<probe summary>)" ahead of
  the user-cancel case. Flags: union --subagent-timeout (default 24h
  from appconfig), --subagent-idle-timeout, --subagent-idle-kill-after
  plus all local/full flags (--bash-timeout, --continue-project, OTP,
  ...); SetIdlePolicy applied to root and children.

- internal/orchestrator/base.go: union of event hardening (non-blocking
  progress sends via trySendProgress + droppedEvents counter, blocking
  terminal/ChildAdded sends) with the activity-aware idle watchdog
  (lastActivity/inFlightTools/nestedSpawns/oldestToolStartAt,
  activityMiddleware outermost, two-stage tool-kill then agent-kill,
  injectable tick, SetIdlePolicy/MarkActivity/IdleKillReason).
  MarkActivity fires in both Execute and run() stream callbacks; the
  idle event is emitted non-blocking (select/default).

- internal/executor/executor.go: harness-note delivery kept; per-call
  toolCtx + SetInFlightToolCancel/Clear and the "tool cancelled by the
  harness idle watchdog" result path added. inFlightKill is captured
  BEFORE the per-call toolCancel() and gates the harness note, so a
  watchdog tool-kill (failure-shaped "Command failed with exit code -1"
  from the shell SIGKILL) never asks the coder to report back; a real
  shell failure or a per-call timeout still gets the note.

- internal/tool/implementations.go: single ShellTool.Execute with
  per-call timeout resolution (resolveShellTimeout: absent=global,
  0/negative=unlimited, invalid=error result), global 10m default wired
  to --bash-timeout via SetShellTimeout, timeout message uses the
  effective bound. Kept local/full's harness-note-free error path (no
  duplicated coder sandwich) and the otp_code parameter; Parameters now
  declare both otp_code and timeout.

- internal/tool/shell_command_unix.go, shell_command_windows.go:
  identical mechanism on both sides (process-group kill + WaitDelay);
  incoming's comments adopted.

- internal/config/config.go: PermissionMode constants + SubagentTimeout
  entry coexist; ResolvePermissionMode and ResolveSubagentTimeout both
  kept.

- internal/common/interfaces.go: RetryEvent + RecoveryEvent (local/full)
  and SubagentIdleEvent + ActivityMarker (incoming) kept together.

- internal/tui/update.go: SubagentIdleEvent status-line case added to
  the hardened update loop.

- internal/tool/shell_timeout_test.go: local/full's de-flaked timeout/
  grandchild/pgrep tests plus incoming's per-call timeout and
  resolveShellTimeout tests.

- docs/quickstart.md: incoming Subagent Control section; Common Flags
  row updated to the 24h default.

Integration test added:
TestExecuteToolCalls_WatchdogToolKillDoesNotAttachNote guards the union
of the in-flight hook with the harness note.

Gates: go build ./... ok, go vet ./... ok, go test ./... -race -count=1
all packages ok, gofmt clean on touched files, ./install-dev.sh check ok.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants