Skip to content

fix(sdk): retry ProcessError exit-143 (gateway stream-drop) instead of failing permanently - #316

Open
mtoslalibu wants to merge 1 commit into
mainfrom
fix/sdk-processerror-143-transient
Open

mtoslalibu wants to merge 1 commit into
mainfrom
fix/sdk-processerror-143-transient

Conversation

@mtoslalibu

Copy link
Copy Markdown
Collaborator

Closes #315

Problem

When the LLM gateway drops a long streaming response mid-turn, the Claude Agent SDK's reader errors and SIGTERMs its claude child, surfacing as ProcessError "(exit code: 143)" / "Fatal error in message reader". The SDK runner's classifier only matched transport error class-names (ConnectionError, TimeoutError, …), so this transient infra fault was re-raised as permanent and aborted the whole campaign iteration — the bounded retry machinery (--max-cli-retries, retry_log.jsonl) never engaged. Observed repeatedly on long Opus DESIGN turns via a litellm→Bedrock proxy; the same campaign/config completes fine when the gateway is healthy, confirming it's transient.

Fix

  • Extract the transient-vs-permanent classification into a pure, unit-tested helper _is_transient_sdk_error(exc) (no behaviour change for the existing 6 transport signals).
  • Classify ProcessError by its structured exit_code: only 143 (the SIGTERM) is transient; every other non-zero exit (bad request, usage, config — exit 1/2/…) stays permanent and fails fast, so deterministic failures don't burn 10 retries.
  • Message-text fallback is exit-code-143-specific for the wrapped-in-ExceptionGroup case (a bare "message reader" banner is intentionally excluded so a wrapped non-143 error can't be mis-retried).
  • Retry remains bounded by --max-cli-retries and every attempt is logged to retry_log.jsonl; the original error is preserved via raise ... from exc.

Tests

New TestSDKTransientClassification (7 cases, offline, no live LLM calls per repo policy):

  • existing transport signals still transient;
  • ProcessError(exit_code=143) transient (fake + real-SDK);
  • ProcessError(exit_code=1/2/None) permanent (fake + real-SDK) — the regression surface;
  • wrapped "exit code 143" text transient; wrapped non-143 banner permanent.

Full suite: 1357 passed, 2 skipped.

Reviewed

Self-reviewed with the PR-review toolkit (code-reviewer + silent-failure-hunter); the key must-fix (don't retry non-143 non-zero exits) and the fallback-breadth edge case were caught and addressed in review rounds before this PR.

…f failing permanently

When the gateway drops a long streaming response, the Agent SDK SIGTERMs its
claude child, surfacing as ProcessError "(exit code: 143)" / "Fatal error in
message reader". The runner's classifier only matched transport error names
(ConnectionError/TimeoutError/...), so this transient infra fault was re-raised
as permanent and aborted the whole iteration with no retry.

Extract the transient-vs-permanent classifier into a pure, unit-tested helper
`_is_transient_sdk_error`. Classify ProcessError by its structured exit_code:
only 143 (the SIGTERM) is transient; every other non-zero exit (bad request,
usage, config) stays permanent and fails fast. A message-text fallback
(exit-code-143-specific) covers the wrapped-in-ExceptionGroup case. Retry stays
bounded by --max-cli-retries and logged to retry_log.jsonl.

Refs #315

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@mtoslalibu

Copy link
Copy Markdown
Collaborator Author

@claude please review this PR. Focus on: (1) the ProcessError exit-143 classification correctness (false-positive/over-broad-catch risk — should only exit_code==143 be retried?), (2) that the refactor preserves the original transient-signal behavior exactly, (3) test coverage of the retry path, and (4) adherence to the repo's no-live-LLM-calls test policy.

@mtoslalibu

Copy link
Copy Markdown
Collaborator Author

@claude please review this PR (re-triggering — runner was wedged). Focus: ProcessError exit-143 classification correctness, refactor preserves original transient behavior, retry-path test coverage, and no-live-LLM test policy.

sriumcp added a commit to sriumcp/agentic-strategy-evolution that referenced this pull request Oct 17, 2026
sriumcp added a commit to sriumcp/agentic-strategy-evolution that referenced this pull request Oct 17, 2026
@sriumcp

sriumcp commented Oct 18, 2026

Copy link
Copy Markdown
Collaborator

This PR's commit (784eb0c) is merged into #302 (merge 347ec43), which lists Closes #316. When #302 merges, this commit lands on main and GitHub marks this PR merged. The plan/build agent calls in #302 use the same exit-143 classifier.

🤖 Generated with Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

SDK ProcessError (exit 143) from a gateway stream-drop is misclassified as permanent, aborting the whole campaign

3 participants