You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
apr serve: drain on SIGTERM/SIGINT — keep listening, 503 + Retry-After for new requests, finish in-flight; today SIGTERM cuts requests to EOF #4449
Found by paiml/infra#1150 (P0). The resident apr serve review lane on gx10 flapped about 20 times in 4h, and every stop cut in-flight requests to EOF.
Defect.apr serve run shuts down gracefully only on ctrl_c. shutdown_signal() in crates/apr-cli/src/commands/serve/server.rs awaits tokio::signal::ctrl_c() alone. As a result:
SIGTERM, which systemd sends by default on systemctl stop, kills the process mid-request, and clients read EOF.
The WGPU path (handlers.rs ~751) calls axum::serve with no graceful shutdown at all.
Even on SIGINT, axum's with_graceful_shutdown closes the listener, so a request that arrives during the drain is refused. It never gets an answer it can act on.
infra works around the first point with KillSignal=SIGINT (infra#1150).
Asked (cop ruling on infra#1150). On SIGTERM or SIGINT:
Enter drain.
Keep the listener open.
Answer every NEW request 503 with Retry-After: <s>. /health answers 503 too.
Let in-flight requests (including streams) finish.
Exit once in-flight reaches 0, or at a --drain-timeout.
Apply the same behaviour on every axum::serve site: CUDA, WGPU and CPU.
Falsifier.
A request in flight at SIGTERM completes 200.
A request sent after SIGTERM gets 503 with Retry-After, never a reset or EOF.
merged into #5312 (master ticket for epic #3994; triage rule 7b, 2026-10-14). It is a checklist row there; reopen this issue when work on it starts or its PR opens.
Moved: this sub-ticket is now a row in #5329 (apr serve request path: BPE tokenization, graceful drain, no 500 on oversize prompts), split out of #5312 by theme.
Activity
apr codeon Qwen3.5 Q4_K_M (4B, 9B) on CUDA: does it complete a scripted edit-and-verify task on lambda and gx10? UNMEASURED, and its driver runs throughapr serve --gpu(#3571) #3719noahgift commentedon Oct 14, 2026
merged into #5312 (master ticket for epic #3994; triage rule 7b, 2026-10-14). It is a checklist row there; reopen this issue when work on it starts or its PR opens.
apr codeon Qwen3.5 Q4_K_M (4B, 9B) on CUDA: does it complete a scripted edit-and-verify task on lambda and gx10? UNMEASURED, and its driver runs throughapr serve --gpu(#3571) #3719noahgift commentedon Oct 14, 2026
Moved: this sub-ticket is now a row in #5329 (apr serve request path: BPE tokenization, graceful drain, no 500 on oversize prompts), split out of #5312 by theme.