You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When a streamable-HTTP session's event replay fails server-side (e.g. the session's channel is closed but the session entry still exists), the rmcp server answers the client's resume GET with HTTP 200 + an immediately-closed empty SSE stream. rmcp's own Rust client treats that as a successful reconnect followed by a graceful stream close — which resets its retry counter — and reconnects with the same Last-Event-Id after retry(0) (1s by default). The result is a permanent 1 Hz request loop per wedged stream that never backs off, never terminates, and burns CPU + log volume on both ends indefinitely.
Both halves behave reasonably in isolation; together they form a closed loop.
Server side
src/transport/streamable_http_server/tower.rs (~line 975 in 1.8.0):
Err(e) => {// Return 200 with an immediately-closed empty stream.// Returning an HTTP error would cause EventSource to retry// with the same Last-Event-ID in an infinite loop. ...
tracing::warn!("Resume failed ({e}), returning empty stream");
The comment's rationale is browser EventSource semantics — but the empty-200 produces exactly the feared infinite same-Last-Event-Id loop in rmcp's own Rust client, just disguised as a successful reconnect (and it emits one unthrottled WARN per iteration).
None => {// Per SEP-1699, a graceful stream close is reconnectable. ...let interval = this.server_retry_interval.take().or_else(|| this.retry_policy.retry(0));// <-- retry_times hardcoded to 0
...SseAutoReconnectStreamState::WaitingNextRetry{ sleep,retry_times:0}
retry_times is reset to 0 on every graceful close; the counter only increments when the reconnect itself errors. A 200-empty response therefore never counts as a failure: with the default ExponentialBackoff { max_times: None, base_duration: 1s }, the loop runs at retry(0) = 1s forever.
last_event_id is preserved across iterations, so every reconnect re-sends the same Last-Event-Id, guaranteeing the server's resume fails again.
Production impact (how we found this)
Fleet of ~17 long-running rmcp clients (defra-agent runtimes) against an rmcp streamable-HTTP server (rmcp 0.15.0, behavior unchanged in 1.8.0). A handful of long-lived idle connections' sessions became unresumable; each entered the loop. Over ~4.7 days: 14.8M Resume failed WARN lines (2.9 GB of server logs), sustained CPU on both ends, and the server's real signals drowned. Observed WARN rate ≈ 36/sec ≈ number of wedged streams × 1 Hz, matching the analysis exactly.
Suggested fixes (either breaks the loop; both seem right)
Server: when resume fails but the session is still alive, fall back to create_standalone_stream for that session (fresh live stream, no replay) instead of an empty stream — the client gets keep-alives and stops looping. If the session is terminated/unknown, return 404 per the spec's session-management rules so clients re-initialize (same spirit as Streamable HTTP: Session not found returns 401 instead of 404 (MCP spec violation) #689). Note 404 is safe for this client: a failed reconnect does increment retry_times, and exponential backoff self-quiets.
Client: treat a graceful close that delivered zero events since connect as a failed attempt (don't reset retry_times), and honor max_times on the graceful-close path. Optionally clear last_event_id after N consecutive failed resumes and re-establish a fresh stream.
At minimum, rate-limit the server's Resume failed WARN.
Happy to contribute a PR for either side if maintainers agree on the direction.
Summary
When a streamable-HTTP session's event replay fails server-side (e.g. the session's channel is closed but the session entry still exists), the rmcp server answers the client's resume GET with HTTP 200 + an immediately-closed empty SSE stream. rmcp's own Rust client treats that as a successful reconnect followed by a graceful stream close — which resets its retry counter — and reconnects with the same
Last-Event-Idafterretry(0)(1s by default). The result is a permanent 1 Hz request loop per wedged stream that never backs off, never terminates, and burns CPU + log volume on both ends indefinitely.Both halves behave reasonably in isolation; together they form a closed loop.
Server side
src/transport/streamable_http_server/tower.rs(~line 975 in 1.8.0):The comment's rationale is browser
EventSourcesemantics — but the empty-200 produces exactly the feared infinite same-Last-Event-Idloop in rmcp's own Rust client, just disguised as a successful reconnect (and it emits one unthrottled WARN per iteration).Client side
src/transport/common/client_side_sse.rs,SseAutoReconnectStream::poll_next, graceful-close arm:retry_timesis reset to 0 on every graceful close; the counter only increments when the reconnect itself errors. A 200-empty response therefore never counts as a failure: with the defaultExponentialBackoff { max_times: None, base_duration: 1s }, the loop runs atretry(0)= 1s forever.last_event_idis preserved across iterations, so every reconnect re-sends the sameLast-Event-Id, guaranteeing the server's resume fails again.Production impact (how we found this)
Fleet of ~17 long-running rmcp clients (defra-agent runtimes) against an rmcp streamable-HTTP server (rmcp 0.15.0, behavior unchanged in 1.8.0). A handful of long-lived idle connections' sessions became unresumable; each entered the loop. Over ~4.7 days: 14.8M
Resume failedWARN lines (2.9 GB of server logs), sustained CPU on both ends, and the server's real signals drowned. Observed WARN rate ≈ 36/sec ≈ number of wedged streams × 1 Hz, matching the analysis exactly.Suggested fixes (either breaks the loop; both seem right)
create_standalone_streamfor that session (fresh live stream, no replay) instead of an empty stream — the client gets keep-alives and stops looping. If the session is terminated/unknown, return 404 per the spec's session-management rules so clients re-initialize (same spirit as Streamable HTTP: Session not found returns 401 instead of 404 (MCP spec violation) #689). Note 404 is safe for this client: a failed reconnect does incrementretry_times, and exponential backoff self-quiets.retry_times), and honormax_timeson the graceful-close path. Optionally clearlast_event_idafter N consecutive failed resumes and re-establish a fresh stream.Resume failedWARN.Happy to contribute a PR for either side if maintainers agree on the direction.