Skip to content

fix: handle tcp_closed/ssl_closed gracefully instead of crashing - #277

Merged
comtihon merged 1 commit into
comtihon:masterfrom
raxdcx:fix/tcp-closed-graceful-shutdown
Aug 5, 2026
Merged

fix: handle tcp_closed/ssl_closed gracefully instead of crashing#277
comtihon merged 1 commit into
comtihon:masterfrom
raxdcx:fix/tcp-closed-graceful-shutdown

Conversation

@samwar

@samwar samwar commented Jul 21, 2026

Copy link
Copy Markdown
Contributor

Summary

When MongoDB closes a TCP connection (idle timeout, server restart, network issues), mc_worker currently crashes with reason tcp_closed, producing SASL CRASH REPORT output and error logs on every closure. This became noticeably more frequent after upgrading from MongoDB 5 to MongoDB 6.

This PR changes handle_info/2 to treat closure messages as an expected shutdown, so the supervisor can restart the worker cleanly without log noise, while still preserving the reason for observability.

Change

In src/connection/mc_worker.erl, the handle_info/2 clause matching {tcp_closed, _} / {ssl_closed, _} now returns:

{stop, {shutdown, NetR}, State}

instead of {stop, tcp_closed, State}, where NetR is the exact closure atom (tcp_closed or ssl_closed).

OTP treats {shutdown, _} as an expected termination and skips crash reports, while preserving the termination reason for supervisor observability and monitoring processes. An error_logger:info_msg/2 line was added before termination so operators still have visibility into each closure.

Error handling for tcp_error / ssl_error is unchanged — those still stop with the underlying error reason.

Testing

  • Adds test/tcp_closure_bug_SUITE.erl covering:
    • {shutdown, tcp_closed} / {shutdown, ssl_closed} on socket closure
    • Preservation of tcp_error / ssl_error termination semantics
    • Preservation of data-message processing (tcp / ssl payload messages)
    • Explicit-stop behavior unchanged
  • rebar3 compile passes.
  • New suite: 7 cases, all passing.

Notes

  • No API or behavior change for callers — connections still terminate and are restarted by the pool supervisor; the only difference is the absence of spurious crash reports.

@samwar

samwar commented Jul 22, 2026

Copy link
Copy Markdown
Contributor Author

@comtihon Any chance you could look at this PR?

@samwar
samwar force-pushed the fix/tcp-closed-graceful-shutdown branch from 3f5f177 to 321610a Compare July 22, 2026 19:46
@comtihon

Copy link
Copy Markdown
Owner

hi, can you please check why tests are failing?

@samwar

samwar commented Jul 23, 2026

Copy link
Copy Markdown
Contributor Author

@comtihon the failure looks unrelated to my my changes and due to a gpg key issue when setting up the test containers. Is this fix something you want to do, or should I just include the change in this PR, even though it's a bit unrelated?

@samwar

samwar commented Jul 23, 2026

Copy link
Copy Markdown
Contributor Author

@comtihon My original PR passed all of the tests, and the push I made only changed comments. I think the test is just flaky. Since you're an admin, you can re-run the failed jobs, and that should make the PR green.

When MongoDB closes a TCP connection (idle timeout, server restart,
network issues), the mc_worker gen_server crashes with reason
'tcp_closed', generating CRASH REPORT and error logs. This became
more frequent after upgrading from MongoDB 5 to MongoDB 6.

Change handle_info/2 to return {stop, {shutdown, NetR}, State} instead
of {stop, tcp_closed, State} for tcp_closed and ssl_closed messages,
where NetR is the exact closure atom (tcp_closed or ssl_closed).
OTP treats `{shutdown, _}` as an expected termination and skips
crash reports, while preserving the termination reason for
supervisor observability and monitoring processes. Info-level
diagnostic logging was added before termination to aid debugging.

This allows the supervisor to restart the worker cleanly without
noisy crash reports, while preserving error termination behavior
for actual tcp_error/ssl_error conditions.

Closes: tcp_closed crash in gen_server:handle_common_reply/8
@samwar
samwar force-pushed the fix/tcp-closed-graceful-shutdown branch from 321610a to 9eaefad Compare July 27, 2026 18:25
@samwar

samwar commented Jul 28, 2026

Copy link
Copy Markdown
Contributor Author

@comtihon I made an empty commit which re-triggered the tests and they all pass now. Can you take a look at this again?

@comtihon
comtihon merged commit 86dc978 into comtihon:master Aug 5, 2026
14 checks passed
@comtihon

comtihon commented Aug 5, 2026

Copy link
Copy Markdown
Owner

sorry, was out in the wild for a week.
thank you for your contribution!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants