fix: handle tcp_closed/ssl_closed gracefully instead of crashing - #277
Merged
Merged
Conversation
Contributor
Author
|
@comtihon Any chance you could look at this PR? |
samwar
force-pushed
the
fix/tcp-closed-graceful-shutdown
branch
from
July 22, 2026 19:46
3f5f177 to
321610a
Compare
Owner
|
hi, can you please check why tests are failing? |
Contributor
Author
|
@comtihon the failure looks unrelated to my my changes and due to a gpg key issue when setting up the test containers. Is this fix something you want to do, or should I just include the change in this PR, even though it's a bit unrelated? |
Contributor
Author
|
@comtihon My original PR passed all of the tests, and the push I made only changed comments. I think the test is just flaky. Since you're an admin, you can re-run the failed jobs, and that should make the PR green. |
When MongoDB closes a TCP connection (idle timeout, server restart,
network issues), the mc_worker gen_server crashes with reason
'tcp_closed', generating CRASH REPORT and error logs. This became
more frequent after upgrading from MongoDB 5 to MongoDB 6.
Change handle_info/2 to return {stop, {shutdown, NetR}, State} instead
of {stop, tcp_closed, State} for tcp_closed and ssl_closed messages,
where NetR is the exact closure atom (tcp_closed or ssl_closed).
OTP treats `{shutdown, _}` as an expected termination and skips
crash reports, while preserving the termination reason for
supervisor observability and monitoring processes. Info-level
diagnostic logging was added before termination to aid debugging.
This allows the supervisor to restart the worker cleanly without
noisy crash reports, while preserving error termination behavior
for actual tcp_error/ssl_error conditions.
Closes: tcp_closed crash in gen_server:handle_common_reply/8
samwar
force-pushed
the
fix/tcp-closed-graceful-shutdown
branch
from
July 27, 2026 18:25
321610a to
9eaefad
Compare
Contributor
Author
|
@comtihon I made an empty commit which re-triggered the tests and they all pass now. Can you take a look at this again? |
Owner
|
sorry, was out in the wild for a week. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
When MongoDB closes a TCP connection (idle timeout, server restart, network issues),
mc_workercurrently crashes with reasontcp_closed, producing SASLCRASH REPORToutput and error logs on every closure. This became noticeably more frequent after upgrading from MongoDB 5 to MongoDB 6.This PR changes
handle_info/2to treat closure messages as an expected shutdown, so the supervisor can restart the worker cleanly without log noise, while still preserving the reason for observability.Change
In
src/connection/mc_worker.erl, thehandle_info/2clause matching{tcp_closed, _}/{ssl_closed, _}now returns:{stop, {shutdown, NetR}, State}instead of
{stop, tcp_closed, State}, whereNetRis the exact closure atom (tcp_closedorssl_closed).OTP treats
{shutdown, _}as an expected termination and skips crash reports, while preserving the termination reason for supervisor observability and monitoring processes. Anerror_logger:info_msg/2line was added before termination so operators still have visibility into each closure.Error handling for
tcp_error/ssl_erroris unchanged — those still stop with the underlying error reason.Testing
test/tcp_closure_bug_SUITE.erlcovering:{shutdown, tcp_closed}/{shutdown, ssl_closed}on socket closuretcp_error/ssl_errortermination semanticstcp/sslpayload messages)rebar3 compilepasses.Notes