Skip to content

Make the fixture-server readiness check detect refused connections - #160

Merged
patvice merged 1 commit into
patvice:mainfrom
jordan-brough:fix-fixture-server-readiness
Oct 8, 2026
Merged

patvice merged 1 commit into
patvice:mainfrom
jordan-brough:fix-fixture-server-readiness

Conversation

@jordan-brough

Copy link
Copy Markdown

What you'll see

Run one of the SSE spec files on its own on a Mac, for example:

bundle exec rspec spec/ruby_llm/mcp/native/transports/sse_spec.rb

Every example fails with SSE transport closed. The streamable spec file fails one example with Connection refused. The full suite passes, because earlier spec files take long enough for the fixture servers to finish booting. So this is easy to miss.

Why it happens

TestServerManager starts the fixture servers and then asks "is the port open yet?" before letting the tests run. That check uses Socket.tcp(host, port, connect_timeout: 1).

On macOS with Ruby up to 4.0.6, that call answers "yes" even when nothing is listening on the port. This is a Ruby bug: https://bugs.ruby-lang.org/issues/22223. The retried connect(2) reports EISCONN and Ruby did not check SO_ERROR, so a refused connection looked like a successful one.

As a result, the readiness check returns right away and the tests start before the servers are up. The first spec to talk to a server gets a refused connection.

The fix

Use TCPSocket.new(host, port, connect_timeout: 1) for the probe instead. Its connect path checks SO_ERROR and raises ECONNREFUSED correctly, so start_server blocks until every fixture port is really accepting connections.

One file, a few lines: spec/support/test_server_manager.rb (wait_for_port and port_open?), plus a short comment explaining why.

What changes in behavior

Server startup now waits until the ports are really listening, which takes about half a second. The isolated runs pass. The full suite is unchanged.

Measured on macOS with Ruby 4.0.6:

Before After
start_server returns after 0.006 to 0.008 s, with none of ports 3005/3006/3007 listening (they came up about 0.37 s later) 0.63 to 0.84 s, with all three ports listening
native/transports/sse_spec.rb alone 48 examples, 48 failures 48 examples, 0 failures
adapters/mcp_sdk_sse_spec.rb alone 6 examples, 6 failures 6 examples, 0 failures
adapters/mcp_sdk_streamable_spec.rb alone 6 examples, 1 failure 6 examples, 0 failures

Ruby 4.0.7 fixes the underlying Socket.tcp bug, but the probe should be correct on older Rubies too.

TestServerManager#wait_for_port (and port_open?) probed the fixture
servers with Socket.tcp(host, port, connect_timeout: 1). On Darwin with
Ruby <= 4.0.6, Socket.tcp returns a socket for a refused connection
because the retry connect(2) answers EISCONN and the library does not
check SO_ERROR (ruby-core Bug #22223, fixed in Ruby 4.0.7). The
readiness check therefore returned immediately after spawn, and the
first spec to touch a server raced its boot: "Connection refused" on
the streamable transport and "SSE transport closed" for every example
in the SSE files, depending on which file ran first.

Use TCPSocket.new(host, port, connect_timeout: 1), whose C connect path
checks SO_ERROR and raises ECONNREFUSED reliably, so start_server blocks
until every fixture port is actually accepting connections.
@jordan-brough
jordan-brough marked this pull request as ready for review September 22, 2026 13:42
@jordan-brough
jordan-brough marked this pull request as draft September 22, 2026 13:42
@jordan-brough
jordan-brough marked this pull request as ready for review September 22, 2026 13:42
@patvice

patvice commented Oct 8, 2026

Copy link
Copy Markdown
Owner

Sweet! Thanks for this @jordan-brough. Merging this now

@patvice
patvice merged commit c4042de into patvice:main Oct 8, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants