Skip to content

fix(server): propagate errors through TimeoutMiddleware - #5670

Open
hsusul wants to merge 1 commit into
bentoml:mainfrom
hsusul:fix/timeout-middleware-propagate-errors
Open

fix(server): propagate errors through TimeoutMiddleware#5670
hsusul wants to merge 1 commit into
bentoml:mainfrom
hsusul:fix/timeout-middleware-propagate-errors

Conversation

@hsusul

@hsusul hsusul commented Jul 23, 2026

Copy link
Copy Markdown

What does this PR address?

When an HTTP handler raised before sending any ASGI response, TimeoutMiddleware waited for the full traffic.timeout and then returned no response. Clients received an empty disconnect instead of a normal 500 response.

The timeout waiter previously completed only when:

  • the application sent its first ASGI message, or
  • the timeout expired.

If the application future had already failed, fut.cancel() returned False. The middleware then neither sent a 504 response nor awaited the future, leaving the exception unobserved.

This PR:

  • completes the waiter when the request future finishes before sending a response
  • awaits the request future when cancellation loses the timeout race, allowing the exception to propagate through Starlette's ServerErrorMiddleware

Successful responses and genuine timeout responses are unchanged.

Validation

  • Added regression coverage for an exception raised before response headers are sent
  • Verified slow requests still return 504
  • Verified successful requests still return 200
  • ruff format passed
  • ruff check passed
  • pre-commit passed on changed files
  • pyright completed with 0 errors
  • git diff --check passed
export PYTEST_PLUGINS=bentoml.testing.pytest.plugin
python -m pytest tests/unit/_internal/server/http/test_traffic.py -q

Unhandled exceptions before any ASGI response previously waited for the
full traffic timeout and then returned no response when cancel failed.
Unblock the waiter when the request future completes and await it on the
timeout race so clients get a proper 500 instead of an empty hang.
@hsusul
hsusul requested a review from a team as a code owner July 23, 2026 17:16
@hsusul
hsusul requested review from jianshen92 and removed request for a team July 23, 2026 17:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant