Skip to content

fix: make uvicorn's keep-alive timeout configurable, default 75s - #148

Open
dvystrcil wants to merge 1 commit into
open-webui:mainfrom
dvystrcil:fix/uvicorn-keep-alive-timeout
Open

fix: make uvicorn's keep-alive timeout configurable, default 75s#148
dvystrcil wants to merge 1 commit into
open-webui:mainfrom
dvystrcil:fix/uvicorn-keep-alive-timeout

Conversation

@dvystrcil

Copy link
Copy Markdown

Problem

I run open-terminal behind Open WebUI as a tool backend. Occasionally a tool call would fail with ConnectionResetError/Connection reset by peer even though the underlying command had actually completed successfully.

Root cause: uvicorn's default timeout_keep_alive is 5 seconds, which can be shorter than the client's own HTTP connection-pool reuse window. If the client (Open WebUI, or any pooled HTTP client) holds a connection open longer than 5s of idle time and then reuses it, the server has already closed it server-side -- the write lands on a dead socket and the client sees a reset, even though nothing about the request itself was wrong.

Fix

Adds OPEN_TERMINAL_UVICORN_TIMEOUT_KEEP_ALIVE (or uvicorn_timeout_keep_alive in config.toml), following the exact same pattern as the existing OPEN_TERMINAL_UVICORN_LOOP option. Defaults to 75s, matching gunicorn/nginx's common default and comfortably exceeding most client-side pool idle timeouts.

Testing

No test harness exists in this repo currently, so verified manually: built a venv from this branch (pip install -e .), started the server (open-terminal run), confirmed it boots cleanly and serves /health correctly with the new parameter wired through uvicorn.run(). Confirmed timeout_keep_alive is a real, documented uvicorn.run() parameter via inspect.signature.

Happy to adjust the default or naming convention if you'd prefer something different.

uvicorn's default timeout_keep_alive (5s) can be shorter than a
client's HTTP connection-pool reuse window, so a pooled-but-idle
connection gets written to right after the server has already closed
it -- surfacing as ConnectionResetError on the client even though the
underlying command completed successfully.

Adds OPEN_TERMINAL_UVICORN_TIMEOUT_KEEP_ALIVE (or
uvicorn_timeout_keep_alive in config.toml), following the same pattern
as the existing OPEN_TERMINAL_UVICORN_LOOP option. Defaults to 75s,
matching gunicorn/nginx's common default.
dvystrcil added a commit to dvystrcil/open-terminal-app-fork that referenced this pull request Aug 1, 2026
dvystrcil added a commit to dvystrcil/open-terminal-docker that referenced this pull request Aug 1, 2026
…opy (#67)

This repo's Dockerfile has always pulled a pre-built upstream image
(FROM .../ghcr-proxy/open-webui/open-terminal:latest) and only added
wrapper tooling on top. It never built or installed this repo's own
vendored open_terminal/ package -- every fix this CHANGELOG has
described as applied to open_terminal/main.py was real, tested,
merged code that the running container never actually ran. See
homelab#822 for the full incident writeup.

Root cause found while investigating why homelab#720's security fix
(process-log retention) wasn't live despite being merged days ago.
Auditing the vendored package's full history surfaced three more real,
undeployed fixes beyond that one, all now submitted upstream:

  - open-webui/open-terminal#148 -- configurable uvicorn keep-alive
    timeout (intermittent ConnectionResetError)
  - open-webui/open-terminal#149 -- process-log retention security fix
  - open-webui/open-terminal#150 -- two-tier process-result expiry
    (a slow caller could lose a finished command's result forever)
  - open-webui/open-terminal#151 -- insert_after/append_to_section/
    append endpoints + a defensive replace_file_content check

Plus one homelab-specific fix NOT appropriate for upstream (ties into
our own GH App token-file convention, not something upstream has any
hook for): refresh_github_token_env(), re-reads the current token
from disk before every subprocess spawn, closing a gap BASH_ENV-based
shell-profile sourcing doesn't cover (plain-shell and PTY spawn paths
never source /etc/profile.d).

## What changed

- Dockerfile: stage 1 now builds open_terminal from
  dvystrcil/open-terminal-app-fork (a real fork carrying all 5 fixes
  above) via git clone + pip install ., mirroring upstream's own
  Dockerfile exactly, instead of pulling the pre-built upstream image.
  TEMPORARY -- revert to a plain upstream FROM once all four PRs merge
  and a release picks them up.
- docker.yml: resolves the fork's current commit SHA via `git
  ls-remote` and passes it as a build-arg on every run. Without this,
  Docker's build cache (keyed on RUN command text, not on what `git
  clone --branch main` actually fetches) would silently keep shipping
  whatever fork commit was cloned the FIRST time this layer built,
  even after new fixes land on the fork -- caught this empirically:
  an initial local build without the explicit build-arg produced a
  stale image missing later fixes despite a fresh fork push.
- Removed the vendored open_terminal/ package and its tests -- dead
  weight now that the real fixes live in a fork with a real upstream
  relationship. Kept tests/test_actor_env.py (tests
  helpers/bible_bridge.py, which *is* deployed) and tests/__init__.py.
  Removed pyproject.toml, dev.sh, .python-version (all specific to
  developing the now-removed vendored package).
- README: documents the new build shape and the incident.

## Testing

Built the actual image locally (multi-stage, full apt/pip install,
~2 min), ran it, and verified all 5 fixes are present in the running
container (direct imports of the sweep function, keep-alive/expiry
env values, refresh_github_token_env, and the three new endpoint
handlers) plus the health endpoint and all wrapper tools (kubectl,
gh, yq, argocd, act) and the entrypoint's token-refresh profile.
Confirmed the FORK_SHA cache-bust actually works: an explicit
--build-arg forces a genuine re-clone (visible in build output, not
a CACHED layer) rather than silently reusing a stale one.

tests/test_actor_env.py still passes standalone (7/7) against the
real helpers/bible_bridge.py, unaffected by the removed package.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant