Load-balance envd HTTP/2 pools by in-flight streams; keep HTTP version a transport-only argument

[devin-ai-integration]

Summary

Fixes EN-2742 (from the 192-concurrent-trial Harbor/mini-swe-agent report: HTTP/2 stream starvation at MAX_CONCURRENT_STREAMS=100). SDK-381 (a public HTTP/1.1 option) was considered and dropped per review: HTTP version stays a transport-layer detail, not a client/sandbox setting.

Python: load-aware envd pools

The sandbox host allows 100 concurrent streams per HTTP/2 connection and a running command holds its stream for its whole lifetime. The SDK spread sandboxes over 4 fixed pools by crc32(sandbox_id) % 4, so a hot sandbox (or an unlucky hash) still queued at 100, and a 1-sandbox process opened 4 connections anyway. envd_pool_shard(config) is replaced by EnvdPoolBalancer in e2b.api:

class EnvdPoolBalancer:
    def acquire(self) -> int:
        # pool with the fewest requests in flight; opens a new pool only when
        # every open pool already carries `streams_per_pool` requests, up to
        # `max_pools` (past that: least-loaded regardless). Ties -> lowest index.
    def release(self, index: int) -> None
    active_streams: Tuple[int, ...]   # in flight per pool

envd_pool_shards  = E2B_ENVD_POOL_SHARDS  (default 4 -> 16, the pool bound)
envd_pool_streams = E2B_ENVD_POOL_STREAMS (default 90, below the peer's 100)

client_sync/client_async gain EnvdPoolTransport, a pyqwest transport over the balancer whose pools are the existing cached get_pyqwest_transport(..., shard) transports (connect-retry included). It re-wraps the response so a request is counted in flight until its body is fully read, fails, or is closed — not just until the head arrives — via a _TrackedContent iterator that releases exactly once (pyqwest responses can't be subclassed, so it's composition over a fresh Response). Envd RPC (create_rpc_client) and envd HTTP (get_envd_transport, get_envd_api) draw on the same get_envd_pyqwest_transport(proxy, read_timeout, http2) instance, so one balancer sees all streams of a process; control-plane traffic stays on get_transport / shard 0. HTTP/1.1 gets a single pool (reqwest opens a connection per concurrent request there).

Live check against real sandboxes: 120 background sleep 30 commands landed as active_streams == (90, 30), a control echo went through immediately, and counts dropped to 0 via wait(), disconnect(), and handle GC.

Python + JS: HTTP version is a transport-only argument

No public option and no env var. The internal http2: bool = True argument that get_transport already had now also exists on get_envd_transport / get_envd_pyqwest_transport (sync + async), and on the JS fetch factories, so a transport can be built for HTTP/1.1 without touching ConnectionConfig / Sandbox:

get_envd_transport(config, http2=False)          # separate cached transport, single pool

createApiFetch(proxy, /* http2 */ false)
createEnvdFetch(proxy, false); createEnvdRpcFetch(proxy, false)   // cached per `${h2|h1}:${proxy}`
buildDispatchedFetch({ ..., http2: false })       // -> undici Agent/ProxyAgent { allowH2: false }

No JS balancer: undici already opens connections on demand (E2B_ENVD_RPC_CONNECTIONS, default 200) — verified locally admitting 192 held HTTP/2 streams + a control request.

Not in this PR

kill_reason exposure on SandboxNotFoundException/get_info() (needs the belt OpenAPI schema first), and the source of the max_age GOAWAY (not emitted by envd/runtime/Terraform; an intermediary).

Tests

test_envd_stream_capacity.py (192 streams over a real HTTP/2 server with MaxConcurrentStreams=100 → (90, 90, 12), releases on close, no starvation for many- and one-sandbox cases), test_api_client_transport.py (balancer growth/caps/thread churn, body-lifetime accounting incl. a refused connect via a stubbed pool — the earlier closed-port version timed out on Windows — envd vs control-plane cache identity, HTTP-version forwarding), test_envd_client_transport.py (RPC + HTTP share the envd transport), test_env_var_parsing.py, JS api/http2 and envd/http2 tests. pnpm run format/lint/typecheck clean; Python suite 1213 passed (2 httpbin-template failures are pre-existing/account-specific); JS unit suite 163 passed; live sync+async commands/files/pty integration tests pass on HTTP/2 and on HTTP/1.1 transports.

Link to Devin session: https://app.devin.ai/sessions/57b07cfa1dc9453eabf11f0617365534
Open in Devin Desktop: https://app.devin.ai/desktop/session/57b07cfa1dc9453eabf11f0617365534?variant=devin
Requested by: @mishushakov

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment, CI, and merge conflict monitoring

[changeset-bot]

🦋 Changeset detected

Latest commit: 9ae4d13

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package

Name	Type
@e2b/python-sdk	Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[linear-code]

EN-2742

SDK-381

[github-actions]

Package Artifacts

Built from dda57e2. Download artifacts from this workflow run.

JS SDK (e2b@2.51.1-devin-1789734207-envd-pool-growth-http1.0):

npm install ./e2b-2.51.1-devin-1789734207-envd-pool-growth-http1.0.tgz

CLI (@e2b/cli@2.20.1-devin-1789734207-envd-pool-growth-http1.0):

npm install ./e2b-cli-2.20.1-devin-1789734207-envd-pool-growth-http1.0.tgz

Code Interpreter JS SDK (@e2b/code-interpreter@2.8.1-devin-1789734207-envd-pool-growth-http1.0):

npm install ./e2b-code-interpreter-2.8.1-devin-1789734207-envd-pool-growth-http1.0.tgz

Desktop JS SDK (@e2b/desktop@2.4.1-devin-1789734207-envd-pool-growth-http1.0):

npm install ./e2b-desktop-2.4.1-devin-1789734207-envd-pool-growth-http1.0.tgz

Python SDK (e2b==2.51.0+devin.1789734207.envd.pool.growth.http1):

pip install ./e2b-2.51.0+devin.1789734207.envd.pool.growth.http1-py3-none-any.whl

Code Interpreter Python SDK (e2b-code-interpreter==2.10.0+devin.1789734207.envd.pool.growth.http1):

pip install ./e2b_code_interpreter-2.10.0+devin.1789734207.envd.pool.growth.http1-py3-none-any.whl

Desktop Python SDK (e2b-desktop==2.6.0+devin.1789734207.envd.pool.growth.http1):

pip install ./e2b_desktop-2.6.0+devin.1789734207.envd.pool.growth.http1-py3-none-any.whl

[devin-ai-integration]

TASTE.md review (sdk-harness) of the changed surface only.

Checked: parity (T-1, T-2, T-10), API shape (T-3/T-3a, T-6, T-12, T-14, T-22/T-23), configuration (T-47, T-49–T-51), docs (T-69–T-71).

1 violation, flagged inline on both SDKs (same issue):

• T-14 — the new public option sandboxHttp2 / sandbox_http2 is a boolean selecting a transport behavior (HTTP/2 vs HTTP/1.1). TASTE prefers a string-literal union for behavior-selecting options so a third variant (e.g. 'auto', HTTP/3) is not a breaking change; the negated spelling (false means "use HTTP/1.1") also reads poorly at the call site. Same applies to the E2B_SANDBOX_HTTP2 env var, which would become E2B_SANDBOX_HTTP_VERSION (T-49).

Compliant elsewhere: option name mirrors across SDKs (T-10) and extends ConnectionOpts / ApiParams; precedence explicit → E2B_ env var → default via ConnectionConfig (T-49), and get_api_params() forwards it so static/instance reconnects keep it (T-6); flag-shaped env parsing with a sentinel is fine per T-50; JSDoc has @default, Python docstring documents the default (T-69/T-70). The Python-only balancer is transport plumbing with no public surface, so no parity concern (T-1).

Not tied to a diff line worth an inline comment: the new internal helpers createEnvdFetch(proxy?, http2 = true) and get_envd_pyqwest_transport(proxy, read_timeout=None, http2=True) take chains of optional positionals (T-3 / T-3a). They follow the pre-existing shape of get_pyqwest_transport, and are not exported from the entry points, so this is a nit rather than a public-surface violation.

[mishushakov]

check the above

[mishushakov]

I'd probably also remove sandbox_http2 setting and rely on global http2 setting instead (that we should already have)

[devin-ai-integration]

I'd probably also remove sandbox_http2 setting and rely on global http2 setting instead

Removed sandbox_http2/sandboxHttp2 in 821efc7. There was no public global setting yet (http2 was only an internal arg of get_transport/buildDispatchedFetch), so this adds one: ConnectionConfig(http_version="http1"|"http2") / { httpVersion }, env E2B_HTTP_VERSION, default http2, applied to API and envd traffic alike. The Windows-only test timeout is fixed in the same commit (stubbed pool instead of a closed port).