Phase 8 — the transport adapters, the file-backed body, and the RxJS async-runtime bridge - #53
Merged
Conversation
…ed body (#52) * feat(transport): phase 8a — the fetch and undici adapters, and the file-backed body. Ships the two transport adapters, the file-backed request body, and the one conformance suite both adapters are proven against — the first code in this SDK that puts bytes on the wire, per product-spec/17-transport-adapter-conformance-contract.md (TRANSPORT-1..30), appendix C's SEAM-12/14/15/16/30, NFR-2/15 and BODY-11/12/13, and docs/superpowers/plans/2026-07-28-phase8a-transport-checklist.md. Four published packages and one private: - `@dexpace/transport-fetch` — a `Transport` over the runtime's global `fetch`, zero dependencies beyond its `@dexpace/core` peer. There is no `proxy` option at all: an absent option, not a silently ignored one, because Node's bare `fetch` exposes no proxy hook that does not route through `undici` internals (TRANSPORT-30, scoped out). `close()` is a sanctioned no-op over a runtime global it does not own, so `send()` keeps working after it — this adapter's documented SEAM-15 mode. - `@dexpace/transport-undici` — the full-featured one, taking exactly one external dependency. Ownership-aware `close()` over the dispatchers it constructed and never a bring-your-own one (SEAM-14), `NO_PROXY` bypass routed over a separate direct `Agent`, direct file-body dispatch honoring `start`/`count` (TRANSPORT-28), and a native-internal cancel told apart from a timeout by undici's own codes (TRANSPORT-8). An `Agent`, not a `Pool`: a `Pool` binds to one origin at construction, and a general-purpose transport must reach whatever origin each `Request` names. `undici` is loaded through `createRequire` by path, because Bun resolves the bare specifier to its own shim, whose `Agent` constructs but has no `request`. - `@dexpace/body-file` — the concrete `fileBody()` factory, fail-fast `node:fs` construction validation, a fresh handle per write, short-write detection. Cannot live in core, which imports no `node:` module. Neither transport depends on it: they recognize it structurally through `body.kind === 'file'` and core's type-only `FileBodyDescriptor`, never a cross-package `instanceof`. - `@dexpace/transport-shared` — the header drop/degrade pass, drop-log dedup policy, abort-to-SDK-error mapping, request-body pump, and delivery-detached signal fork. `@internal` exports only; published because a transport's `dependencies` must resolve for consumers. Exists so neither transport has to depend on its sibling. - `@dexpace/transport-conformance` — unpublished. The single `TRANSPORT-N` suite plus its `node:http` fixture server, run once per transport through each package's own `*.conformance.test.ts`, so no requirement is proven for one adapter and assumed for the other. Core gains `TransportFailureError` (TRANSPORT-20's canonical retryable no-response failure) and the type-only `FileBodyDescriptor`, plus a `'file'` member on `Body['kind']`. `IoError` is promoted from `@internal` to `@public` as its base class. The subtyping is the requirement, not modelling convenience: `classify.ts`'s cause-walk already returns true for every `IoError`, so a no-response failure is retryable with no edit to the retry layer. It costs a third hierarchy level against the styleguide's two-level cap, recorded as Deviation Ledger row 17 rather than left silent. SEAM-16 drove `signal-fork.ts`. Both native clients tie the response body's lifetime to whatever signal they were handed, so passing the caller's straight through would let a later `abort()` truncate a body the caller is still reading. Each transport dispatches over a fork it detaches at delivery: cancellation stays live for the whole in-flight window and goes inert afterwards. Five defects found reviewing the staged phase against the plan. One would have taken a consumer's process down. `transport-undici` never kept a handler on a streaming request body's producer. When a server answers before the body finishes — an early 413, a redirect — `send()` has already resolved, and a producer that then fails reaches Node's default `unhandledRejection` policy and terminates the process. `transport-fetch` was immune only incidentally, through the `Promise.race` it uses to surface producer failures. That race is now `producerFailure` in `transport-shared`, used by both, and the guarantee is a conformance row driven by a new `/early-response` fixture — verified to fail on the reintroduced defect and pass once reverted. Its absence is why the suite missed this: TRANSPORT-19 had no undici row at all, against the suite's own rule that no requirement is proven for one adapter and assumed for the other. Every non-abort dispatch failure was classified `TransportFailureError`, and `classify.ts` is an allow-list that returns true for every `IoError` — so undici's argument-validation codes, which are permanent and perfectly reproducible, were reported as always-retryable and would have spent a caller's whole retry budget re-proving the same rejection. `UND_ERR_INVALID_ARG` and `UND_ERR_NOT_SUPPORTED` now leave the `IoError` tree as a `TypeError`, matching `selectDispatchers`, which already reports caller misconfiguration that way. Reachable through a bring-your-own `ProxyAgent`, whose per-request `Proxy-Authorization` the owned-proxy drop set does not cover. The adaptation-throw path released the response body but not the request producer, leaving it parked on backpressure — both adapters, and the one non-delivering exit the TRANSPORT-19 audit trail claimed was covered. `verify:seam-1` had quietly weakened. Generalizing it to an NFR-2 allow-list replaced `deepEqual(dependencies, {})` with a key scan, so a package that omits `dependencies` entirely passed; an omitted field is not a hard-committed empty one. Every package outside the allow-list is held to the original assertion again, with the banner comment rewritten to describe the allow-list model it now implements. BODY-11 and TRANSPORT-28 were each tested in isolation and never together: no test sent a real `fileBody()` through a real transport, which matters most for undici, whose file path bypasses `writeTo` entirely for its own `createReadStream`. Covered now in `test/node-conformance/`, the only layer where a Node-only package and a transport can meet, whole and ranged, for both adapters. Gates: `verify:seam-1` becomes a per-package allow-list, because NFR-2 grants each optional capability core plus at most one external library — `transport-undici` takes `undici`, the rest take none. `verify:dual-consumption` exercises all five new packages under plain `node`; `verify:consumer-types` references every symbol the three consumer-facing ones promote and asserts only that `transport-shared`'s artifact exists, since no consumer is meant to import its `@internal` surface. `lint:publish` and `api` extend to all four published packages. Tests: 72 colocated cases across the four packages, 26 conformance rows run once per transport (capability-gated where §17 scopes a clause to one reference implementation), and eleven Node-runtime cases per adapter under `node --test`, because Bun's `fetch`, `AbortSignal`, and Web Streams are an independent implementation of the surfaces a transport is made of. Each header cites the IDs it exercises. Deliberate gaps, recorded rather than silent. TRANSPORT-18's re-subscribable producer is unbuildable here — neither client drives writes through one, so there is no native internal resend to make idempotent, and 5a's replayability gate covers the SDK's own retries. TRANSPORT-28's literal zero-copy path has no `sendfile`-shaped API in Node's HTTP client stack. TRANSPORT-27's Content-Length half is N/A: `Response.body` is a raw `ReadableStream`, with no declared-length field for a -1 sentinel to live in. TRANSPORT-14's degrade path is tested at its source, not end to end, because both native parsers reject a control byte in a header value at the wire first. TRANSPORT-30's custom `challengeHandler` cannot be dispatched on undici at all — `ProxyAgent` takes its credential solely from its constructor, which runs before any challenge is seen — so it warns at construction and again on the first real 407, and proxy auth falls back to Basic. That last one is a deviation from the phase plan, which had specified a retry-with-stamped-credential flow; Deviation Ledger row 13. * chore: resolve failing ci checks, add a new skill to run CI checks locally. * chore: resolve failing ci checks, fixes on the ci skill. * chore: resolve failing ci checks, fixes on the ci skill.
Ships `@dexpace/rx`, exposing Phase 6b's `SseStream`/`typedSseStream` and Phase 6c's `Paginator` as RxJS `Observable`s, per docs/product-spec/18-asynchronous-runtime-adapter-contract.md (ASYNC-1..ASYNC-22), docs/product-spec/13-server-sent-events-and-streaming.md (SSE-41), and docs/superpowers/specs/2026-07-28-phase8b-async-runtime-design.md. New `packages/rx/src/`, the workspace's fifth package: - `sse.ts` — `sseEvents$`, `typedSse$` (SSE-41, SSE-33..SSE-36, ASYNC-21). Single-subscription, and deliberately so: `SseStream` wraps an already-open, single-use response body (BODY-14) and is itself single-pass (SSE-26), so a second `subscribe()` reaches the facade's own guard and surfaces `SseStreamError` through the error channel. The restriction is inherited and documented, not reimplemented — there is no honest way to make re-subscription mean anything without a second HTTP call this package does not make. - `pagination.ts` — `pageItems$`, `pages$` (PAGE-8). Cold and repeatable, the opposite asymmetry: `.items()`/`.pages()` build a fresh generator per call, and the wrapper defers that call to `[Symbol.asyncIterator]()` so each subscription drives an independent fetch sequence rather than reusing the first walk's exhausted generator. - `from-async-iterable.ts` — `fromAsyncIterable`, `@internal` (ASYNC-6, ASYNC-13, ASYNC-21). See below; this file is a deviation from the plan, not a component it called for. - `index.ts` — the four wrappers, nothing else. **The bridge is hand-written, against this phase's own instruction.** The design (§1) and plan (Global Constraints) both said: do not write an `AsyncIterable`→`Observable` pull loop, use RxJS's `from()`, and *prove* it satisfies each clause instead of assuming it. The proof failed on one. `rxjs@7.8.2`'s async-iterable path (`internal/observable/innerFrom.js`) is a bare `for await` that tests `subscriber.closed` only *after* a pull resolves, so unsubscribing while a pull is suspended reaches the source only if and when the server sends again. For pagination that is invisible; for SSE it is the common case — an idle event stream is permanently suspended, so `unsubscribe()` would leave the response body unreleased and the connection open indefinitely. That is ASYNC-6's bidirectional-cancellation clause unsatisfied, and SSE-30's release obligation with it. `fromAsyncIterable` is the same loop plus a teardown that releases the caller-supplied source and drives `iterator.return()` — release first, because closing the source is what settles the suspended pull an async generator's queued `return()` would otherwise sit behind. Scope is exactly the failing clause: no scheduler, no error re-wrapping, no retry, no buffering. The conformance suite's last case asserts the *defect* in RxJS's own `from()` alongside this module's fix, so it fails the day RxJS closes the gap and the module should be deleted. No RxJS scheduler appears in this package's production code, which is what keeps ASYNC-8..ASYNC-11 free: every pull runs inside the continuation chain that called `subscribe()`, exactly what `AsyncLocalStorage` tracks. A caller who adds `observeOn`/`subscribeOn` downstream reintroduces the boundary 7b's snapshot helper exists for, and the TSDoc says so rather than leaving them to find out from a missing trace ID. `rxjs` is a **required** peer (`optional: false`), unlike `pino`/`debug` in the logging adapters. Those peer on a library they never import — they are structural over `PinoLike`/`DebugLike` and work with anything of that shape. This package imports `Observable` unconditionally at module load, so an optional marking would suppress the one warning that catches a missing install and hand the consumer `ERR_MODULE_NOT_FOUND` instead. Peer rather than dependency for the usual reason: a duplicate copy breaks `Observable`/`Subscription` identity the way two classloaders break `instanceof`, the same hazard core's own peer-dedup guard exists to prevent. Tests: 25 across three colocated files, plus 8 Node-runtime cases in `test/node-conformance/rx-bridge.test.mjs`. The Node layer is not precautionary here — whether the release lands depends on Node's `ReadableStream.cancel()` settling a suspended read and on Node's async-generator `return()` queueing behind an in-flight `next()`, both independent implementations of Bun's. The conformance suite covers all four cancellation shapes (unsubscribe from inside `next()`, while a pull is suspended, before the first emission, and with a rejected release); ASYNC-21's poll-once-per-demand case uses a source that outlives demand — ten available, two taken, two pulled — since a generator yielding exactly what the subscriber consumes cannot tell one-pull-per-emission from a bridge that prefetches. Gate wiring: `typecheck`, `build`, `api`, and `lint:publish` extended to the new package; `verify-dual-consumption.mjs` drives a real `SseStream` through `sseEvents$` under plain `node`; `verify-consumer-types.mjs` compiles all four signatures against the built `.d.ts`. `verify:seam-1` and `verify:runtime-floor` pick the package up on their own. Open items registered in docs/open-items.md §M rather than left silent: M1 records the hand-written bridge and names its removal trigger; M2 flags that the checklist's eight 🚫 `ASYNC-*` rows collapse onto `TRANSPORT-*` requirements **no shipped package implements yet**, so an appendix-B sweep run between 8b and 8a does not count them as covered; M3 confirms at implementation time what the design predicted — ASYNC-18's scheduled-delay primitive is a full-port collapse, not an 8b scope boundary, since the as-built package contains no timer, scheduler, or backoff at all.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
New packages
@dexpace/transport-fetch— aTransportover the runtime's globalfetch. It has nodependencies. It offers no
proxyoption, because Node's barefetchexposes no proxy hook.@dexpace/transport-undici— the full-featured transport. It closes only the dispatchers itbuilt, honors
NO_PROXYover a directAgent, and dispatches file bodies withstart/count.@dexpace/body-file— thefileBody()factory. It validates the file at construction and opensa fresh handle per write. Core cannot hold it, because core imports no
node:module.@dexpace/transport-shared— the parts both transports share: header drop and degrade rules,drop-log dedup, abort mapping, the request-body pump, and the delivery-detached signal fork.
@dexpace/transport-conformance— private. OneTRANSPORT-Nsuite and itsnode:httpfixtureserver. Each transport runs the same suite, so no clause is proven for one adapter only.
@dexpace/rx— exposesSseStreamandPaginatoras RxJS observables:sseEvents$,typedSse$,pageItems$,pages$.rxjsis a required peer. The package ships its ownfromAsyncIterable, because RxJSfrom()does not cancel a suspended pull.Core
TransportFailureErrorand the type-onlyFileBodyDescriptor.'file'toBody['kind'].IoErrorto@publicas the base class, so retry classification needs no edit.Gates
build:depsreplacesbuild:coreas the prefix oftypecheck,lint,fix, andbuild. Itbuilds core and
transport-shared.typecheck,build,api, andlint:publishnow cover every new package.verify:seam-1becomes a per-package allow-list for NFR-2. It again fails a package that omitsdependenciesinstead of committing an empty object.verify:dual-consumptionandverify:consumer-typesexercise the new packages.New CI preflight check
ci-preflightskill.node .claude/skills/ci-preflight/run-ci.mjsruns all 14 blockingCI steps locally, in CI's order, and reports every failure at once.
node_modules/.cache/ci-preflight/<step>.log. Only a summary reaches stdout.--cleandeletes eachdist/and*.tsbuildinfofirst, so the run starts from the tree CIchecks out. Use it before a push.
.bun-versionby default, because transport behavior differs between Bunreleases.
Tests
@dexpace/rx, plus its conformance suite for the four cancellation shapes.rx bridge.
Docs
rows 13 and 17.
CLAUDE.mdrecords thebuild:depsrule and preflight runner.