Phase 8b — the RxJS async-runtime bridge - #51
Merged
Wahbeh-Mohammad merged 1 commit intoAug 29, 2026
Conversation
Wahbeh-Mohammad
changed the title
Ships `@dexpace/rx`, exposing Phase 6b's `SseStream`/`typedSseStream` and Phase 6c's `Paginator` as RxJS `Observable`s, per docs/product-spec/18-asynchronous-runtime-adapter-contract.md (ASYNC-1..ASYNC-22), docs/product-spec/13-server-sent-events-and-streaming.md (SSE-41), and docs/superpowers/specs/2026-07-28-phase8b-async-runtime-design.md. New `packages/rx/src/`, the workspace's fifth package: - `sse.ts` — `sseEvents$`, `typedSse$` (SSE-41, SSE-33..SSE-36, ASYNC-21). Single-subscription, and deliberately so: `SseStream` wraps an already-open, single-use response body (BODY-14) and is itself single-pass (SSE-26), so a second `subscribe()` reaches the facade's own guard and surfaces `SseStreamError` through the error channel. The restriction is inherited and documented, not reimplemented — there is no honest way to make re-subscription mean anything without a second HTTP call this package does not make. - `pagination.ts` — `pageItems$`, `pages$` (PAGE-8). Cold and repeatable, the opposite asymmetry: `.items()`/`.pages()` build a fresh generator per call, and the wrapper defers that call to `[Symbol.asyncIterator]()` so each subscription drives an independent fetch sequence rather than reusing the first walk's exhausted generator. - `from-async-iterable.ts` — `fromAsyncIterable`, `@internal` (ASYNC-6, ASYNC-13, ASYNC-21). See below; this file is a deviation from the plan, not a component it called for. - `index.ts` — the four wrappers, nothing else. **The bridge is hand-written, against this phase's own instruction.** The design (§1) and plan (Global Constraints) both said: do not write an `AsyncIterable`→`Observable` pull loop, use RxJS's `from()`, and *prove* it satisfies each clause instead of assuming it. The proof failed on one. `rxjs@7.8.2`'s async-iterable path (`internal/observable/innerFrom.js`) is a bare `for await` that tests `subscriber.closed` only *after* a pull resolves, so unsubscribing while a pull is suspended reaches the source only if and when the server sends again. For pagination that is invisible; for SSE it is the common case — an idle event stream is permanently suspended, so `unsubscribe()` would leave the response body unreleased and the connection open indefinitely. That is ASYNC-6's bidirectional-cancellation clause unsatisfied, and SSE-30's release obligation with it. `fromAsyncIterable` is the same loop plus a teardown that releases the caller-supplied source and drives `iterator.return()` — release first, because closing the source is what settles the suspended pull an async generator's queued `return()` would otherwise sit behind. Scope is exactly the failing clause: no scheduler, no error re-wrapping, no retry, no buffering. The conformance suite's last case asserts the *defect* in RxJS's own `from()` alongside this module's fix, so it fails the day RxJS closes the gap and the module should be deleted. No RxJS scheduler appears in this package's production code, which is what keeps ASYNC-8..ASYNC-11 free: every pull runs inside the continuation chain that called `subscribe()`, exactly what `AsyncLocalStorage` tracks. A caller who adds `observeOn`/`subscribeOn` downstream reintroduces the boundary 7b's snapshot helper exists for, and the TSDoc says so rather than leaving them to find out from a missing trace ID. `rxjs` is a **required** peer (`optional: false`), unlike `pino`/`debug` in the logging adapters. Those peer on a library they never import — they are structural over `PinoLike`/`DebugLike` and work with anything of that shape. This package imports `Observable` unconditionally at module load, so an optional marking would suppress the one warning that catches a missing install and hand the consumer `ERR_MODULE_NOT_FOUND` instead. Peer rather than dependency for the usual reason: a duplicate copy breaks `Observable`/`Subscription` identity the way two classloaders break `instanceof`, the same hazard core's own peer-dedup guard exists to prevent. Tests: 25 across three colocated files, plus 8 Node-runtime cases in `test/node-conformance/rx-bridge.test.mjs`. The Node layer is not precautionary here — whether the release lands depends on Node's `ReadableStream.cancel()` settling a suspended read and on Node's async-generator `return()` queueing behind an in-flight `next()`, both independent implementations of Bun's. The conformance suite covers all four cancellation shapes (unsubscribe from inside `next()`, while a pull is suspended, before the first emission, and with a rejected release); ASYNC-21's poll-once-per-demand case uses a source that outlives demand — ten available, two taken, two pulled — since a generator yielding exactly what the subscriber consumes cannot tell one-pull-per-emission from a bridge that prefetches. Gate wiring: `typecheck`, `build`, `api`, and `lint:publish` extended to the new package; `verify-dual-consumption.mjs` drives a real `SseStream` through `sseEvents$` under plain `node`; `verify-consumer-types.mjs` compiles all four signatures against the built `.d.ts`. `verify:seam-1` and `verify:runtime-floor` pick the package up on their own. Open items registered in docs/open-items.md §M rather than left silent: M1 records the hand-written bridge and names its removal trigger; M2 flags that the checklist's eight 🚫 `ASYNC-*` rows collapse onto `TRANSPORT-*` requirements **no shipped package implements yet**, so an appendix-B sweep run between 8b and 8a does not count them as covered; M3 confirms at implementation time what the design predicted — ASYNC-18's scheduled-delay primitive is a full-port collapse, not an 8b scope boundary, since the as-built package contains no timer, scheduler, or backoff at all.
Wahbeh-Mohammad
force-pushed
the
23-phase-8b-async-runtime-bridge-dexpacerx
branch
from
August 29, 2026 07:50
c85f5da to
23b32c1
Compare
Wahbeh-Mohammad
added a commit
that referenced
this pull request
Aug 29, 2026
…async-runtime bridge (#53) * Phase 8a — the fetch and undici transport adapters, and the file-backed body (#52) * feat(transport): phase 8a — the fetch and undici adapters, and the file-backed body. Ships the two transport adapters, the file-backed request body, and the one conformance suite both adapters are proven against — the first code in this SDK that puts bytes on the wire, per product-spec/17-transport-adapter-conformance-contract.md (TRANSPORT-1..30), appendix C's SEAM-12/14/15/16/30, NFR-2/15 and BODY-11/12/13, and docs/superpowers/plans/2026-07-28-phase8a-transport-checklist.md. Four published packages and one private: - `@dexpace/transport-fetch` — a `Transport` over the runtime's global `fetch`, zero dependencies beyond its `@dexpace/core` peer. There is no `proxy` option at all: an absent option, not a silently ignored one, because Node's bare `fetch` exposes no proxy hook that does not route through `undici` internals (TRANSPORT-30, scoped out). `close()` is a sanctioned no-op over a runtime global it does not own, so `send()` keeps working after it — this adapter's documented SEAM-15 mode. - `@dexpace/transport-undici` — the full-featured one, taking exactly one external dependency. Ownership-aware `close()` over the dispatchers it constructed and never a bring-your-own one (SEAM-14), `NO_PROXY` bypass routed over a separate direct `Agent`, direct file-body dispatch honoring `start`/`count` (TRANSPORT-28), and a native-internal cancel told apart from a timeout by undici's own codes (TRANSPORT-8). An `Agent`, not a `Pool`: a `Pool` binds to one origin at construction, and a general-purpose transport must reach whatever origin each `Request` names. `undici` is loaded through `createRequire` by path, because Bun resolves the bare specifier to its own shim, whose `Agent` constructs but has no `request`. - `@dexpace/body-file` — the concrete `fileBody()` factory, fail-fast `node:fs` construction validation, a fresh handle per write, short-write detection. Cannot live in core, which imports no `node:` module. Neither transport depends on it: they recognize it structurally through `body.kind === 'file'` and core's type-only `FileBodyDescriptor`, never a cross-package `instanceof`. - `@dexpace/transport-shared` — the header drop/degrade pass, drop-log dedup policy, abort-to-SDK-error mapping, request-body pump, and delivery-detached signal fork. `@internal` exports only; published because a transport's `dependencies` must resolve for consumers. Exists so neither transport has to depend on its sibling. - `@dexpace/transport-conformance` — unpublished. The single `TRANSPORT-N` suite plus its `node:http` fixture server, run once per transport through each package's own `*.conformance.test.ts`, so no requirement is proven for one adapter and assumed for the other. Core gains `TransportFailureError` (TRANSPORT-20's canonical retryable no-response failure) and the type-only `FileBodyDescriptor`, plus a `'file'` member on `Body['kind']`. `IoError` is promoted from `@internal` to `@public` as its base class. The subtyping is the requirement, not modelling convenience: `classify.ts`'s cause-walk already returns true for every `IoError`, so a no-response failure is retryable with no edit to the retry layer. It costs a third hierarchy level against the styleguide's two-level cap, recorded as Deviation Ledger row 17 rather than left silent. SEAM-16 drove `signal-fork.ts`. Both native clients tie the response body's lifetime to whatever signal they were handed, so passing the caller's straight through would let a later `abort()` truncate a body the caller is still reading. Each transport dispatches over a fork it detaches at delivery: cancellation stays live for the whole in-flight window and goes inert afterwards. Five defects found reviewing the staged phase against the plan. One would have taken a consumer's process down. `transport-undici` never kept a handler on a streaming request body's producer. When a server answers before the body finishes — an early 413, a redirect — `send()` has already resolved, and a producer that then fails reaches Node's default `unhandledRejection` policy and terminates the process. `transport-fetch` was immune only incidentally, through the `Promise.race` it uses to surface producer failures. That race is now `producerFailure` in `transport-shared`, used by both, and the guarantee is a conformance row driven by a new `/early-response` fixture — verified to fail on the reintroduced defect and pass once reverted. Its absence is why the suite missed this: TRANSPORT-19 had no undici row at all, against the suite's own rule that no requirement is proven for one adapter and assumed for the other. Every non-abort dispatch failure was classified `TransportFailureError`, and `classify.ts` is an allow-list that returns true for every `IoError` — so undici's argument-validation codes, which are permanent and perfectly reproducible, were reported as always-retryable and would have spent a caller's whole retry budget re-proving the same rejection. `UND_ERR_INVALID_ARG` and `UND_ERR_NOT_SUPPORTED` now leave the `IoError` tree as a `TypeError`, matching `selectDispatchers`, which already reports caller misconfiguration that way. Reachable through a bring-your-own `ProxyAgent`, whose per-request `Proxy-Authorization` the owned-proxy drop set does not cover. The adaptation-throw path released the response body but not the request producer, leaving it parked on backpressure — both adapters, and the one non-delivering exit the TRANSPORT-19 audit trail claimed was covered. `verify:seam-1` had quietly weakened. Generalizing it to an NFR-2 allow-list replaced `deepEqual(dependencies, {})` with a key scan, so a package that omits `dependencies` entirely passed; an omitted field is not a hard-committed empty one. Every package outside the allow-list is held to the original assertion again, with the banner comment rewritten to describe the allow-list model it now implements. BODY-11 and TRANSPORT-28 were each tested in isolation and never together: no test sent a real `fileBody()` through a real transport, which matters most for undici, whose file path bypasses `writeTo` entirely for its own `createReadStream`. Covered now in `test/node-conformance/`, the only layer where a Node-only package and a transport can meet, whole and ranged, for both adapters. Gates: `verify:seam-1` becomes a per-package allow-list, because NFR-2 grants each optional capability core plus at most one external library — `transport-undici` takes `undici`, the rest take none. `verify:dual-consumption` exercises all five new packages under plain `node`; `verify:consumer-types` references every symbol the three consumer-facing ones promote and asserts only that `transport-shared`'s artifact exists, since no consumer is meant to import its `@internal` surface. `lint:publish` and `api` extend to all four published packages. Tests: 72 colocated cases across the four packages, 26 conformance rows run once per transport (capability-gated where §17 scopes a clause to one reference implementation), and eleven Node-runtime cases per adapter under `node --test`, because Bun's `fetch`, `AbortSignal`, and Web Streams are an independent implementation of the surfaces a transport is made of. Each header cites the IDs it exercises. Deliberate gaps, recorded rather than silent. TRANSPORT-18's re-subscribable producer is unbuildable here — neither client drives writes through one, so there is no native internal resend to make idempotent, and 5a's replayability gate covers the SDK's own retries. TRANSPORT-28's literal zero-copy path has no `sendfile`-shaped API in Node's HTTP client stack. TRANSPORT-27's Content-Length half is N/A: `Response.body` is a raw `ReadableStream`, with no declared-length field for a -1 sentinel to live in. TRANSPORT-14's degrade path is tested at its source, not end to end, because both native parsers reject a control byte in a header value at the wire first. TRANSPORT-30's custom `challengeHandler` cannot be dispatched on undici at all — `ProxyAgent` takes its credential solely from its constructor, which runs before any challenge is seen — so it warns at construction and again on the first real 407, and proxy auth falls back to Basic. That last one is a deviation from the phase plan, which had specified a retry-with-stamped-credential flow; Deviation Ledger row 13. * chore: resolve failing ci checks, add a new skill to run CI checks locally. * chore: resolve failing ci checks, fixes on the ci skill. * chore: resolve failing ci checks, fixes on the ci skill. * feat(rx): phase 8b — the RxJS async-runtime bridge (#51) Ships `@dexpace/rx`, exposing Phase 6b's `SseStream`/`typedSseStream` and Phase 6c's `Paginator` as RxJS `Observable`s, per docs/product-spec/18-asynchronous-runtime-adapter-contract.md (ASYNC-1..ASYNC-22), docs/product-spec/13-server-sent-events-and-streaming.md (SSE-41), and docs/superpowers/specs/2026-07-28-phase8b-async-runtime-design.md. New `packages/rx/src/`, the workspace's fifth package: - `sse.ts` — `sseEvents$`, `typedSse$` (SSE-41, SSE-33..SSE-36, ASYNC-21). Single-subscription, and deliberately so: `SseStream` wraps an already-open, single-use response body (BODY-14) and is itself single-pass (SSE-26), so a second `subscribe()` reaches the facade's own guard and surfaces `SseStreamError` through the error channel. The restriction is inherited and documented, not reimplemented — there is no honest way to make re-subscription mean anything without a second HTTP call this package does not make. - `pagination.ts` — `pageItems$`, `pages$` (PAGE-8). Cold and repeatable, the opposite asymmetry: `.items()`/`.pages()` build a fresh generator per call, and the wrapper defers that call to `[Symbol.asyncIterator]()` so each subscription drives an independent fetch sequence rather than reusing the first walk's exhausted generator. - `from-async-iterable.ts` — `fromAsyncIterable`, `@internal` (ASYNC-6, ASYNC-13, ASYNC-21). See below; this file is a deviation from the plan, not a component it called for. - `index.ts` — the four wrappers, nothing else. **The bridge is hand-written, against this phase's own instruction.** The design (§1) and plan (Global Constraints) both said: do not write an `AsyncIterable`→`Observable` pull loop, use RxJS's `from()`, and *prove* it satisfies each clause instead of assuming it. The proof failed on one. `rxjs@7.8.2`'s async-iterable path (`internal/observable/innerFrom.js`) is a bare `for await` that tests `subscriber.closed` only *after* a pull resolves, so unsubscribing while a pull is suspended reaches the source only if and when the server sends again. For pagination that is invisible; for SSE it is the common case — an idle event stream is permanently suspended, so `unsubscribe()` would leave the response body unreleased and the connection open indefinitely. That is ASYNC-6's bidirectional-cancellation clause unsatisfied, and SSE-30's release obligation with it. `fromAsyncIterable` is the same loop plus a teardown that releases the caller-supplied source and drives `iterator.return()` — release first, because closing the source is what settles the suspended pull an async generator's queued `return()` would otherwise sit behind. Scope is exactly the failing clause: no scheduler, no error re-wrapping, no retry, no buffering. The conformance suite's last case asserts the *defect* in RxJS's own `from()` alongside this module's fix, so it fails the day RxJS closes the gap and the module should be deleted. No RxJS scheduler appears in this package's production code, which is what keeps ASYNC-8..ASYNC-11 free: every pull runs inside the continuation chain that called `subscribe()`, exactly what `AsyncLocalStorage` tracks. A caller who adds `observeOn`/`subscribeOn` downstream reintroduces the boundary 7b's snapshot helper exists for, and the TSDoc says so rather than leaving them to find out from a missing trace ID. `rxjs` is a **required** peer (`optional: false`), unlike `pino`/`debug` in the logging adapters. Those peer on a library they never import — they are structural over `PinoLike`/`DebugLike` and work with anything of that shape. This package imports `Observable` unconditionally at module load, so an optional marking would suppress the one warning that catches a missing install and hand the consumer `ERR_MODULE_NOT_FOUND` instead. Peer rather than dependency for the usual reason: a duplicate copy breaks `Observable`/`Subscription` identity the way two classloaders break `instanceof`, the same hazard core's own peer-dedup guard exists to prevent. Tests: 25 across three colocated files, plus 8 Node-runtime cases in `test/node-conformance/rx-bridge.test.mjs`. The Node layer is not precautionary here — whether the release lands depends on Node's `ReadableStream.cancel()` settling a suspended read and on Node's async-generator `return()` queueing behind an in-flight `next()`, both independent implementations of Bun's. The conformance suite covers all four cancellation shapes (unsubscribe from inside `next()`, while a pull is suspended, before the first emission, and with a rejected release); ASYNC-21's poll-once-per-demand case uses a source that outlives demand — ten available, two taken, two pulled — since a generator yielding exactly what the subscriber consumes cannot tell one-pull-per-emission from a bridge that prefetches. Gate wiring: `typecheck`, `build`, `api`, and `lint:publish` extended to the new package; `verify-dual-consumption.mjs` drives a real `SseStream` through `sseEvents$` under plain `node`; `verify-consumer-types.mjs` compiles all four signatures against the built `.d.ts`. `verify:seam-1` and `verify:runtime-floor` pick the package up on their own. Open items registered in docs/open-items.md §M rather than left silent: M1 records the hand-written bridge and names its removal trigger; M2 flags that the checklist's eight 🚫 `ASYNC-*` rows collapse onto `TRANSPORT-*` requirements **no shipped package implements yet**, so an appendix-B sweep run between 8b and 8a does not count them as covered; M3 confirms at implementation time what the design predicted — ASYNC-18's scheduled-delay primitive is a full-port collapse, not an 8b scope boundary, since the as-built package contains no timer, scheduler, or backoff at all.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
This pull request adds
@dexpace/rx. It is the fifth package in the workspace.The package makes the Phase 6 SSE streams and paginators available as RxJS
observables. It satisfies SSE-41 and the non-collapsed ASYNC-* subset.
The package has four public functions:
sseEvents$(stream)emits the parsed events of an SSE stream.pageItems$(paginator)emits each item across all pages.pages$(paginator)emitsThe two SSE functions acceptseStream
wraps a response body that you can read one time (BODY-14). The stream itself is single-pass (SSE-26). A seco the guard of the stream and sendsSseStreamError` to the error channel. The package inheritsthis limit. It does not add
The two pagination functionsginator.items()
andPaginator.pages()` build a new generator for each call (PAGE-8). Eachsubscription therefore startfetches.
The package does not use theplan instructed us to
use it. A conformance test showed that
from()fails one requirement clause.RxJS 7.8.2 examines the subsl completes. An idle
SSE stream never completes a pull. An unsubscribe operation therefore does not
reach the source. The responction stays open. This
is ASYNC-6 and SSE-30, unsatisfied.
from-async-iterable.tscorrects this. The module is internal. It contains thesame pull loop. It adds a tep releases the source
first. Then it calls
return()on the iterator. This order is necessary. Therelease settles the suspendeturn waits behind that
pull. The scope of the module is one requirement clause. It adds no scheduler,
no error wrapper, no retry l
One test asserts the defect . That test fails when
a future version of RxJS corrects the defect. At that time, delete
from-async-iterable.tsandrxjsis a required peer de The package importsObservableat module load. Every import fails withoutrxjs. The loggingadapters mark their peers as never import their
peer. They use structural types instead. This package is different.
The package installs no RxJS scheduler. Diagnostic context therefore propagates
without help (ASYNC-8 to ASY the continuation chain
that called
subscribe().AsyncLocalStoragetracks that chain. The TSDocstates this. It also states veOn
orsubscribeOn`owns the context on the far side.
The gate scripts now cover the new package.
verify-dual-consumption.mjsruns areal
SseStreamthroughssify-consumer-types.mjscompiles all four signatures against the built type declarations.
Tests: 25 unit tests in three files, and 8 Node-runtime tests in
`test/node-conformance/rx-brr is necessary here.
The cancellation path depends on the runtime. Node Web Streams and Node async
generators are independent iivalents.
Reviews done
Three review passes ran on t
against the architectural and structural rules.
paths, lifetimes, cancellation, ordering, and error types.
quality, test strength, and line-level style conformance.
A final audit ran after these passes. The audit repeated the full gate sequence
from a clean install. It appe the
rxjspeerdependency required, it corrected the documented contract of the internal
release callback, it strengtso that the test can
detect a prefetch, and it added two error-path tests for the pagination
wrappers.
All gates are green:
typechecklintbuildbun testapilint:publishverify:dual-consumptionverify:consumer-typesverify:seam-1verify:runtime-floortest:nodeauditOpen and deferred items
Recorded in `docs/open-items blocks the merge.
in the plan. It is not a deviation from the product specification. ASYNC-6,
ASYNC-13, ASYNC-21, and SStten. The deviation
ledger in
docs/sdk-design-nodejs/10-...is unchanged. Removal trigger: anRxJS release that correctsformance suite detects
that release and fails.
-5, -15, -16, -17, -20, and -22 with a collapse symbol. Each one collapses
onto a TRANSPORT- requireements those
requirements yet. Phase 8a owns them. Do not count these eight MUST clauses as
covered in an appendix-B s
in this port needs a schedlt package confirms it.
It contains no timer, no scheduler, and no backoff logic. SSE reconnection
stays with the caller. Retase 5a engine. No
further action is needed.