Extract GitHub API client into @exodus/stasis-api package - #149
Open
exo-nikita wants to merge 7 commits into
Open
Extract GitHub API client into @exodus/stasis-api package#149exo-nikita wants to merge 7 commits into
exo-nikita wants to merge 7 commits into
Conversation
…client Registry access lived inside `stasis` as `src/apis/npm/`, which meant the npm advisories client and the lazily-bound `semver` shim were only reachable by importing the whole CLI package. Move them into a separate zero-dependency workspace package so API clients are versioned and consumed on their own. - `@exodus/stasis-api/npm` — `advisories()`, moved verbatim - `@exodus/stasis-api/npm/semver` — the npm-CLI-backed semver shim, moved verbatim - `@exodus/stasis-api/github` — new: `releases()`, `release()`, `latestRelease()` and `asset()` for listing releases and fetching their attachments `stasis` now depends on `@exodus/stasis-api`; `audit.js`, `audit-corrections.js` and `sbom.js` import through the package specifier instead of relative paths. The GitHub client is transport-only, matching the npm one: no disk, no caching, and no credential discovery — a `token` is always passed explicitly, so one is never picked up from the environment. Owner/repo slugs are validated (not just escaped) and tags are percent-encoded so a caller-supplied value can't reshape the endpoint it lands in; `Link: rel="next"` pagination refuses to follow a link off the API host, which would carry the token elsewhere. `asset()` verifies the listing's `digest` before returning bytes, so a truncated or swapped download fails at the boundary rather than downstream. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DKbtN73UaD2stKyg6XSKKN
Cleanup pass over the new package; no intended behavior change beyond the slug fix below. - Lift the fetch + error-wrap shell into `src/request.js` (`requestOk`). The `—` separator, the 200-char body cap, the swallowed body-read failure and the cause chain were encoded once per client, so a third client would have copied them again and anything GitHub needs later (retry, Retry-After) would land in N places. Both clients' error strings are unchanged. - Validate repo slugs as path segments instead of re-encoding GitHub's account naming policy. The old rule required a leading word character, which refused real repos — `ExodusOSS/.github` among them — while `.`/`..` and non-segment characters are what actually need rejecting. - `repoSlug()` rebuilt a string identical to its input; it is now `assertRepo()` and reads as the validator it always was. - Name the two request timeouts rather than repeating `30_000` in three signatures, and drop `request()`'s unreachable `token = null` default. - Trim comments that restated the code or repeated a nearby comment, and correct two that overclaimed: the normalizers narrow to a curated view rather than to "what is needed to fetch and verify", and the package is not disk-free — the semver shim reads npm's bundled copy. - README: fix the same disk claim, and stop showing a serialized download loop as the canonical example. - Tests: rejection-path tests now install a fetch that throws, so "no request was made" holds by construction instead of by a trailing assertion; add coverage for the repo names GitHub actually allows. `stasis/README.md` said `audit` builds on the npm and GitHub clients; it builds on the npm one only. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DKbtN73UaD2stKyg6XSKKN
`subtree(repo, ref, { path, format })` returns a repo subtree at an exact
commit, tag or branch, as a Map of repo-relative posix path -> bytes. No git
client is involved and nothing is written to disk: one request to GitHub's
archive endpoint, decompressed and parsed in memory.
- `src/archive.js` — in-memory readers for both containers GitHub serves a tree
as: gzipped tar (`/tarball`, default) and zip (`/zipball`). Node's zlib does
the decompression; the container framing is parsed here, so the package stays
dependency-free.
- tar: ustar with `prefix` long names, GNU 'L' long names, pax 'x' path
overrides, and base-256 sizes.
- zip: central-directory driven, so sizes are read from the authoritative
record rather than a local header that may defer them to a data descriptor.
Stored and deflated entries; zip64 is rejected rather than misread.
- Only regular files are returned. Directories and symlinks carry no usable
content, and a symlink target is precisely what could point out of the tree.
- Entries that escape the tree (absolute, `..`, backslash, NUL) are rejected
rather than sanitized, as are archives with more than one top-level directory.
- The archive's top-level directory is returned as `root` — it records the
commit the ref resolved to — and is taken from the entries rather than
reconstructed from a name pattern and assumed to match.
- `maxBytes` (256 MiB default) bounds the download from both `content-length`
and the streamed byte count, since the archive is held in memory to parse.
- A `path` matching nothing throws: that is a typo far more often than an
intentionally empty subtree, and an empty Map would look like success.
Verified against real GitHub archives, not only the hand-built fixtures: the tar
and zip readers are independent code paths and decode the same commit to
identical bytes across all 476 files of a real repo. That check is also what
caught the archive endpoints answering 415 to `Accept: application/octet-stream`
— the header the asset endpoint requires — so the archive request keeps the
default JSON Accept and lets the codeload redirect serve the bytes.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DKbtN73UaD2stKyg6XSKKN
- Drop the zip container. GitHub serves /tarball for every repo, so supporting both was a second way to do one job: `subtree()` loses its `format` option, `readZip` and its central-directory/EOCD/zip64 handling are gone, and `zipball_url` is no longer carried on a normalized release beside the tarball. - Split github/index.js by altitude. core.js holds the plumbing that knows how to talk to GitHub but not what to ask it — host, headers, token policy, identifier validation, Link pagination, capped body reads — and each endpoint family is its own module on top: releases.js (releases + assets) and subtree.js. index.js re-exports just the five public functions, so core.js stays internal and can change without breaking a consumer. - Assert an archive never repeats a path. tar can legally carry the same path twice, and reading into a Map would keep whichever came last, so a second entry could shadow the first and the shadowed bytes would never be seen by any caller. Refuse the archive instead of silently picking a winner. Re-verified against real GitHub archives after the restructure: the whole tree of a real repo still reads (476 files, no duplicate-path trip) and a subtree filter still resolves. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DKbtN73UaD2stKyg6XSKKN
Cleanups from a review pass over the new package. No public API change --
`asset, latestRelease, release, releases, subtree` and their option shapes are
untouched.
Structure:
- Add `repoUrl(repo, ...segments)` as the single route by which a slug becomes a
URL. `assertRepo` was a separate step every endpoint had to remember before
hand-interpolating `${API}/repos/${repo}/...` five times over; now the check
cannot be forgotten, and `API`/`assertRepo` stop being exported for string
concatenation at the leaves.
- Hold both halves of the timeout policy in core.js: `ASSET_TIMEOUT` and
`ARCHIVE_TIMEOUT` were the same 300s under two names in two files, while the
comment describing the policy lived in a third. Now one `TRANSFER_TIMEOUT`.
- Let `request()` own the token policy. The `token = null` sentinel was restated
in all five signatures and interpreted with a strict `!== null`, so an
endpoint that omitted the default would report "Expected a non-empty token"
for a caller who supplied none.
- Move `readCapped` to subtree.js. Its remediation hint names `maxBytes`, one
endpoint's option, so it was never core policy -- asset() reads its body
whole, whose size the caller already knows from the release listing.
- Fold `releasePage` into the pagination loop, and `release`/`latestRelease`
into one `oneRelease` -- they differed only in how the release is addressed.
Efficiency:
- Filter the archive while reading it. `subtree({ path })` copied every regular
file out of the tarball and then discarded most of them; `readTarGz` now takes
a `select` hook that re-keys and narrows during the walk, so a skipped entry's
bytes are never copied. On a 2000-file 9.2 MB archive keeping 20 files: 9.4 ms
and 0.08 MB copied, against 15.3 ms and 8.19 MB. The safety and duplicate-path
checks still run on every entry, so an archive that is unsafe or
self-shadowing outside the requested subtree is still refused -- covered by a
new test.
- Collapse three full Maps into one. readTar built one, stripRoot rebuilt it
re-keyed, subtree rebuilt it again to apply the prefix; the root is now taken
during the same walk.
- Read tar text fields by range instead of slicing two Buffer views per call
(three calls per entry), and test `..` with one regex instead of splitting
every path into segments.
- Drop `view()`: correct, but unreachable -- both call sites already pass a
Buffer, so it never converted anything.
Tests and packaging:
- One `tests/fetch.helper.js` replaces three copies of the same global-fetch
swap (audit.test.js had already drifted from the other two).
- Ship `src` instead of enumerating every module in `files`, which is one added
module away from publishing a tarball that omits it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DKbtN73UaD2stKyg6XSKKN
… maxBytes subtree() capped only the downloaded bytes; gunzipSync then decompressed without a bound, so an archive well under the cap could still attempt a huge allocation (gzip of zeros expands ~1000:1). Enforce maxBytes on the decompressed size via zlib's maxOutputLength, reported as the size refusal it is rather than as a malformed archive. Also fix a stale comment in github/index.js: capped body reads live in subtree.js, not core.js. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DKbtN73UaD2stKyg6XSKKN
Quality pass over the branch, no behavior changes intended: - The timeout numbers move to request.js, the shared shell both clients import, so npm's inline 30s literal becomes the same METADATA_TIMEOUT the GitHub endpoints use instead of coinciding with it. - The `<label> request failed` shape is now built only by request.js (requestFailed), and a new core.js readBody owns the `github <what>` labeling for body reads, so asset() stops hand-assembling both. - readTarGz inflates via promisified zlib on the threadpool -- a hundreds-of-MB decompression no longer blocks the event loop -- and loses its unreachable uncapped mode: both callers-to-be must pass select and maxBytes explicitly. - Comments right-sized: core.js and github/index.js claim slug/ref validation and next-link parsing (not all validation and pagination, parts of which are endpoint-level), archive.js describes the generic reader it is rather than defining itself by its GitHub caller. - Dead `token !== undefined` check dropped: the destructuring default already turns undefined into null. - Tests: audit.test.js adopts the json() helper its own extraction created (five inline expansions replaced, a shadowing local renamed), and the digest case test uppercases the whole string since asset() lowercases it anyway. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DKbtN73UaD2stKyg6XSKKN
exo-nikita
force-pushed
the
claude/stasis-api-directory-sbsuvz
branch
from
August 10, 2026 07:21
c3a6fbc to
7dacc40
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR creates a standalone
@exodus/stasis-apipackage: zero-dependency, transport-only registry API clients (npm, GitHub) reusable across projects. The npm advisories client moves over from@exodus/stasis; the GitHub client (releases + repo subtrees) is new.Summary
@exodus/stasis-apihas no disk access, no credential discovery, and no caching. Three export paths:@exodus/stasis-api/npm— npm bulk security advisories client (moved from@exodus/stasis)@exodus/stasis-api/npm/semver— lazily-bound semver from the npm CLI bundled with the running Node@exodus/stasis-api/github— GitHub releases, release assets, and repo subtreesGitHub client
Split into shared plumbing and endpoint families:
src/github/core.js— host, headers (versioned media type, User-Agent), optional Bearer token, identifier validation (repo slugs as strict path segments; refs percent-encoded after rejecting what git itself forbids), andLink: rel="next"pagination that only follows URLs on the GitHub API host, so a token can never be carried elsewheresrc/github/releases.js—releases()(paginated listing withlimit,Infinityto walk every page),release()(by tag),latestRelease(), andasset()(attachment download with optional SHA256/384/512 digest verification before the bytes are handed back)src/github/subtree.js—subtree(): a repo tree at an exact commit/tag/branch, fetched as one tarball request and read into aMapof repo-relative path → bytes, optionally narrowed to apathprefix while reading (entries outside it are vetted but never copied)src/archive.js— in-memory tar.gz reader (Node zlib + ustar/pax/GNU-longname parsing, no external tar). Only regular files are kept; unsafe paths (absolute,.., backslash, NUL), duplicate paths, and multi-root archives are refused rather than sanitized.maxBytes(256 MiB default) is enforced on the downloaded bytes and again on the decompressed size, so a compression bomb fails instead of exhausting the heapsrc/request.js— one shared transport shell used by both the npm and GitHub clients, producing the single uniform "… request failed: …" error shape and owning the package-wide timeout defaultsResponses are normalized to a fixed, curated shape (volatile fields like
node_id/uploaderdropped; onlytarball_urlkept since GitHub serves a tarball for every repo), so consumers cannot come to depend on the raw API payloads.Security posture
nextlinks off the API host; asset/tarball redirects rely on fetch droppingAuthorizationcross-originAbortSignal.timeout(30 s metadata, 300 s transfers), overridable viasignalTests
tests/github-api.test.js— releases/release/latestRelease/asset: normalization, pagination (Link following, limits, off-host link refusal), input validation before any network call, digest verification including mismatches and malformed digests, error shapestests/github-subtree.test.js— tar fixtures built byte-by-byte: path filtering, binary round-trips, pax/ustar-prefix long names, symlink/dir dropping, escape/duplicate/multi-root refusal, download and decompression caps (including a compression-bomb case), corrupt archivestests/fetch.helper.js— shared fetch stub, also adopted bytests/audit.test.jstests/public-exports.test.js— the new export paths are importable and expose the documented functions@exodus/stasisupdatesaudit.js/audit-corrections.jsimport from@exodus/stasis-api/npm,sbom.jsfrom@exodus/stasis-api/npm/semver; the embeddedsrc/apis/npm/files are removedstasisnow depends on@exodus/stasis-api@1.0.0-beta.2(exact pin, matchingstasis-core/stasis-plugins)Workspace
stasis-apiadded to the pnpm workspace and lockfile; root README and package description updated🤖 Generated with Claude Code
https://claude.ai/code/session_01DKbtN73UaD2stKyg6XSKKN