Skip to content

fix(skills): scope the GitHub import size cap to the skill subpath#394

Open
pjdoland wants to merge 1 commit into
plmbr:mainfrom
pjdoland:fix/skills-monorepo-size-cap
Open

fix(skills): scope the GitHub import size cap to the skill subpath#394
pjdoland wants to merge 1 commit into
plmbr:mainfrom
pjdoland:fix/skills-monorepo-size-cap

Conversation

@pjdoland

Copy link
Copy Markdown
Collaborator

Summary

Installing a skill from a subpath of a large repo fails with ValueError: Extracted archive exceeds size limit (20 MB) even when the skill itself is tiny. _extract_skill counted every file in the GitHub tarball against MAX_EXTRACTED_BYTES, but GitHub tarballs always contain the whole repo while only the subpath subtree gets installed. In practice this means any skill hosted in a monorepo larger than 20 MB cannot be installed at all, and in a managed-skills manifest pointing at such a repo every entry fails identically on reconcile. This was hit in production with a skills monorepo where all four skills failed with the same trace through skill_reconciler.reconcile -> install_managed_from_github -> stage_skill_from_github -> _extract_skill.

Solution

Scope both the size count and the extraction to the {top_dir}/{subpath} subtree. The load-bearing change is the member filter in _extract_skill: members outside the subtree are skipped entirely, so they neither count against the cap nor get written to the temp dir (previously the whole repo was extracted, then everything but the subtree deleted). Two related behaviors were kept deliberately:

  • The unsafe-path check (absolute names, .. segments) still runs over every member before the subtree filter, so a hostile name anywhere in the archive fails the import loudly rather than being silently skipped.
  • When the subtree filter leaves no members for a non-empty subpath, the import raises Path '<subpath>' not found in repo early, before any extraction.

Side benefits: unrelated repo content is never written to disk during an import, and extraction I/O for a small skill in a large monorepo drops accordingly. The cap semantics for whole-repo skills (no subpath) are unchanged.

Testing

  • Six new regression tests in tests/test_skill_github_import.py: the monorepo case (large sibling content no longer trips the cap, and is not extracted), the cap still enforced on the skill subtree itself (with and without a subpath), sibling-directory prefix confusion (skills/foobar must not match skills/foo), a subpath naming a file (clean "not found"), and an unsafe path outside the subtree (still rejected).
  • Full Python suite: 1312 passed. jlpm tsc --noEmit, jlpm lint:check, and jlpm jest green (no TypeScript changes).
  • Playwright verification skipped: this is backend tar-extraction logic with no JupyterLab runtime surface; the unit tests fully pin the behavior.

Risks / follow-ups

  • The error message for an oversized skill changed from "Extracted archive exceeds size limit" to "Extracted skill exceeds size limit"; nothing in the codebase matches the old string.
  • Pre-existing, out of scope: parse_github_url permits . path segments, which now surface as a "not found" error; and duplicate tar entries can raise OSError from extractall instead of a clean ValueError (cleanup still runs in both cases).

Reported via user support with production stack traces; no GitHub issue exists for this.

_extract_skill applied MAX_EXTRACTED_BYTES to every file in the repo
tarball, but GitHub tarballs always contain the whole repo while only
the subpath subtree gets installed. Any skill hosted in a monorepo
whose total content exceeded 20 MB therefore failed to install with
"Extracted archive exceeds size limit", regardless of the skill's own
size. This hit every skill in an affected managed-skills manifest
identically.

Count sizes and extract members only within the {top_dir}/{subpath}
subtree. This also stops unrelated repo content from being written to
the temp dir and raises a clear "not found" error when the subpath
does not exist in the archive. The unsafe-path check still runs over
every member before the subtree filter, so a hostile name anywhere in
the archive fails the import loudly.

Regression tests pin the monorepo case, the still-enforced cap on the
skill subtree itself (with and without a subpath), sibling-directory
prefix confusion (skills/foobar vs skills/foo), a file-valued subpath,
and unsafe paths outside the subtree.
@pjdoland pjdoland added the bug Something isn't working label Jul 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant