Skip to content

Security audit 2026-10: certifi fix, CI hardening, chunk cap, redirect guard, markup escaping - #17

Merged
erxxc merged 3 commits into
mainfrom
security/audit-2026-10
Oct 23, 2026
Merged

erxxc merged 3 commits into
mainfrom
security/audit-2026-10

Conversation

@erxxc

@erxxc erxxc commented Oct 22, 2026

Copy link
Copy Markdown
Owner

Summary

Security audit of 2026-10-22: dependency vulnerabilities, secrets history, GitHub-side settings, CI supply-chain posture, and a static code review with limited offline probing of the archive, feed, LLM, and output surfaces. Three commits:

  1. Lockfile refresh — fixes the one known vulnerability (certifi 2026.7.22, compromised eMudhra intermediates, PYSEC-2026-4320) and moves the lock to the resolution CI already installs, including Anthropic SDK 2.1.0 (released 2026-10-21; verified: 307 tests and mypy on the latest set). pip-audit now reports nothing.
  2. CI and repo hygiene — actions SHA-pinned (checkout v7.0.1, setup-python v7.1.0, were two majors behind), read-only GITHUB_TOKEN, Python 3.13 and 3.14 in the matrix (3.14 verified locally), dependabot.yml for pip and GitHub Actions, SECURITY.md.
  3. Review fixes — see the commit message for detail:
    • High: per-analysis chunk cap (CHAINWATCH_MAX_CHUNKS_PER_ANALYSIS, default 40). A hostile release touching thousands of small files could force thousands of billed LLM calls from one diff. Dropped files are recorded in diff_summary.files_not_analyzed (schema 0.4.0) and named in a caveat.
    • Medium: outbound URL guard on the shared HTTP client, so registry/feed requests and every redirect hop get the same checks the tarball downloader already had.
    • Medium: all externally sourced text in the Rich report rendered literally, closing a [link=...]/style injection via LLM-quoted package comments.
    • Low: scoped package names URL-encoded in the Scorecard repo-resolution URLs; .envrc gitignored.

What the review found solid

Tar and zip extraction (traversal, symlink, hardlink, absolute-path, oversized-member and lied-size zip-bomb probes all rejected with nothing written), integrity verification fails closed, setup.py is AST-only, the API key is unwrapped once and never logged, lockfile parsers and the chunker scale linearly on adversarial input.

Needs the repository owner (cannot be done from a token without admin scopes)

  • Enable Dependabot alerts and security updates (Settings → Code security). dependabot.yml only requests version-update PRs.
  • Enable secret scanning and push protection.
  • Enable private vulnerability reporting, which SECURITY.md now points to.
  • Consider a ruleset on main requiring the CI check before merge.

Test plan

  • 323 passed (16 new), coverage 90.9%; ruff and mypy clean
  • All 34 dataset reports validate against schema 0.4.0; a live stub-LLM run against the npm registry works through the guarded client
  • Full suite also green on Python 3.14 and on the latest dependency set in isolated venvs

🤖 Generated with Claude Code

https://claude.ai/code/session_01SrGsKknvZvNgFwvTVnHdUA

erxxc and others added 3 commits October 22, 2026 08:22
pip-audit against the previous lock reported one known vulnerability:
certifi 2026.7.22 bundles the "emSign ECC Root CA - G3" root, whose
eMudhra intermediates have compromised private keys. An on-path attacker
holding one of those keys could forge TLS for the registry and feed hosts
this tool talks to. certifi 2026.10.19 removes the affected roots.

The rest of the refresh moves the lock to the same resolution CI already
installs from scratch (the project pins only lower bounds): anthropic
1.4.0 -> 2.1.0 (released 2026-10-21; the mocked AsyncAnthropic paths and
the TextBlock import are unchanged), pydantic 2.13.5 -> 2.14.1,
cryptography 50.0.1 -> 50.0.2, and patch bumps elsewhere. Verified on the
refreshed lock: 307 tests pass, coverage 90.8%, ruff and mypy clean,
pip-audit reports no known vulnerabilities.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SrGsKknvZvNgFwvTVnHdUA
CI: actions pinned to commit SHAs instead of mutable major tags
(checkout v7.0.1, setup-python v7.1.0 -- both were two majors behind), a
read-only GITHUB_TOKEN via top-level `permissions: contents: read`, and
Python 3.13 and 3.14 added to the matrix (3.14 verified locally, 307
tests). For a tool whose subject is supply-chain compromise, a tag that
can be moved to different code is the wrong kind of pin.

.github/dependabot.yml: weekly grouped PRs for pip and github-actions so
the lockfile and the action SHAs stop drifting silently; the last CI run
on main was five weeks old and had never seen the current SDK major.

SECURITY.md: how to report, what is in scope (the archive and feed
boundary, secret leakage, prompt injection that changes behaviour rather
than scores), what is out of scope (model mis-scoring is a research
finding, not a vulnerability), and the one-release support window.

pyproject classifiers now list 3.13 and 3.14 to match the matrix.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SrGsKknvZvNgFwvTVnHdUA
…ndering

Three findings from the 2026-10-22 review of the archive, feed, LLM, and
output surfaces, plus one correctness fix it noticed in passing.

High — unbounded LLM calls per analysis. Each chunk is one billed call,
and the count was bounded only by byte limits, never by file count: a
hostile release touching thousands of small source files (well under
max_archive_files) would cost thousands of calls from a single `diff`,
multiplied again under `scan`. New CHAINWATCH_MAX_CHUNKS_PER_ANALYSIS
(default 40) is enforced in chunk_diff(); files past the cap are listed
in diff_summary.files_not_analyzed (schema 0.3.0 -> 0.4.0, additive) and
named in a caveat, matching how skipped and truncated files are already
recorded.

Medium — redirects on the shared HTTP client. download_with_limit() already
re-validated every redirect hop against https-only / no-credentials /
no-localhost / no-non-public-IP rules, but the client used for registry
metadata, OSV, Rekor, and Scorecard had follow_redirects=True with no
check, so a compromised mirror or feed could steer a request at localhost
or a cloud metadata address. The same rules now run as a request event
hook on every request and hop (validate_outbound_url), and
build_http_client() accepts a transport so tests can exercise the hook.

Medium — Rich markup injection. The LLM quotes attacker-controlled diff
text verbatim, and Rich parses plain strings for [link=...] and style
tags, so a hostile comment could plant a real terminal hyperlink or a
styled "all clear" inside the tool's own framing. Reasoning, summary,
feed details and URLs, package names, hook names, caveat file paths, and
scan errors now render as Text() or through markup.escape().

Low — scoped package names were not URL-encoded in the two
_resolve_github_repo URLs, silently degrading Scorecard to no_data for
@scope/name packages. .envrc joins .env in .gitignore.

Tests: 323 passed (16 new: chunk cap, redirect guard, markup rendering),
coverage 90.9%, ruff and mypy clean; all 34 dataset reports validate.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SrGsKknvZvNgFwvTVnHdUA
@erxxc
erxxc merged commit f7c42c3 into main Oct 23, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant