Repository navigation
feat: phase 6 — distribution & publishing (48 → 54 controls) - #71
Conversation
Before this, sscsb stopped where the artifact left. Phases 1-5 harden the repository, the dependencies it consumes, the CI that builds it and the files a GitHub Release carries; none of them covers the moment that artifact goes to crates.io, npm, PyPI, Homebrew, Chocolatey or WinGet, and none of them covers the credential that lets it. That is the Shai-Hulud and chalk/debug shape exactly: phish a maintainer, take a long-lived publish token, ship malware under a trusted name. No commit is involved, so every control we already had was looking the other way. 47 controls become 53. New `src/distribution.rs` owns offline, file-based detection of what a repository PUBLISHES — deliberately a separate type from `deps::Ecosystem`, which is what it CONSUMES, because a Rust crate that vendors npm packages consumes npm and publishes to crates.io — plus `.sscsb/policy/distribution.toml` and six verifiers: `publish-targets` (inventory, never grades), `trusted-publishing`, `maintainer-mfa`, `publish-tokens`, `publish-provenance`, `dist-manifests`. All six ship on, and phase 6 adds no off-by-default control, because the gate is DETECTION rather than configuration. A repository with no package.json is not a repository failing its npm posture, and saying so would be the false positive this project exists to avoid. Every verifier goes quiet on its own when its target is not in the tree, and every one of them degrades — naming which of the `degraded_reason` vocabulary applies — rather than failing when a tool or a credential is simply absent. `verify_branch_protection` is the precedent, and the GitHub environments read follows it literally: GitHub answers 404 both for "no such environment" and for "your token cannot see environments", which are opposite verdicts, so the ambiguous case degrades and says so. Telling a maintainer to create an environment that already exists is how a tool teaches people to stop reading it. Three publish templates ship in their own `DIST_ARTIFACTS` table with a detection-gated installer, NOT in `workflows::ARTIFACTS`. That table installs on control-enabled alone and `trusted-publishing` is on by default, so registering them there would have dropped `publish-npm.yml` into every repository sscsb ever touched. A publish workflow you do not publish with is not a harmless extra file; it is a workflow_dispatch button wired to somebody else's namespace. A test asserts the separation rather than only commenting it. Six phase bounds were hard-coded to 5, and one of them would have silently mis-scored every repository. `controls.rs` (twice), `cli.rs`, `compliance.rs` and `tests/skill_docs.rs` each iterated `1..=5`, so phase-6 controls would simply not have appeared in `status`, `report`, or the registry's own sparseness guard. The one that mattered is `machine.rs`'s `score_of`, which built `score.phases[]` from `1u8..=5`: a phase-6 row would have fallen out of both the numerator and the denominator while still counting toward `scoped`, so adding six PASSING controls would have LOWERED every repository's evidence coverage in the directory record. That literal is now `PHASE_COUNT`, derived from the registry, because the number is a wire contract with the site. `config.rs`'s banner had the same latent bug in prose form — `_ => "Phase 5 — Observability & governance"` is correct exactly until a sixth phase exists and then prints a wrong heading in the one file every user edits. Phases 5 and 6 get explicit arms, the fallback is a visible bug marker, and a test asserts no control renders it. Two defects were found by dogfooding this on sscsb itself, not by reading code. `fuzz/Cargo.toml` carries `publish = false` and detection called it a crates.io target — a crate that cannot be published even deliberately, dragging a real verdict down; `publish = false` and `publish = []` are now honoured. And this repository's own root manifest carries `publish = false` too, so sscsb is not on crates.io at all — while its README's first install line is `brew install p4gs/p4gs/sscsb`, from a tap in another repository that nothing in this tree can prove. Without `[targets] homebrew = "on"`, sscsb would have reported that its own project publishes nowhere while thousands of installs came through a channel it never looked at. That declaration is committed here, with the reasoning, as the worked example of why the override exists — and the checksum controls now say the manifest lives elsewhere instead of degrading over a file that was never supposed to be in this tree. Two smaller judgement calls worth naming. npm's `tfa.mode = auth-only` FAILS rather than degrading: it gates login but not the publish itself, which is the precise gap a stolen session walks through, and a control that called that "enabled" would be worse than none. GitHub's absent `two_factor_authentication` field DEGRADES rather than failing, because GitHub omits it for a token without `read:user` and reading that as `false` would accuse a maintainer holding a passkey of having no second factor. That same read is gated on a resolvable GitHub remote: without one, the account `gh` happens to be logged in as is whoever owns the laptop, not whoever owns the package. Attestations document and never upgrade an outcome — `[[token]] expires` and `[signing] expires` go through `signers.rs`'s own `evaluate_expiry`, and an expired claim fails. `[[account]] attested` deliberately does NOT: it counts BACKWARD from the day a human last checked, where a bigger gap is worse, and feeding it to an expiry evaluator would type-check while grading a claim dated tomorrow as comfortably fresh. Both directions are pinned by a test. `sscsb dist status` and `sscsb dist check` land; `sscsb publish` deliberately does not, and docs/phase-6.md records all three reasons so nobody re-proposes it — git hooks fire on git events and nothing intercepts `npm publish`, the doctrine here is to move publishing INTO CI behind an OIDC identity, and reimplementing six registries' publish clients buys no security property. Dogfood verdict on this repo: 0 failed, 1 degraded (maintainer-mfa, because the local gh token lacks `read:user` — the designed degrade, exercised live).
…tion # Conflicts: # .claude/skills/sscsb/SKILL.md # README.md # templates/skills/sscsb/SKILL.md
…/O off `src/distribution.rs` landed at 83.6% lines in CI — below this project's floor, and low for a reason worth fixing rather than noting: the branches that were missing are the ones an operator actually hits. Every verdict that depends on `gh`, on `npm`, or on a registry answering was unreachable from a fixture, so the tests proved the offline half of a module whose whole job is the online half. 83.6% -> ~94% lines, and the new cases are all assertions on behaviour rather than lines touched. The registry probe is now split the way `deps.rs` already splits `registry_exists` out of `deps_check_with`: `classify_probe` takes the fetch result as a value, `fetch` keeps the socket, and the four verdicts become testable. Those four are not interchangeable — a never-published package is a SKIP (you cannot fail to attest something you never shipped), an unreachable registry is UNVERIFIED and must never render as "no provenance", a real attestation is the only Pass, and npm's remediation (`--provenance`) is wrong advice for PyPI, where attestations are automatic. All five are pinned now; none of them was before, because the socket call sat inline. `testutil`'s decoy-PATH helpers cover the rest. A fake `gh` answers each environments-API shape so the environment gate is exercised through `verify_trusted_publishing` and not only through its pure classifier — an environment with no protection rules FAILS, because an OIDC identity nobody reviews is a workflow_dispatch button wired to your namespace. A fake `gh api user` covers 2FA on, 2FA off, and the response with the field absent, which is the case that must degrade rather than convict a maintainer holding a passkey. A fake `npm` covers `auth-and-writes`, `auth-only`, and an unrecognised mode, which fails rather than being waved through as probably fine. Three distinctions that were previously asserted only in prose now have tests. `gh` absent and `gh` refusing are different degrades (`tool-missing` versus `no-access`) because one says install something and the other says grant a scope. A repo with no GitHub remote degrades `no-remote` even when a `gh` that WOULD answer is on PATH — the fixture proves the gate is the remote rather than the tool, which is the difference between reading the package's owner and reading the laptop's. And a `CHOCO_API_KEY` in a workflow is reported, not failed, because Chocolatey has no OIDC path: failing it would demand a migration that does not exist, which is the fastest way to teach someone to ignore a tool. Also: the `.npmrc` fixture's value is now `EXAMPLE` rather than something shaped like a real npm granular token. The detector keys on the `_authToken` marker and never on the value, so the realistic-looking literal bought the test nothing while putting a token-shaped string into the history of the repository whose own control exists to keep them out. Suite 1087 passed / 33 failed locally, the 33 identical to the `main` baseline and all environmental to this sandbox (blocked DNS, a broken `opengrep`, denied socket bind). fmt and clippy clean.
CodeQL: 12
|
| Location | Sink |
|---|---|
src/distribution.rs ×11 |
assert!(…, "{:?}", r.messages) inside #[cfg(test)] |
tests/library.rs:1317 |
pre-existing — this is main's tests/library.rs:1316, re-reported because line numbers shifted |
This class is pre-existing on main
main currently has 5 open rust/cleartext-logging alerts of the same shape:
src/hooks.rs:532
tests/library.rs:1316
tests/library.rs:148
tests/library.rs:122
tests/library.rs:156
So this PR does not introduce the class; it adds instances of an existing, unresolved one.
Evidence-based determination
The taint source CodeQL reports is the function itself — verify_maintainer_mfa(...), verify_trusted_publishing(...), references_token_secret(...). That is the rule's name-matching heuristic firing on mfa / token / secret in identifiers, not a value-flow finding.
I audited every path that builds a message in src/distribution.rs. No credential value can reach one, by construction:
verify_publish_tokensreads.npmrc/.pypirccontent and testsline.contains(marker). The message carries the path and the marker key (_authToken) — never the line, never the value.references_token_secretreturns secret names (NPM_TOKEN), matched againstsecrets.<NAME>; it never reads a value, and values are not available to it.npm_tfa/github_two_factoruseprofile["name"]anduser["login"]— usernames, not credentials — plustfa.mode.- The only unbounded external text in any message is
out.stderr.lines().next(), which is the house-standard pattern already used bywebhooks.rsandaudit.rs.
A grep for file/command content flowing into messages.push/format! returns only paths and key names.
What I changed anyway
One genuine hygiene fix, not an appeasement: the .npmrc test fixture's value is now EXAMPLE instead of a string shaped like a real npm granular token. The detector keys on the marker and never on the value, so the realistic literal bought the test nothing while putting a token-shaped string into this repository's history — which is the thing the control exists to prevent.
What I deliberately did not do
The only mechanical way to clear the remaining alerts is to stop printing r.messages in assertion failures, or to rename the verifiers away from mfa/token/secret. Both make the code worse — the first removes the diagnostics that make a failing test debuggable, the second makes the functions lie about what they do — and neither removes a real exposure. Per the repo rule, a suppression needs your documented determination, not mine, so:
Decision for you. Either (a) accept the class as it stands on main, (b) have me strip the message payloads from these assertions repo-wide (I'd argue against it), or (c) dismiss the alerts in the Security tab with the rationale above. I've done none of them.
CodeQL flagged `rust/cleartext-logging` against `verify_maintainer_mfa`, and my first read of it was wrong. I checked the sinks CodeQL named — assertion strings in the test module — decided no credential reached them, and stopped. That missed the part that matters: a `VerifyResult`'s `messages` are not only printed. `machine.rs` serializes them into `--format json` AND into the local-scan record that is signed, committed, and published to the public directory. A message is an artifact with a signature on it. `maintainer-mfa` and `trusted-publishing` are the only controls here that ask a CREDENTIAL-HOLDING command about an ACCOUNT. `npm profile get --json` and `gh api user` both answer as the logged-in maintainer, and six production sites were copying their answers into messages verbatim: - the first stderr line of a failed `gh api`, `gh api user` and `npm profile get`. npm puts the registry URL in its errors, and a misconfigured `.npmrc` makes that URL carry inline basic-auth, so this is the dangerous one rather than the theoretical one; - `login` from the GitHub user payload and `name` from the npm profile payload (whose siblings include `email`); - the raw `tfa.mode` string; - an unrecognised `protection_rules[].type` from the environments payload. The rule for this module is now narrow and absolute: no byte of a tool's stdout or stderr is ever copied into a message. Only the specific field a control needs is read — `two_factor_authentication`, `tfa.mode` — and any of it that is free-form goes through `safe_label` (bounded charset, bounded length, refused whole rather than truncated, because a truncated secret is still a secret prefix). Failures are reported by `tool_failure` as an HTTP status, which is the entire diagnostic value of the line it replaces — authenticate, grant a scope, look elsewhere — and the one part that cannot carry a credential. Parse failures withhold the payload too: a profile that will not parse is still a profile. Spawn errors stay as they are, deliberately: they carry our own argv and an OS message, never anything a registry said. `no_tool_output_is_ever_echoed_into_a_published_message` hands both verifiers four plausible hostile answers with a planted marker and asserts it appears in no message. It earned its place immediately: it caught `tfa.mode` still leaking AFTER the sanitizer was in, because `safe_label` correctly passed the planted value — a secret can be perfectly well-formed as an identifier. `tfa.mode` is a closed enum, not a name, so a value outside npm's vocabulary is now reported as undocumented and WITHHELD rather than echoed. That is the bug the test existed to find, found on its first run. No suppression of any kind: no `#[allow]`, no inline CodeQL comment, no query-filter exclusion. Suite 1090 passed / 33 failed, the 33 identical to the `main` baseline and all environmental to this sandbox. fmt and clippy clean.
Six phase-6 tests were green on a developer's Mac and failed the CI runner with `Exec format error (os error 8)`, which names nothing the author did wrong. macOS execs a shebang-less text file by falling back to /bin/sh; Linux returns ENOEXEC. Every pre-existing `fake_tool` caller in audit.rs already spells `#!/bin/sh` — the convention was real and undocumented, and nothing enforced it. Fixed once in `write_shim` rather than six times at the call sites, so the next caller's omission is harmless too: the interpreter line is prepended only when the script does not already start with one, leaving every existing caller byte-identical. A regression test RUNS a shebang-less shim rather than inspecting its bytes, which is what makes it fail on the platform it protects. This is the fourth leak class named in CLAUDE.md's test-isolation section — the PATH-shaped one that passes locally and fails only in CI — arriving through a different door.
`only_git_on_path()` masks PATH down to git's own directory, which hides gh on a Mac — git is /usr/bin/git, gh is /opt/homebrew/bin/gh — and does NOT hide it on a GitHub runner, where both live in /usr/bin. Two tests asserting a tool-missing degrade therefore set up nothing and asserted the opposite of what they meant, green locally and red in CI. `hide_from_path(&["gh"])` mirrors each PATH directory minus that one binary, which is the helper that actually answers the question these tests ask. Each of the three now also asserts its own precondition (`find_in_path` is None) before running the verifier, so a fixture that silently fails to hide a tool fails there rather than passing a conclusion it never established.
CodeQL follow-up: I was wrong first time, there WAS a real bug, and it is fixed. The remaining alerts are name-driven.Two separate things were tangled together here. Splitting them. 1. There was a real exposure, and my first analysis missed itMy earlier comment checked the sinks CodeQL named — assertion strings in the test module — concluded no credential reached them, and stopped. That was the wrong place to stop. A
Fixed in
Thank you for pushing back on the first read. 2. The 17 remaining alerts are not about the data, and redaction did not move themThe count went 12 → 17 across the redaction commit, which by itself shows the alerts do not track the payload. The reported sources say why:
This is the "scanner fundamentally cannot model it" determination, with evidence. The three ways to clear these:
Over to you, as agreed — I am not making this call unilaterally. Options: (a) accept the class as it already stands on Required checks
Two CI-only failures were found and fixed along the way, both the "green locally, red on the runner" class this repo's
|
…t that scores them (#56) Registers the six ids p4gs/sscs-bootstrapper#71 adds and, more importantly, teaches the scorer that phase 6 exists. **The load-bearing change is `scoring.ts`'s PHASES, and getting it wrong would have failed silently.** `computeScore` reduces `totalPass` and `totalCountable` over PHASES, but derives `coverage` from `scoped.length`, which has no phase filter. Registering six phase-6 controls while PHASES still read 1..5 would therefore not merely have omitted a row: every phase-6 pass would have counted for nothing, a phase-6 FAIL would have been invisible in the overall percentage, and the six controls would still have inflated the coverage denominator — so every listing's published coverage would have dropped, for a reason nothing reported. `reclassify()` fail-closes on an unknown control id; there is no equivalent guard on phase, which is exactly why this is worth spelling out in the constant's own doc comment. The tool found the same class independently on its side (`machine.rs`'s `score_of`, which that PR fixes by deriving from the registry). One bug, two codebases, both silent. Classes, and the reasoning that is not obvious: - `maintainer-mfa` is **C**, not B. It reads the MAINTAINER's own accounts (`gh api user`, `npm profile get`) with the maintainer's own credentials; no repository scan can see it, which is the definition of class C. - `publish-provenance` is **B** — the one control here that asks a third-party registry a question about published artifacts. - `publish-targets` maps to the posture-disclosure sentinel `[]`. It is an inventory of where a project publishes; it blocks no technique, and saying so is more honest than inventing an attack class for it. - The rest are A or A′ by whether they read committed manifests or committed workflows. A2 (stolen publisher identity) is the class this whole phase exists for, so four of the six carry it. Also replaced a literal `phase <= 5` assertion in `local.test.ts` with membership in PHASES, so the next phase cannot pass tests while being silently unscored — the failure mode this commit exists to prevent. Counts move 48 → 54 across both assertions and nine prose sites; the fixture is regenerated, never hand-edited. Verified: tsc clean, build clean, `bun test` 450 pass / 0 fail including the cross-repo parity case run against the tool branch.
DO NOT MERGE YET
The sibling site repo must register these six control ids — with the classes in the table below — before this lands, or every directory scan fails closed. That ordering is being handled separately.
sscsb stopped where the artifact left. Phases 1–5 harden the repository, the dependencies it consumes, the CI that builds it and the files a GitHub Release carries; none of them covers the moment that artifact goes to crates.io, npm, PyPI, Homebrew, Chocolatey or WinGet, and none covers the credential that lets it. That is the Shai-Hulud / chalk-debug shape exactly: phish a maintainer, take a long-lived publish token, ship malware under a trusted name. No commit is involved, so every control we already had was looking the other way.
48 controls → 54. (Branched one commit before
socket-firewall-cilanded;mainhas since been merged in, so the counts here are against currentmain.)publish-targetstrusted-publishingreleaseenvironment; no stored registry token.maintainer-mfatfa.mode, and a dated phishing-resistance claim.publish-tokenspublish-provenancedist-manifestssha256, Chocolateychecksum+checksumType, WinGetInstallerSha256.All six ship on, and phase 6 adds no off-by-default control, because the gate is detection rather than configuration. Every verifier goes quiet when its target is not in the tree, and degrades — naming the
degraded_reason— rather than failing when a tool or credential is simply absent.Design decisions worth reviewing
Publish templates live in their own
DIST_ARTIFACTStable, notworkflows::ARTIFACTS. That table installs on control-enabled alone andtrusted-publishingis on by default, so registering them there would have droppedpublish-npm.ymlinto every repository sscsb ever touched. A publish workflow you do not publish with is not a harmless extra file; it is aworkflow_dispatchbutton wired to somebody else's namespace. A test asserts the separation rather than only commenting it.The environments read follows
verify_branch_protectionliterally. GitHub answers 404 for both "no such environment" and "your token cannot see environments" — opposite verdicts — so the ambiguous case degrades and says which two things it might be. Telling a maintainer to create an environment that already exists is how a tool teaches people to stop reading it.npm
tfa.mode = auth-onlyFAILS; a missing GitHubtwo_factor_authenticationfield DEGRADES.auth-onlygates login but not the publish itself, which is the precise gap a stolen session walks through. GitHub omits the field for a token withoutread:user, and reading that asfalsewould accuse a maintainer holding a passkey of having no second factor. That read is also gated on a resolvable GitHub remote: without one, the accountghis logged in as is whoever owns the laptop, not whoever owns the package.Attestations document, never upgrade.
[[token]] expiresand[signing] expiresgo throughsigners.rs's ownevaluate_expiry; an expired claim fails.[[account]] attesteddeliberately does not — it counts backward from the day a human last checked, where a bigger gap is worse, and feeding it to an expiry evaluator would type-check while grading a claim dated tomorrow as comfortably fresh. Both directions are pinned by a test.Six phase bounds were hard-coded to 5 — one would have mis-scored every repo
controls.rs(×2),cli.rs,compliance.rsandtests/skill_docs.rseach iterated1..=5, so phase-6 controls would not have appeared instatus,report, or the registry's sparseness guard.The one that mattered is
machine.rs'sscore_of, which builtscore.phases[]from1u8..=5. A phase-6 row would have fallen out of both the numerator and the denominator while still counting towardscoped— so adding six passing controls would have lowered every repository's evidence-coverage percentage in the directory record. NowPHASE_COUNT, derived from the registry, because the number is a wire contract with the site.config.rshad the same latent bug in prose:_ => "Phase 5 — Observability & governance"is correct exactly until a sixth phase exists, then prints a wrong heading in the one file every user edits. Phases 5 and 6 get explicit arms, the fallback is a visible bug marker, and a test asserts no control renders it.Two defects found by dogfooding, not by reading code
publish = falsewas ignored.fuzz/Cargo.tomlcarries it, and detection called it a crates.io target — a crate that cannot be published even deliberately.publish = falseandpublish = []are now honoured;publish = ["registry"]still counts.publish = falsetoo, so sscsb is not on crates.io at all — while the README's first install line isbrew install p4gs/p4gs/sscsb, from a tap in another repository that nothing in this tree can prove. Without[targets] homebrew = "on", sscsb would have reported that its own project publishes nowhere. That declaration is committed here with its reasoning, as the worked example of why the override exists — and the checksum controls now say the manifest lives elsewhere instead of degrading over a file that was never supposed to be here.sscsb dist, and nosscsb publishdist statusanddist check [--strict]land. A publish wrapper was considered and rejected;docs/phase-6.mdrecords all three reasons so nobody re-proposes it — git hooks fire on git events and nothing interceptsnpm publish; the doctrine here is to move publishing into CI behind an OIDC identity; and reimplementing six registries' publish clients buys no security property.Verification
Local suite (
--no-fail-fast, hermetic invocation per CLAUDE.md), against amainbaseline taken the same way:main@8cad3f0)main)The 33 are identical in both runs and all environmental to this agent sandbox: DNS is blocked (so
deps baselinecannot reach crates.io, which fails the sharedrust_repo()fixture and takes all 15tool_orchestrationtests with it),opengrepis on PATH butopengrep --versionfails, and binding a local socket is denied (dtrack).cargo fmt --checkandcargo clippy --all-targets --locked -- -D warningsare both clean.Dogfood verdict on this repo — 0 failed, 1 degraded, exit 0:
That degrade is the designed path, exercised live against the real GitHub API rather than a fixture.
Known-stale, deliberately untouched
docs/example-walkthrough.mdstill says 43 controls — it was already stale by four onmain, nothing tests it, and re-capturing it honestly needs a terminal with network and every scanner present, which this environment does not have. Fabricating updated numbers would be worse than a disclosed pre-existing drift. Worth its own pass.