Skip to content

pr-review gate has no self-service path for fork/external contributors without pmat/agy/minisign #4462

Closed as not planned

Description

@alfredodeza

What

The pr-review quorum gate (Arm 4 / pr-review-quorum / present, required on every
PR via pull_request_target) has no self-service path for a contributor who lacks
pmat/agy/minisign locally — which in practice means every external, fork-based
contributor
. Today the only way through is a maintainer manually running the full
review protocol on the contributor's behalf, which does not scale and defeats the
purpose of taking outside contributions at all.

How I confirmed this (not a guess)

  • pr-review-quorum.yml's present job (Arm 4, scripts/check_pr_review_arm4.sh) is
    pull_request_target + merge_group, so it is REQUIRED on every PR regardless of
    fork origin.
  • The only thing that can produce a signed receipt, pr-review-sign
    (ci/sections.yml), carries: if: github.event.pull_request.head.repo.full_name == github.repository. Its own header comment states the reasoning: "Forks get no
    secret, so it skips them rather than failing them."
    That is correct GitHub security
    practice (a fork PR must never see a repo secret) — but it means the signer
    unconditionally does not run for fork PRs. There is no other job, label trigger, or
    code path in ci.yml / ci/sections.yml / contracts/pr-review-skill-v2.yaml that
    produces a receipt for a fork PR.
  • check_pr_review_arm4.sh's only accommodation for an unreachable review path is
    PR_REVIEW_CUTOFF (a one-time historical PR-number grandfather from the day the gate
    was armed) and a 24-hour PR_REVIEW_LEGACY_* exemption tied to the P0: pr-review receipt binds by diff patch-id --verbatim, fails closed on the merge-queue sha #4421 patch-id
    migration. Neither is a repeatable path for a contributor showing up today or in the
    future.
  • I grepped scripts/check_pr_review_*.sh, scripts/pr_review_sign_receipt.sh,
    contracts/pr-review-skill-v2.yaml and .claude/skills/pr-review/ for fork,
    external contributor, human reviewer, maintainer — no accommodation exists
    beyond the two exemptions above.

Impact

A contributor who opens a fork PR with working code and tests, but has no pmat /
agy / minisign installed (i.e. essentially anyone outside this project's own
tooled dev environment), cannot produce an accepted receipt themselves. Arm 4 stays
RED forever on their PR unless a maintainer:

  1. checks out their branch locally (or asks them to enable "Allow edits from
    maintainers"),
  2. runs the entire pr-review skill by hand, including the agy cross-vendor step,
  3. pushes the resulting receipt back onto their branch (only possible if maintainer
    edits are enabled — many contributors leave this off by default).

That is real, unscalable, per-PR manual labor, and it is a hidden precondition to
accepting any outside contribution at all — nothing in CONTRIBUTING-type docs
currently states it.

Proposed solution: a maintainer-attestation label that CI acts on, doing 100% of the mechanical work itself

The goal: a maintainer's only action is clicking "add label" on GitHub. No local
scripts, no checkout, no agy/pmat/minisign on the maintainer's machine either.

  1. New label, e.g. pr-review:attest. GitHub already restricts who can add labels
    to accounts with triage/write permission on the repo — so the label itself is
    the authorization signal; no separate allowlist needs to be maintained. (Still worth
    double-checking github.event.sender.login's permission server-side rather than
    trusting the label's mere presence, in case of race conditions with permission
    changes.)

  2. New job, triggered on pull_request_target types: [labeled] (the same safe
    pattern pr-review-quorum.yml already uses and documents: runs with the BASE
    branch's workflow definition and secrets, and — this is the part to preserve
    carefully — never executes the fork's code, only reads its diff/files as text
    via git archive <fork-head-sha>).

  3. When the label fires from an authorized sender, this job does the full review
    server-side
    , on CI's own fleet, exactly like this repo's own PRs get today:

    • pmat consultation (build/confirm index, duplication scan) — static analysis
      over text, no execution of contributor code, same as today.
    • The §3.B/§3.C/§3.D trigger checks via the existing guard's --match-*
      predicates.
    • The §3.E agy cross-vendor consultation, run the same way it runs today
      (disposable git archive copy, agy reads files as text — it does not execute
      the contributor's scripts either).
    • Assembles and signs the receipt with the existing escrowed key (available
      here, since pull_request_target runs with the base repo's secrets — the same
      reason pr-review-sign exists as a separate, base-defined job today).
  4. Where the signed receipt lives. This is the one real design decision, because
    Arm 4 today expects evidence/pr-review/<pr>/ to exist inside the PR's own tree at
    the reviewed commit — which CI cannot write into a fork it has no push access to.
    Proposal: extend check_pr_review_arm4.sh's receipt lookup so that, for a PR whose
    head.repo.full_name != github.repository, it also looks for a receipt committed
    by this bot job into a base-repo-owned location keyed by (pr_number, diff_patch_id) — e.g. a dedicated orphan branch pr-review-fork-receipts or a
    evidence/pr-review-fork/<pr>/<diff_patch_id>/ path on main — rather than
    requiring it inside the contributor's own branch. The diff-patch-id binding (P0: pr-review receipt binds by diff patch-id --verbatim, fails closed on the merge-queue sha #4421)
    already exists precisely so a receipt can be verified against a diff without
    depending on commit ancestry; storing it out-of-band is a natural extension of that,
    not a new trust mechanism.

  5. What this preserves. The actual security boundary — contributor code is never
    executed with repo secrets present — is unchanged; this only widens where the
    receipt can be filed from
    , not what is trusted to run with what access. The human
    trust boundary shifts from "did a maintainer manually run the tooling" to "did a
    maintainer (with real GitHub write/triage permission) vouch that this PR should be
    auto-reviewed" — which is a much lighter, one-click, auditable action (the label
    application itself is a permanent, visible record of who attested and when).

Non-goals / what I'm not proposing

  • Not proposing contributors get a weaker review than today's PRs — the same
    consultations run, just server-side instead of on a maintainer's laptop.
    • Not proposing to relax pr-review-sign's fork exclusion for secrets — that stays;
      this adds a different job that never needs the fork to have the secret because the
      base branch's context already has it.
  • Not proposing an implementation in this issue — this needs a security-conscious
    design review (particularly step 4, the receipt storage location and Arm 4's lookup
    change) before anyone writes the workflow YAML.

Activity

  1. added this to the 0.70.1 milestone on Sep 26, 2026
  2. modified the milestones: 0.70.1, 0.71.0 on Oct 4, 2026
  3. added
    P1High priority
    and removed
    P0Critical priority
    on Oct 14, 2026
  4. noahgift commented on Oct 14, 2026

    @noahgift
    Contributor

    P0 to P1 (2026-10-14 P0 triage): P0 now means it blocks the next release or carries the operator's own words on it; this issue does neither, so it stays on the 0.71.0 milestone at P1.

  5. noahgift commented on Oct 14, 2026

    @noahgift
    Contributor

    Folded into #4741 as a checklist row (TRIAGE-100, 2026-10-14): the open-ticket cap is 100, so this ticket is now tracked on that master and closed as not planned. Nothing is lost: reopen it when its work resumes, and an open PR for it reopens it.

    Agent: aprender-clerk

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions