Skip to content

check_silicon_coverage: a failed runner listing (403 body on stdout) counts as 1 online runner, so probe 2a never fires on the PR path #5213

Closed as not planned

Description

@noahgift

What

scripts/check_silicon_coverage.sh counts a failed runner listing as one online runner. As a result, its zero-runner NO-GO (probe 2a) cannot fire whenever the job token cannot read runners, which is every PR and merge-queue run.

Mechanism (measured)

  • On an HTTP error, gh api writes the error JSON body to stdout with no trailing newline.
  • The runner loop appends stdout with >> "$RUNNERS", so both 403 bodies (Resource not accessible by integration, from the org listing and from the repo listing) end up on one non-empty line.
  • n_runners counts non-empty lines, so it reads 1. The guard prints online runners visible (org + this repo): 1, and the -eq 0 NO-GO is skipped.
  • The job token never has the scope to list runners, so this happens on every guard-tree run of the check. The guard tolerates the 403 (rc 0); only the count is wrong.

Effect on the verdict

The runner list feeds only two things:

  • probe 2a, the zero-runner NO-GO;
  • the label UNCOVERED vs MISSING on an axis that already fails either way.

The PASS/NO-GO on axes comes from job evidence, which the token reads in full. So today probe 2a passes vacuously on the PR and merge-queue path.

Why it is not in #5212

Counting honestly makes probe 2a fire on every PR, because the token cannot list runners. Which way to resolve that is a separate decision, made by a quorum round. #5212 (5xx retry) does not touch the runner loop.

Refs #5212

Activity

  1. added this to the 0.71.1 milestone on Oct 12, 2026
  2. modified the milestones: 0.71.1, backlog on Oct 14, 2026
  3. noahgift commented on Oct 14, 2026

    @noahgift
    ContributorAuthor

    Evidence, 2026-10-14 (no code change; this comment does not close the issue).

    Fixed on main in two steps:

    Self-test rows m2–m5 in scripts/check_silicon_coverage.sh pin the cases: 403 on both listings, pull_request, merge_group, and a missing axis still failing.

    Proposed: close as fixed. Closing is the triage owner's call, not this lane's.

    Agent: aprender-a7

  4. noahgift commented on Oct 14, 2026

    @noahgift
    ContributorAuthor

    Folded into #2876 as a checklist row (TRIAGE-100, 2026-10-14): the open-ticket cap is 100, so this ticket is now tracked on that master and closed as not planned. Nothing is lost: reopen it when its work resumes, and an open PR for it reopens it.

    Agent: aprender-clerk

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions