Skip to content

arm_a_scaling emits nothing on a c=1-only receipt and the gate still returns VERDICT PASS #2830

Closed

Description

@noahgift

arm_a_scaling in scripts/perf_gate.sh emits zero lines on a receipt whose only band is c=1, and the gate still returns VERDICT PASS.

The mechanism

for c in sorted(bands):
    if c == 1:
        continue
    ...

c=1 is the denominator of scaling_efficiency(c) = (agg(c)/agg(1))/c, so skipping it is correct. But with only c=1 present the loop body never executes, the arm prints nothing, and nothing downstream notices its silence.

Observed in a W1 pre-flight on lambda, --phase merge, real receipt:

PASS   ArmC integrity
SKIP   ArmC-sig  (release-phase rule)
REPORT ArmB c=1 UNMEASURED (Arm A still gates this cell)
REPORT ArmD instrumentation absent: kv.*
SKIP   ArmE (W1)
VERDICT PASS host=lambda phase=merge workload=W1     rc=0

Arm A — the primary release-blocking arm, and the one the matrix calls "comparator-free, so it is the primary gate" — contributed nothing to that PASS.

The opposite polarity IS armed, which is what makes this a gap rather than a design

A c=16-only receipt gives FAIL ArmA band c=1 absent. So:

receipt Arm A
c=16 only FAIL — loudly
c=1 only silent, gate PASSes

Arm A fails when its denominator is missing and stands down when everything but the denominator is missing. That asymmetry is the defect.

Why it has not bitten yet

cell_completeness catches a short band set — but it runs only at --phase release. At --phase merge, which is what a PR runs, nothing does. So a merge-phase PASS on a one-band receipt is not evidence Arm A passed, and there is currently no way to tell the two apart from the gate's output.

Suggested shape

Arm A should be unable to return without saying something. Either:

  1. it emits an explicit SKIP ArmA (no band > 1 in this receipt) line, so silence becomes impossible; or
  2. it FAILs a receipt that declares bands containing only c=1, on the grounds that such a receipt cannot exercise the arm it is nominally gated by.

(1) is the smaller change and preserves the merge-phase REPORT semantics. (2) is stronger but may collide with legitimate single-band diagnostic runs.

Whichever ships needs a fixture in --selftest for the c=1-only case specifically. The existing selftest is 43 passed / 0 broken and covers Arm A's REPORT path via expiry_w1_fixed_date_still_reports — but no row exercises the empty-loop case, which is exactly why a silent arm survived a green selftest.

Context

This is the seventh instance today of one defect class: a check that cannot report its own failure. The others were an EPIPE-inverted grep (105 sites, #2804), a chown swallowing its errors, a job timeout naming no step (#2811), five make targets advertising CI that no workflow invokes (#2808), a sed draining the stream its sibling needed, and a self-test satisfied by its own vacuity floor (#2829).

Refs APR-PERF-GATE-001 (#2706)

Activity

  1. noahgift commented on Sep 5, 2026

    @noahgift
    ContributorAuthor

    Blocks PP-066 C0 (spec §4). Also: F-28, master row 0a.

    Named in #2873's body as a blocker → C0, and named in C0's own check: 'the #2830 ticket's perf_gate.sh --selftest row (c=1-only receipt ⇒ not PASS) is GREEN at HEAD and was RED on the mutation in its PR body'. arm_a_scaling skips c=1 and prints nothing on a c=1-only receipt while the gate returns VERDICT PASS; the opposite polarity (c=16-only) fails loudly. PRs #2831/#2833 checked 'Arm A speaks' by hand in the gate output rather than by a rule.

    Instance of class #2876 — gate answers empty/degenerate input with a PASS or a crash instead of a named refusal.

    PP-066 triage 2026-09-05 (issue #2873); ledger docs/audits/pp-066-triage-2026-09-05.md.

  2. added this to the 0.68.0 milestone on Sep 10, 2026
  3. added
    automationBot-maintained dashboard, not a defect — skip in triage
    P1High priority
    on Sep 11, 2026
  4. modified the milestones: 0.68.0, 0.69.0, 0.72.0 on Sep 16, 2026
  5. noahgift commented on Sep 24, 2026

    @noahgift
    ContributorAuthor

    Closed by paiml-implement — fixed by #3007 (merged to main) (triage evidence/triage-1418.md)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    P1High priorityautomationBot-maintained dashboard, not a defect — skip in triage

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions