Skip to content

coverage-nightly has never completed on the yoga pool (4/4 red since 2026-09-09): each night a different unit test asserts the box, so the 95 % ratchet has no baseline #3185

Description

@noahgift

What happened

coverage-nightly.yml (#3123: low-priority coverage run on yoga toward the 95 % target) concluded failure on every scheduled run since it moved to the yoga pool — 34325094997 (09-09), 34451014241 (09-10), 34575134766 (09-11), 34680781819 (09-12) — and on the 0.67.0 cut when dispatched by hand (34702490406, yoga-build2, 2026-09-12T15:31Z). Every time make coverage (Makefile:586) exits 2 because ONE unit test fails under llvm-cov, and the test is different each time:

So the coverage number for 0.67.0 does not exist, and §5 P3 of APR-RELEASE-001 ("target = measured baseline + 2 points per ratchet toward 95 %") cannot start.

Why

Two classes, both known here: a test that asserts the hardware it runs on (#3129, #3135, #3184 are the same shape on the GPU jobs), and a wall-clock/throughput assertion in a unit test (the rule in memory and in #3131, which already moved one such benchmark behind bench-gates).

Ask

  1. benchmark.rs:522 — move the ratio assertion behind the bench-gates feature (as fix(tests): eight spec_checklist targets read repo paths from the crate dir — 77 vacuous rows now read the tree (#3130) #3131 did), or assert correctness (round-trip bytes) and record the throughput.
  2. backward.rs:1103/1120 — skip with a named reason when no adapter is present, counted as skipped (the fix(compute): nine GPU tests panicked without an adapter — the nightly coverage run was RED six days on an environment fact (PMAT-1106) #3116 treatment).
  3. make coverage — run the suite --no-fail-fast so a single box-assertion does not zero the night's report, and print the coverage number even when a test fails (report-only is the job's declared contract).
  4. Then record the first baseline in docs/build-ledger/ per APR-RELEASE-001 §5 P3.

Refs #3123, #3116, #3131, #2307, PMAT-1098 (0.67.0 train).

Activity

  1. added this to the 0.68.0 milestone on Sep 12, 2026
  2. added
    bugSomething isn't working
    P2Medium priority
    on Sep 12, 2026
  3. noahgift commented on Sep 15, 2026

    @noahgift
    MemberAuthor

    Reopening: this was closed COMPLETED at 2026-09-14T03:01:12Z, and Coverage Nightly
    failed again 5 h 17 m later on the test this issue's own body already names.

    Run 34822086599, created
    2026-09-14T08:18:52Z, job coverage:

    test benchmark::tests::test_decompression_faster_than_compression ... FAILED
    thread '…' panicked at crates/aprender-zram-core/src/benchmark.rs:522:9
    test result: FAILED. 403 passed; 1 failed; 2 ignored; 0 measured; 2 filtered out
    error: test failed, to rerun pass `-p aprender-zram-core --lib`
    

    403 passed, 1 failed. One wall-clock ratio assertion is taking down the whole
    coverage measurement.

    What the closure got right: the GPU-adapter half (test_silu_backward_at_zero and
    …_length_mismatch, the #3116 class) is genuinely fixed. What it missed is that this
    issue documented two independent causes, and only one of them was addressed — the
    body already recorded test_decompression_faster_than_compression as what failed on
    the 0.67.0 cut once #3116 landed. That is the instance still live.

    Consequence while this is open: make coverage exits 2, the nightly reports
    "Line coverage: unknown", and COV_FLOOR has had no fresh measurement. The last
    real number is 88.78% from 2026-07-29 (commit 95145584f) — 48 days old.

    Fix in flight: #3290 (PMAT-3237), which reports rather than asserts the ratio
    under #[cfg(coverage)] and keeps the 0.25 assertion under #[cfg(not(coverage))].
    That PR's body currently says "a red-nightly fix with no filed issue" — that is wrong,
    and it is this issue; I am correcting the reference.

    Falsifier for the close: Coverage Nightly concludes success and publishes a line
    coverage figure. Until a run does that, the lane is unmeasured, not fixed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    P2Medium prioritybugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions