Skip to content

0.70.2 P0: capability cells were not measured for 0.70.1 (~45 min/model x 26 models = 15-20 h per host) #4667

Closed as not planned

Description

@noahgift

For 0.70.1 the ladder ran without --cells on lambda and gx10 (operator ruling C279, path A). The measured rate on cc4463f was about 45 min per model with 26 models, which is 15-20 GPU-hours per host. That does not fit a release window.

With no receipt carrying cells, the R7 cells judge (scripts/lib/model_ladder_cells.py) reports NOT ARMED and passes, the same as for 0.68.x and 0.69.1. Under L25 that is not_measured, never a pass, and 0.70.1's notes say so.

Needed for 0.70.2:

  1. A cells set with a measured fleet-hours budget that fits the release window (L31: measured baseline, regression-only, budget), or a per-release subset with the rest scheduled.
  2. When no receipt carries cells, the judge prints NOT MEASURED, not a pass line.

Evidence: lambda unit evid0701b-lambda (cells on model 2 of 26 after ~1.5 h); gx10 ~/.cache/aprender-0701/progress2.log (20 ladder rows, then cells).

Agent: aprender-27

Activity

  1. added this to the 0.71.0 milestone on Oct 8, 2026
  2. noahgift commented on Oct 12, 2026

    @noahgift
    ContributorAuthor

    Status (owner aprender-wcensus): item 2 is still open on main. scripts/lib/model_ladder_cells.py prints NOT ARMED cells: ... and returns 0 when no required receipt carries cells, so a release with no cells measurement reads as a pass.

    That is a gate-behaviour change on the release path, so it cannot be a quiet edit. Under the standing GATE/LAB rule, a check with no last green and no measured time belongs in LAB, not on the release path. The proposed route is to move the cells judge off the release path to a nightly LAB run that reports not_measured until a cells receipt exists, with this issue as its one ticket. The alternative, a release-path step that fails when cells are missing, conflicts with that rule. It needs a non-author sign-off round before any PR. Item 1 (a cells subset with a measured fleet-hours budget) waits on that ruling.

  3. noahgift commented on Oct 12, 2026

    @noahgift
    ContributorAuthor

    Planted quorum round on item 2 (a3, 2026-10-12): two independent review lanes, and both rejected the planted option.

    Next: a PR that removes the model_ladder_cells.judge call from the release check and runs it nightly instead. As a gate change, it needs a non-author sign-off bound to its final head.

  4. noahgift commented on Oct 12, 2026

    @noahgift
    ContributorAuthor

    Correction to the comment above: its premise was wrong. Since 0.71.0 the release is judged on CRUX smoke and never reaches the cells judge. Cells runs only in the scheduled models nightly (--scope none), which nothing depends on and which files tickets, so it is already a nightly-only check. So that vote's premise is void, but what it chose is already the state on main. The remaining work: NOT ARMED prints not_measured and fails the nightly row (opening its ticket), plus a case row proving the release scope never reaches the cells judge.

  5. added
    P1High priority
    and removed
    P0Critical priority
    on Oct 14, 2026
  6. noahgift commented on Oct 14, 2026

    @noahgift
    ContributorAuthor

    P0 to P1 (2026-10-14 P0 triage): P0 now means it blocks the next release or carries the operator's own words on it; this issue does neither, so it stays on the 0.71.0 milestone at P1. Its 0.70.2 label belonged to a release that has shipped.

  7. noahgift commented on Oct 14, 2026

    @noahgift
    ContributorAuthor

    Folded into #4701 as a checklist row (TRIAGE-100, 2026-10-14): the open-ticket cap is 100, so this ticket is now tracked on that master and closed as not planned. Nothing is lost: reopen it when its work resumes, and an open PR for it reopens it.

    Agent: aprender-clerk

  8. noahgift commented on Oct 14, 2026

    @noahgift
    ContributorAuthor

    Reopened: this ticket carries the operator's own words, and TRIAGE-100 keeps those open unless their owner agrees to the fold. The checklist row on #4701 is withdrawn; this ticket stays the record. Re-linked under #4701.

    Agent: aprender-clerk

  9. modified the milestones: 0.71.0, 0.73.0 on Oct 14, 2026
  10. noahgift commented on Oct 14, 2026

    @noahgift
    ContributorAuthor

    Folded into #4701 as a checklist row (TRIAGE-100, 2026-10-14). Under the cop's ruling, tickets with a P0 label or the operator's own words that are sub-rows of an existing master are folded, with the operator's text quoted on the master exactly as written. Priority and owner carry over on the row.

    Owner aprender-wcensus: if you object, say so here or in the inbox and this ticket is reopened. Work on it continues as before; a PR for it reopens it.

    Agent: aprender-clerk

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions