Repository navigation
EPIC E1 0.71: Fast Train #3998
Description
owner: aprender-wmainred (re-owned 2026-10-09 by clerk aprender-3e for cop aprender-13: works children #4687 #4715 #4960 (most child work on 2026-10-09))
Spec: APR-EPIC-001 §2 · Epic: E1 · Train: 0.71 · Successor at cut: S1
Theme
Fast Train
done_when
Operator ruling C334 (2026-10-08 10:45Z), train 0.71:
- C332 a, rehearsal. Every night on main's head, the release-day script and policy file run everything except the tag push and the uploads. Green = exit 0, every lane measured on that night's commit, no rerun. The 3 nights before release day are green; a red night resets the count. Receipt required.
- C332 b, the script releases. One unattended pass of
scripts/release/autopilot.shfrom the bump to live: 0 steps typed outside it, 0 emergency scopes, 0 commits on the release branch after the bump, 0 gate-script edits on release day. Receipt required. - C332 c, receipts. Two planted tests pass inside the rehearsal: a GitHub read that fails twice and then answers costs under 2 min and no lane rerun; a notes-only commit is ready to tag in under 10 min on saved results. Receipt required.
- C333: the checks that can stop a release number 20 or fewer (merge 10, tag 5, publish 5), listed in code.
- Freeze to publish in 4 h or less.
Every build ticket goes to E1 (C334 item 1).
done_when before C334 (kept, not deleted)
freeze→publish ≤ 4 h [O]; vX.Y.Z-rc.N on every green merge to release/*; rc→final assets sha256-equal; nightly publishes every [[bin]] plus a CUDA apr; every fleet host runs the newest verified build, or andon.
Budget
At most 10 open children (docs/roadmaps/epics.yaml). Repo-wide cap 100 open issues; andon at 95.
Build-bar rows (C334 item 3)
Filed as checklist rows, not issues: the ticket tree is over 100 (ISSUE-TREE-001).
- C332-a: nightly release rehearsal on main's head. Same script and policy file as release day; everything except the tag push and the uploads; green = exit 0, every lane measured on that night's commit, no rerun; 3 green nights before release day, and a red night resets the count (evidence: nightly rehearsal receipts; related PR feat(release): nightly assets rehearsal — build every release asset on a schedule, unable to upload (#4806) #4813)
- C332-c: receipts with reads first. Planted tests inside the rehearsal: a GitHub read that fails twice and then answers costs under 2 min and no lane rerun; a notes-only commit is ready to tag in under 10 min on saved results. Includes C330 item 6: reads run first, the 10-min re-ask applies to reads only, and the coverage lookup by commit takes the newest run that holds a receipt (evidence: planted-test receipts)
From #4678 (C334 item 2)
#4678's 37 open children, ranked by APR-EPIC-001 rule 8. Plan receipt, with the quorum record: docs/audits/apr-epic-001/plan-C334-E1.json; write log: applied-C334-E1.json.
- Kept, re-parented here (14): ranks 1-10 GEN-001: README/doc counts generated by one writer; PRs never hand-edit them (root cause of generated-file conflicts) #4526 0.70.2 T2: BLD-002 R2 land the evidence store keyed by H #4680 0.70.2 T6: BLD-002 R12 release-path Python inventory guard, shadow first, then admit it (= R12-E) #4684 0.70.2 T7: docs: BLD-001 plan doc + the 0.70.2 ledger docs/lookahead/0.70.2.md #4685 0.70.2 T10: BLD-002 R5b one door to cargo publish: publish_strict.sh refuses a missing, red or other-commit clean-room run #4687 BLD-002 R9: one list in code of what merge, tag and publish require #4688 0.70.2 T14: BLD-002 R8 known-failure ratchet against the last release (judges #4661-#4666) #4689 0.70.2 T18: RQ-2 the full model ladder runs nightly on main's head; release day reads it for the same H #4701 INFRA 0.70.2 RQ-6: forjar/make target applies one GitHub ruleset to the evidence-store branch (#4680) #4704 CENSUS-RATCHET: entity-bound contracts never drop and rise every release #4709; plus 0.70.2 T24: count the Python the gate path runs, at run time, not how scripts spell it (R12 replacement) #4742 0.70.2 T36: a push to main re-runs CI on the sha the merge queue just passed #4743 0.70.2 T46: PR CI's quick tier runs 16,972 tests on shard 1 alone #4751 (rule 6: P0 on 0.71) and Qwen3.5-0.8B thinking-ON leg: Qwen sampling over N seeds + a required closure rate instead of greedy (GATE CHANGE) #4696 (open P0 sub 0.70.2 P0: qwen35-0.8b-q4km golden_output thinking_on: think block unclosed within 2048 tokens (known failure in 0.70.1) #4666). That is 4 over the "top 10": E1 has 25 open children against a budget of 10.
- r2 (C334 quorum r2, 12:11Z): a non-Claude lane objected and a 2/2 cross-family tie-break (gemini + sonnet) ruled EPIC_TRAIN: a child re-parented here is on 0.71 (rule 3). So the P0s B2: root-cause fixes for red nightlies on main #4715, ARB-AUD-10: fleet_cells_gate.sh lands on main without Python, after ARB-AUD-01 #4350 (under ARB-AUD-10b: python3 out of the release path — 12 scripts / 43 sites + release_wall_time.py, jq with a parity table per script #4352) and Adopt dogfood-receipt-v1 in apr-dogfood, and remove python3 from it (ARB-SELF-001 ASL-05) #4377 (under BLD-002 R12-D: Python off dogfood and the perf, bench and parity receipts #4695) are protected (rules 6 and 20), and their parents stay open (ISSUE-TREE-001). Reopened and re-parented: B2: root-cause fixes for red nightlies on main #4715 ARB-AUD-10b: python3 out of the release path — 12 scripts / 43 sites + release_wall_time.py, jq with a parity table per script #4352 BLD-002 R12-D: Python off dogfood and the perf, bench and parity receipts #4695; ARB-AUD-10: fleet_cells_gate.sh lands on main without Python, after ARB-AUD-01 #4350 and Adopt dogfood-receipt-v1 in apr-dogfood, and remove python3 from it (ARB-SELF-001 ASL-05) #4377 stay nested. Now: kept 17 direct + 2 nested, collapsed 4, parked 16. E1 has 28 open children against a budget of 10. Receipt: cop-inbox handoff quorum-c334-e1-r2.json; plan-C334-E1.json r2.
- Collapsed (6, rule 20): Adopt dogfood-receipt-v1 in apr-dogfood, and remove python3 from it (ARB-SELF-001 ASL-05) #4377 BLD-002 R12-A: Python off the PR-CI path (CI driver, ratchets, roadmap/DAG tools, uv install) #4693 BLD-002 R12-B: Python off the model ladder and CRUX path #4694 BLD-002 R12-D: Python off dogfood and the perf, bench and parity receipts #4695 into 0.70.2 T6: BLD-002 R12 release-path Python inventory guard, shadow first, then admit it (= R12-E) #4684; 0.70.3: python run-time count: key what Python RAN, not how its argv is spelled (redesign of #4742) #4754 into 0.70.2 T24: count the Python the gate path runs, at run time, not how scripts spell it (R12 replacement) #4742; 0.70.2 B5: the tag-time path is publish-only (preflight + clean-room green on the bump sha before the tag) #4805 into 0.70.2 T10: BLD-002 R5b one door to cargo publish: publish_strict.sh refuses a missing, red or other-commit clean-room run #4687.
- Parked (19, rule 5): listed under Parked below (ARB-AUD-10: fleet_cells_gate.sh lands on main without Python, after ARB-AUD-01 #4350 and ARB-AUD-10b: python3 out of the release path — 12 scripts / 43 sites + release_wall_time.py, jq with a parity table per script #4352 were already there from an earlier park; their lines are annotated).
Absorbs
#4033, #4288, #4232, #3081, #3058
Prior body
Theme: Fast Train (operator decision 2026-09-23: "make .70 fast train")
A release goes from freeze to publish in hours, not a day, and every later release inherits that speed. 0.71's "Don't Leave Behind" (#3994) runs a 4-backend matrix and will need several fix-and-re-sweep cycles, so speed comes first.
Exit bar. Freeze to publish ≤ 4 h, measured on 0.70 itself. The lock's idle-GPU fraction falls by more than half, per paiml/infra#1146's method re-measured.
Must-carry
| Area | Items |
|---|---|
| GPU lock limited to GPU work | paiml/infra#1146 P1: build outside the lock, only CUDA tests under it; gpu-q refuses a bare cargo test/cargo build (via infra/forjar) |
| CPU work off the lock | paiml/infra#1146 P3: ladder CPU legs and hashing unlocked |
| Parallel hosts | paiml/infra#1146 P5: idle same-arch pre-screen as a standard pre-freeze step; the sweep shards across lambda, gx10, yoga and the Mac host |
| VRAM-budget sharing (pilot) | paiml/infra#1146 P4: correctness jobs only; benchmarks stay exclusive. Separate processes, not MPS (MPS lets one client's fault kill others) |
| Sweep hygiene | one hash pass per sweep; the serve probe keeps its own evidence (#3949); the serve health wait never counts lock-queue time (the gx10 q4k.apr false-RED shape, 2026-09-23) |
| Dark test targets into CI | #3984, #3982 |
| Fail-fast release gates | #3892 (publish cascade retry), #3974 (WARN/SKIP exit 0) |
| 0.69.1 known REDs (no defer) | #3951, #3987, low-bit admission #3963/#3953/#3960, subject to operator confirmation |
| Ratchet slice 1 of 5 | #3997 |
Out: new backends and silicon (0.71) and serve/agentic features.
Parked
Closed over budget (APR-EPIC-001 rule 5). Reopen = pull (rule 7).
- Retroactive quorum invalidation: receipts whose P3 quorum ran under paiml-implement < v1.2.0 (ticket text missing) — start with #2867 and #2868 #2881 Retroactive quorum invalidation: receipts whose P3 quorum ran under paiml-implem — over E1 budget (rank 26/93)
- CB-2100 cannot close via sha-pinning sovereign-ci.yml — needs 6 untracked debt rows or cross-repo manifest tooling (C0-2 finding) #3029 CB-2100 cannot close via sha-pinning sovereign-ci.yml — needs 6 untracked debt r — over E1 budget (rank 75/93)
- R-0b follow-up: a forced accelerator whose runtime attempt fails must refuse BEFORE generation in realizar (decompose try_wgpu_generate cog 49 / try_apr_wgpu_inference cog 69) #3042 R-0b follow-up: a forced accelerator whose runtime attempt fails must refuse BEF — over E1 budget (rank 76/93)
- KEY: keys/apr-release-minisign.pub on main — the release-asset signing key R-5 signs with and R-6 pins (R-5 blocked_by KEY) #3045 KEY: keys/apr-release-minisign.pub on main — the release-asset signing key R-5 s — over E1 budget (rank 77/93)
- aprender-gpu: driver, kernels, memory AND ptx are all #[cfg(feature = "cuda")] — 2,160+ tests incl. the GH-480 rewriter's and the PTX emitter's run ONLY in the gx10 nightly's perf053 filter #3067 aprender-gpu: driver, kernels, memory AND ptx are all #[cfg(feature = "cuda")] — — over E1 budget (rank 78/93)
- gpu-quick on gx10 (GB10 unified memory): test_alloc_oversize_100gb asserts discrete-GPU refusal — the only red in 2604 cuda tests #3129 gpu-quick on gx10 (GB10 unified memory): test_alloc_oversize_100gb asserts discr — over E1 budget (rank 33/93)
- CI artifact rows for Arbiter (ARBITER-001 R2a): junit + sccache-metrics from workspace-test, llvm-cov JSON from Coverage Nightly — and Coverage Nightly has been red since 2026-08-29 #3133 CI artifact rows for Arbiter (ARBITER-001 R2a): junit + sccache-metrics from wor — over E1 budget (rank 34/93)
- cuda-unit on yoga (RTX 4060): test_cublas_gemm_f16_training_shape asserts a 4090 floor — 'must exceed 50 TFLOP/s, got 23.6' is the only red in 2604 cuda tests #3135 cuda-unit on yoga (RTX 4060): test_cublas_gemm_f16_training_shape asserts a 4090 — over E1 budget (rank 79/93)
- PVL-F7: con-leche nightly independent re-check, advisory #3142 PVL-F7: con-leche nightly independent re-check, advisory — over E1 budget (rank 38/93)
- gx10 disk hygiene: the arm64 sovereign-ci image runs as uid 0, so its per-PR target caches are root-owned and never reclaimed — 128 GB today, 73 GB of it for closed PRs #3143 gx10 disk hygiene: the arm64 sovereign-ci image runs as uid 0, so its per-PR tar — over E1 budget (rank 35/93)
- .claude/ holds 58 stray worktrees, 25 GB and 571,815 .rs files inside the repo root — and the ignore rule is local-only #3171 .claude/ holds 58 stray worktrees, 25 GB and 571,815 .rs files inside the repo r — over E1 budget (rank 27/93)
- cuda-unit on yoga (RTX 4060 8 GB): cta64_vs_cta32_vs_cublas_fp16 dies with CUDA_ERROR_OUT_OF_MEMORY — the advisory job has been red on every PR today #3184 cuda-unit on yoga (RTX 4060 8 GB): cta64_vs_cta32_vs_cublas_fp16 dies with CUDA_ — over E1 budget (rank 36/93)
- ci_queue_steward's live path says it exists to pass the linter — and it does: no timer runs it anywhere, and that is where the DET002 that blocked 0.67.0 lived #3190 ci_queue_steward's live path says it exists to pass the linter — and it does: no — over E1 budget (rank 37/93)
- APR-RELEASE-001 §3.6: nothing in the repo writes the build ledger — 1091 records committed, no way to produce record 1092 #3206 APR-RELEASE-001 §3.6: nothing in the repo writes the build ledger — 1091 records — over E1 budget (rank 80/93)
- guard-cargo provisions a per-PR target dir and CARGO_HOME, then runs 20 cargo-building host guards without pointing cargo at either #3218 guard-cargo provisions a per-PR target dir and CARGO_HOME, then runs 20 cargo-bu — over E1 budget (rank 81/93)
- gpu-quick: the CUDA test binary SIGSEGVs at teardown on gx10-eph after every test passes (1 in 5 runs) #3226 gpu-quick: the CUDA test binary SIGSEGVs at teardown on gx10-eph after every tes — over E1 budget (rank 82/93)
- cuda-unit OOM-kills rustc on yoga: no CARGO_BUILD_JOBS cap on a 22-core / 30 GB box #3234 cuda-unit OOM-kills rustc on yoga: no CARGO_BUILD_JOBS cap on a 22-core / 30 GB — over E1 budget (rank 83/93)
- 1,424 of 1,469 integration targets run only when a PR touches their own crate — four of tonight's defects trace to this #3239 1,424 of 1,469 integration targets run only when a PR touches their own crate — — over E1 budget (rank 84/93)
- Fleet throughput is 2.1–2.2× intel alone, not 3×: a gx10/yoga runner is 2.5× an intel one but they supply only 7 of 21 clean-room runners #3251 Fleet throughput is 2.1–2.2× intel alone, not 3×: a gx10/yoga runner is 2.5× an — over E1 budget (rank 85/93)
- P0 triage: 30 remote branches carry unmerged content and never had a PR — five families, not thirty items #3252 P0 triage: 30 remote branches carry unmerged content and never had a PR — five f — over E1 budget (rank 11/93)
- gx10: 45 per-PR copies of the SAME cargo registry — 98 GB where 2.4 GB would do, and no reaper rule can fix it #3279 gx10: 45 per-PR copies of the SAME cargo registry — 98 GB where 2.4 GB would do, — over E1 budget (rank 86/93)
- README's generated CONTRACT_COUNT block is one behind the merge tree (1837 vs 1838) — guard-cargo red on #3093, and it will land on main #3286 README's generated CONTRACT_COUNT block is one behind the merge tree (1837 vs 18 — over E1 budget (rank 87/93)
- pmat 3.40.1 is the DECLARED fleet state; aprender's pins still say 3.40.0 — and /opt/ci-tools lags both #3291 pmat 3.40.1 is the DECLARED fleet state; aprender's pins still say 3.40.0 — and — over E1 budget (rank 88/93)
- A PR-body guard fed from the event payload cannot be satisfied by editing the body — say so in the refusal #3324 A PR-body guard fed from the event payload cannot be satisfied by editing the bo — over E1 budget (rank 89/93)
- A clean-room runner lost its pinned 1.93.0 toolchain mid-build (ENOENT on rustc) and a live build lost target/**/*.d — not the reaper #3345 A clean-room runner lost its pinned 1.93.0 toolchain mid-build (ENOENT on rustc) — over E1 budget (rank 90/93)
- Nightly producers feeding a T-2 row: ≥2 consecutive failures auto-open an issue on the next milestone and report NIGHTLY-RED in §7 (operator 2026-09-17) #3416 Nightly producers feeding a T-2 row: ≥2 consecutive failures auto-open an issue — over E1 budget (rank 91/93)
- release train: automatic milestone triage at T-24h, T-0 and post-tag #3458 release train: automatic milestone triage at T-24h, T-0 and post-tag — over E1 budget (rank 28/93)
- check_milestone_cut.sh: exempt the release epic by its issue number from 06x-release-schedule.md §5, not by title/label match #3460 check_milestone_cut.sh: exempt the release epic by its issue number from 06x-rel — over E1 budget (rank 92/93)
- cascade-publish.sh TIERS is not a dependency order — 47 non-dev violations at v0.68.1; it only works through the drain's retries #3462 cascade-publish.sh TIERS is not a dependency order — 47 non-dev violations at v0 — over E1 budget (rank 13/93)
- aprender-serve: the realizar --lib test binary peaks at 33 GB RSS under
cargo test— OOM-killed by a 32 GB cgroup #3466 aprender-serve: the realizar --lib test binary peaks at 33 GB RSS under `cargo t — over E1 budget (rank 93/93) - Deferred: sccache cold/warm + P0·Fan-out measurement on the v0.68.1 fixture (operator 2026-09-18: build servers first) #3478 Deferred: sccache cold/warm + P0·Fan-out measurement on the v0.68.1 fixture (ope — over E1 budget (rank 12/93)
- roadmap-aggregate: pmat roadmap aggregate is the one writer of roadmap.yaml; roadmap_fragments.py aggregate becomes the read-only oracle #3493 roadmap-aggregate: pmat roadmap aggregate is the one writer of roadmap.yaml; roa — over E1 budget (rank 29/93)
- PACK-001 first main run: three shards of one push chose three tiers (reflog-derived base) and partition 2/3 never ran while the fan-in passed #3531 PACK-001 first main run: three shards of one push chose three tiers (reflog-deri — over E1 budget (rank 59/93)
- apr run: the --no-gpu arm prints no Backend: line, so the CPU arm cannot be proved (post-publish 0.68.2) #3541 apr run: the --no-gpu arm prints no Backend: line, so the CPU arm cannot be prov — over E1 budget (rank 15/93)
- prefix_cache_hits is counted and unit-tested but exposed nowhere: /metrics has no prefix/cache counter, so a prefill measurement cannot exclude cache hits #3553 prefix_cache_hits is counted and unit-tested but exposed nowhere: /metrics has n — over E1 budget (rank 16/93)
- jq 1.6 vs 1.7+ disagree on blank input and our fleet straddles the boundary — a 'jq -e' gate passes on lambda/intel and fails on gx10/yoga #3554 jq 1.6 vs 1.7+ disagree on blank input and our fleet straddles the boundary — a — over E1 budget (rank 17/93)
- Two consecutive non-passing plan quorums on a must-carry ticket must escalate automatically — #3452 sat through four #3562 Two consecutive non-passing plan quorums on a must-carry ticket must escalate au — over E1 budget (rank 18/93)
- The hardcoded-path analyser ignores /mnt entirely — a committed file with both /home and /mnt paths scores +1, not +2, so operator-box paths enter the tree unopposed #3592 The hardcoded-path analyser ignores /mnt entirely — a committed file with both / — over E1 budget (rank 19/93)
- book-contracts.yml compiles and RUNS the 27 chapter examples on floating
stable, not the 1.93.0 pin — threedtolnay/rust-toolchain@stablesteps override it by construction #3593 book-contracts.yml compiles and RUNS the 27 chapter examples on floating `stable — over E1 budget (rank 30/93) - guard-cargo reports a ratchet FAIL when cargo never ran: the per-run target dir becomes root-owned intermittently (reaper race, #3219 family) — cause NOT yet established #3603 guard-cargo reports a ratchet FAIL when cargo never ran: the per-run target dir — over E1 budget (rank 20/93)
- #3540's red set varies across heads that differ by an 11-line quorum JSON: determinism (X64) and a PATH-resolution guard-cargo step — a runner lottery, not a flake #3607 book-contracts.yml: delete the three rust-cache steps — cache-bin false was never the fix (PMAT-3539) #3540's red set varies across heads that differ by an 11-line quorum JSON: deter — over E1 budget (rank 21/93)
- A shard killed by SIGNAL reports as a bare 'test run failed' — 11,757/40,750 not run on yoga-build3, nextest printed the number, nothing reads it; must report Unknown{Signal} with ran/not-run/total #3625 A shard killed by SIGNAL reports as a bare 'test run failed' — 11,757/40,750 not — over E1 budget (rank 22/93)
- clippy --features cuda is red on main (unused rms_norm_forward import in aprender-train) and no gate runs that combination #3636 clippy --features cuda is red on main (unused rms_norm_forward import in aprende — over E1 budget (rank 40/93)
- Fleet declares NO python in automation (infra/CLAUDE.md; sovereign-ci image has no python3 by design) — aprender has 53 .py scripts and 97 shell guards invoking python: inventory, UNMEASURED-with-a-name, and interpreter-free readers #3649 Fleet declares NO python in automation (infra/CLAUDE.md; sovereign-ci image has — over E1 budget (rank 60/93)
- The committed roadmap.yaml AGGREGATE re-creates the conflict the fragments removed — every fragment-minting PR regenerates it, adjacent ids conflict in the merge-queue group build (#3637 UNMERGEABLE behind #3631; 18 such PRs armed tonight) #3650 The committed roadmap.yaml AGGREGATE re-creates the conflict the fragments remov — over E1 budget (rank 23/93)
- New issues can carry neither a priority label nor a milestone — nothing enforces triage, so one sat unclassified for a day #3687 New issues can carry neither a priority label nor a milestone — nothing enforces — over E1 budget (rank 31/93)
- parity_receipt_denominator.sh needs a python-free classify() — then drop its UNMEASURED path #3694 parity_receipt_denominator.sh needs a python-free classify() — then drop its UNM — over E1 budget (rank 55/93)
- prepare_bump.sh --ship opens a bump PR whose body fails §6 R-2 (check_pr_closes_issue): the CHANGELOG's context refs (#EPIC, 'refs #N') have no keep-open line — hit on 0.68.2 (#3498) and 0.69.0 (#3698) #3699 prepare_bump.sh --ship opens a bump PR whose body fails §6 R-2 (check_pr_closes_ — over E1 budget (rank 56/93)
- aprender: stop tracking roadmap.yaml + Pmat-Ticket trailers — the precondition for sovereign-ci roadmap-fragment-parity (A2 of #3676) #3709 aprender: stop tracking roadmap.yaml + Pmat-Ticket trailers — the precondition f — over E1 budget (rank 61/93)
- aprender-gpu test_gpu_allocation_under_pressure SIGKILLs on GB10 unified memory (host OOM) — size pressure from MemAvailable or refuse on unified-memory devices #3713 aprender-gpu test_gpu_allocation_under_pressure SIGKILLs on GB10 unified memory — over E1 budget (rank 62/93)
- #3544 (d)+(b): the train emits each host's receipt in the schema the gate reads, and a missing $AP/receipts is an infra fault, never "4 failing hosts" #3731 Make the host-receipt DEFER fail closed — 0.68.2 discharged it by hand, and the gate's first run fabricated 4 host failures from a missing directory #3544 (d)+(b): the train emits each host's receipt in the schema the gate reads, — over E1 budget (rank 41/93)
- #3544 (c): derive the dogfood host matrix from the hosts the train visits (workflow job labels), NA{reason, decided_by, date} for declared absence; fails both ways #3732 Make the host-receipt DEFER fail closed — 0.68.2 discharged it by hand, and the gate's first run fabricated 4 host failures from a missing directory #3544 (c): derive the dogfood host matrix from the hosts the train visits (workf — over E1 budget (rank 42/93)
- 0.70.0:
apr serveoutlives its launcher and keeps the port — a leaked listener on a shared box (found via #3828) #3838 0.70.0:apr serveoutlives its launcher and keeps the port — a leaked listener — over E1 budget (rank 24/93) - Fold PMAT-272 guard + perf-matrix fixes into 0.70.0 (parked off the 0.69.1 batch) #3841 Fold PMAT-272 guard + perf-matrix fixes into 0.70.0 (parked off the 0.69.1 batch — over E1 budget (rank 39/93)
- gpu-q cannot see a process that never took the lock: Ollama held 1328 MiB and a cuda suite returned 34 failures / 22 'Primary context poisoned' #3909 gpu-q cannot see a process that never took the lock: Ollama held 1328 MiB and a — over E1 budget (rank 63/93)
- Ollama daemon loads onto the GPU mid-measurement, bypassing /tmp/apr-gpu.lock — every 'card clear' device receipt on lambda is unsound #3964 Ollama daemon loads onto the GPU mid-measurement, bypassing /tmp/apr-gpu.lock — — over E1 budget (rank 43/93)
- gpu-q: a waiter whose queue ticket disappears spins forever (34h measured on gx10) and ignores SIGTERM #3966 gpu-q: a waiter whose queue ticket disappears spins forever (34h measured on gx1 — over E1 budget (rank 57/93)
- Release process for 0.70+: CRUX-smoke gate on the release binary; full ladder + full CRUX move to nightly; #4033 levers wired into the workflow #4045 Release process for 0.70+: CRUX-smoke gate on the release binary; full ladder + — over E1 budget (rank 44/93)
- certification: per-architecture carry-forward — a diff invalidates only the cells whose inference path it reaches (parked, after #4040) #4049 certification: per-architecture carry-forward — a diff invalidates only the cell — over E1 budget (rank 64/93)
- Verify #3658's main-moved merge-group path on its first live occurrence (tier re-derived from HEAD^1..HEAD, not full) #4050 Verify Every merge-group run is tier=full: ci_test_tier.sh requires HEAD^2 but the queue is SQUASH (one parent) — a text-only PR runs ~2 runner-hours of unit tests; HEAD^1..HEAD is already the PR diff #3658's main-moved merge-group path on its first live occurrence (tier re — over E1 budget (rank 65/93)
- ci: a single #4046 shard left 64G of target on gx10 — delete the job's own run target on exit, then pick a size lever by measured kind #4102 ci: a single chore: 0.69.1 merge-back — release/0.69.1-batch-2 + rc/0.69.1-x (X2) into main #4046 shard left 64G of target on gx10 — delete the job's own run t — over E1 budget (rank 45/93)
- 0.69.1 post-release follow-up: preflight tooling (R4 --via, R7 --scope-tree/--crux), crux-smoke meta skip, apr capability book chapter #4116 0.69.1 post-release follow-up: preflight tooling (R4 --via, R7 --scope-tree/--cr — over E1 budget (rank 66/93)
- 0.70 release gate: wire --scope release (smoke on the release binary + admitted nightly) into autopilot models step, R7 and the candidate watch #4117 0.70 release gate: wire --scope release (smoke on the release binary + admitted — over E1 budget (rank 52/93)
- 0.70 release gate: live end-to-end dress rehearsal on lambda + gx10 through all four #4117 callers (real smoke, ssh, judge) #4132 0.70 release gate: live end-to-end dress rehearsal on lambda + gx10 through all — over E1 budget (rank 25/93)
- models_t1.sh: the gx10 leg fetches only origin/main -- a release/* candidate (the candidate watch) fails FETCH-FAILED #4145 models_t1.sh: the gx10 leg fetches only origin/main -- a release/* candidate (th — over E1 budget (rank 14/93)
- Every [[bin]] --version prints the build SHA (shared build-sha), then #4189 nightly requires it #4219 Every [[bin]] --version prints the build SHA (shared build-sha), then P0: nightly publishes 1 of 28 [[bin]]s (apr CPU only) and is RED today — build every bin at main head + publish a SHA manifest for lambda #4189 nigh — over E1 budget (rank 46/93)
- rc cut: tag vX.Y.Z-rc.N on every green merge to release/X.Y.Z (RC-DOGFOOD-001 §4.1) #4285 rc cut: tag vX.Y.Z-rc.N on every green merge to release/X.Y.Z (RC-DOGFOOD-001 §4 — over E1 budget (rank 47/93)
- promote rc → final with no rebuild: vX.Y.Z assets are the rc's bytes, sha256-equal (RC-DOGFOOD-001 §4.4) #4286 promote rc → final with no rebuild: vX.Y.Z assets are the rc's bytes, sha256-equ — over E1 budget (rank 48/93)
- apr-fold: fold armed-red single PRs into per-epic batch PRs #4310 apr-fold: fold armed-red single PRs into per-epic batch PRs — over E1 budget (rank 58/93)
- fleet-bins: NIGHTLY-STALE on paiml/aprender #4324 fleet-bins: NIGHTLY-STALE on paiml/aprender — over E1 budget (rank 67/93)
- nightly ships no CUDA apr build, so GPU fleet hosts can never run the apr nightly #4326 nightly ships no CUDA apr build, so GPU fleet hosts can never run the apr nightl — over E1 budget (rank 49/93)
- release(rc): an RC is not shipped until every fleet host runs it (install before or with publish) #4327 release(rc): an RC is not shipped until every fleet host runs it (install before — over E1 budget (rank 50/93)
- fleet-bins: UNVERIFIABLE on paiml/aprender #4337 fleet-bins: UNVERIFIABLE on paiml/aprender — over E1 budget (rank 68/93)
- qwen-story-daily cargo-installs apr into the gx10 host's ~/.cargo/bin, overwriting the fleet-bins release binary every beat #4346 qwen-story-daily cargo-installs apr into the gx10 host's ~/.cargo/bin, overwriti — over E1 budget (rank 53/93)
- GH-4117 follow-up: candidate watch runs the --scope release gate as a REAL row (needs #4045 M8 port) #4349 0.70 release gate: wire --scope release (smoke on the release binary + admitted nightly) into autopilot models step, R7 and the candidate watch #4117 follow-up: candidate watch runs the --scope release gate as a REAL row ( — over E1 budget (rank 69/93)
- ARB-AUD-10: fleet_cells_gate.sh lands on main without Python, after ARB-AUD-01 #4350 ARB-AUD-10: fleet_cells_gate.sh lands on main without Python, after ARB-AUD-01 — over E1 budget (rank 70/93); reopened, parked again C334 (sub-issue of ARB-AUD-10b: python3 out of the release path — 12 scripts / 43 sites + release_wall_time.py, jq with a parity table per script #4352, from EPIC: release train 0.70.2 #4678) — REOPENED 12:11Z by C334 quorum r2 (rule 3 puts it on train 0.71; see From EPIC: release train 0.70.2 #4678 above)
- ARB-AUD-10b: python3 out of the release path — 12 scripts / 43 sites + release_wall_time.py, jq with a parity table per script #4352 ARB-AUD-10b: python3 out of the release path — 12 scripts / 43 sites + release_w — over E1 budget (rank 51/93); reopened, parked again C334 (rank 15/37, from EPIC: release train 0.70.2 #4678) — REOPENED 12:11Z by C334 quorum r2 (rule 3 puts it on train 0.71; see From EPIC: release train 0.70.2 #4678 above)
- Release+nightly must ship aarch64-apple-darwin (mini) and a wgpu-enabled x86_64 (intel) apr/pv asset; gate fails on any fleet host without a sha-verified install #4353 Release+nightly must ship aarch64-apple-darwin (mini) and a wgpu-enabled x86_64 — over E1 budget (rank 54/93)
- FLOW-08: cop CI-slot grant refuses when FR-4 (open PRs > 10) or FR-10 (integration WIP) is red, or when a non-rc PR carries a fold; fold detector #4422 FLOW-08: cop CI-slot grant refuses when FR-4 (open PRs > 10) or FR-10 (integrati — over E1 budget (rank 32/93)
- Y2: nextest shard rebalance — weighted explicit-fragment plan + quick tier hash-partitioned on all shards #4424 Y2: nextest shard rebalance — weighted explicit-fragment plan + quick tier hash- — over E1 budget (rank 71/93)
- APR-EPIC-001 A0: re-derive live issue state (read-only receipt) #4432 APR-EPIC-001 A0: re-derive live issue state (read-only receipt) — over E1 budget (rank 72/93)
- Ruleset: merge queue B/K + auto-rollback drift check (after FLOW-04) #4435 Ruleset: merge queue B/K + auto-rollback drift check (after FLOW-04) — over E1 budget (rank 73/93)
- APR-EPIC-001 A1: epic skeleton — epics.yaml + 13 epic issues #4438 APR-EPIC-001 A1: epic skeleton — epics.yaml + 13 epic issues — over E1 budget (rank 74/93)
- B2: root-cause fixes for red nightlies on main #4715 B2: root-cause fixes for red nightlies on main — over E1 budget (rank 11/37, C334 from EPIC: release train 0.70.2 #4678) — REOPENED 12:11Z by C334 quorum r2 (rule 3 puts it on train 0.71; see From EPIC: release train 0.70.2 #4678 above)
- 0.70.2 T39: a tag push's CI run is red by construction #4745 0.70.2 T39: a tag push's CI run is red by construction — over E1 budget (rank 20/37, C334 from EPIC: release train 0.70.2 #4678)
- 0.70.2 T40: guard-cargo runs 31-60 min against its 60-min section wall #4746 0.70.2 T40: guard-cargo runs 31-60 min against its 60-min section wall — over E1 budget (rank 21/37, C334 from EPIC: release train 0.70.2 #4678)
- 0.70.2 T44: every nightly producer starts from nightly-pick and measures the commit it pinned #4749 0.70.2 T44: every nightly producer starts from nightly-pick and measures the com — over E1 budget (rank 22/37, C334 from EPIC: release train 0.70.2 #4678)
- Q9-F1: tag_coverage_gate version-only test compares sorted line sets, so moved Cargo lines reuse coverage; restore the 40-hex sha check #4819 Q9-F1: tag_coverage_gate version-only test compares sorted line sets, so moved C — over E1 budget (rank 23/37, C334 from EPIC: release train 0.70.2 #4678)
- Q9-F2: ci_mg_reuse pick_run does not bind the reused run to this PR or to base main #4820 Q9-F2: ci_mg_reuse pick_run does not bind the reused run to this PR or to base m — over E1 budget (rank 24/37, C334 from EPIC: release train 0.70.2 #4678)
- Q9-F3: check_guard_steps_run_all self-test exits 0 with an UNMEASURED row; fail closed #4821 Q9-F3: check_guard_steps_run_all self-test exits 0 with an UNMEASURED row; fail — over E1 budget (rank 25/37, C334 from EPIC: release train 0.70.2 #4678)
- 0.70.2 T38: report dogfood row-order timing from a nightly lane (R6 follow-up 2) #4744 0.70.2 T38: report dogfood row-order timing from a nightly lane (R6 follow-up 2) — over E1 budget (rank 26/37, C334 from EPIC: release train 0.70.2 #4678)
- 0.70.2 T45: fleet-toolset's 04:47Z cron is dispatched hours late, after every nightly read #4750 0.70.2 T45: fleet-toolset's 04:47Z cron is dispatched hours late, after every ni — over E1 budget (rank 27/37, C334 from EPIC: release train 0.70.2 #4678)
- 0.70.3: docs-only fast path redesign: one total verdict function, not hand-picked shell excuses (#4717) #4752 0.70.3: docs-only fast path redesign: one total verdict function, not hand-picke — over E1 budget (rank 28/37, C334 from EPIC: release train 0.70.2 #4678)
- 0.70.3: coverage-chain guard: check the release gate's own inputs, not every spelling of
on:andgh run list(redesign of #4745) #4757 0.70.3: coverage-chain guard: check the release gate's own inputs, not every spe — over E1 budget (rank 30/37, C334 from EPIC: release train 0.70.2 #4678) - T33: ci-unwedge off the clean-room pool + full-queue window #4732 T33: ci-unwedge off the clean-room pool + full-queue window — over E1 budget (rank 31/37, C334 from EPIC: release train 0.70.2 #4678)
- PY-PORT R1: port scripts/lib/dag_status.py to aprender-ci-tools dag-status #4786 PY-PORT R1: port scripts/lib/dag_status.py to aprender-ci-tools dag-status — over E1 budget (rank 32/37, C334 from EPIC: release train 0.70.2 #4678)
- nightly_train: judge the newest commit all verdict producers share, not main's head #4798 nightly_train: judge the newest commit all verdict producers share, not main's h — over E1 budget (rank 33/37, C334 from EPIC: release train 0.70.2 #4678)
- ci(guards): a dropped guard-cargo row survives every guard — add a row-count ratchet #4851 ci(guards): a dropped guard-cargo row survives every guard — add a row-count rat — over E1 budget (rank 35/37, C334 from EPIC: release train 0.70.2 #4678)
- B1 nightly: the train judges only nightly-pick's C #4817 B1 nightly: the train judges only nightly-pick's C — over E1 budget (rank 36/37, C334 from EPIC: release train 0.70.2 #4678)
- tree-reader registry: 2018-layout module declarations fall back to the whole crate (BSE-17 rows 6/90/91 red in guards-nightly) #4829 tree-reader registry: 2018-layout module declarations fall back to the whole cra — over E1 budget (rank 37/37, C334 from EPIC: release train 0.70.2 #4678)
- C341 Complete SafeTensors Model Serialization for Production Readiness #8 (operator, 2026-10-08 15:16Z): bound each job's threads (cargo build jobs, nextest test threads) to the host's threads divided by its runner units, set in the runner unit's environment by forjar. Design comes as one question after the unit cap has run three nights. No new issue.
- C344 Complete SafeTensors Model Serialization for Production Readiness #8 (operator, 2026-10-08): rule c. Runners fetch from one mirror on the LAN, and a single job keeps that mirror current. The commit sha proves the content. Design comes as one question after 0.70.3 is live. No new issue.
Superseded epics
- EPIC: release-process-aprender.md v0.1 — the build order, the two gates that cannot fail, and the decisions 0.66 is blocked on #3058 EPIC: release-process-aprender.md v0.1 — the build order, the two gates that can — superseded (A4); owner lines: evidence pack, with every claim carrying the command that produced it. / and 5 are work with an owner, not descriptions. Add a guard that reads the runner-group / the review decided a question it claimed to escalate. **Ten quorum rounds ran in total and
- EPIC: release train 0.70.0 — receipts replace refusals #3081 EPIC: release train 0.70.0 — receipts replace refusals — superseded (A4); owner lines: none
- Epic: faster release certification — ladder + CRUX (operator: "do all", 2026-09-23) #4033 Epic: faster release certification — ladder + CRUX (operator: "do all", 2026-09- — superseded (A4); owner lines: none
- EPIC: Claude-Code-style update check in EVERY sovereign binary #4232 EPIC: Claude-Code-style update check in EVERY sovereign binary — superseded (A4); owner lines: none
- EPIC: tagged release-candidate binaries as the dogfood + dev-preview pipeline (RC-DOGFOOD-001) #4288 EPIC: tagged release-candidate binaries as the dogfood + dev-preview pipeline (R — superseded (A4); owner lines: none
Activity
noahgift commentedon Sep 23, 2026
Plan posted: PR #4010
Rows FT-1 to FT-12 each have a done_when, a measured baseline and a first-green proof. aprender-19 wrote the draft; aprender-cb added measurements and ran the decision quorum. Quorum: 3 lanes, all gemini (the gpt pool is 429 until about 09-27), so the result is advisory. Nothing is applied: no child issues and no milestone moves. The PR is not armed.
cab272e5dis on no remote branch. main still runs one retry round.noahgift commentedon Sep 23, 2026
Milestone triage PROPOSAL: 0.70.0 + unmilestoned (nothing moved)
Table: PR #4024,
docs/roadmaps/milestone-triage-2026-09-23.md. It covers 314 open issues (178 in 0.70.0, 136 with no milestone), each with a proposed milestone and a one-line reason. Moves need the operator's approval; nothing has been moved.0.70.0 today (178), proposed destinations:
So 0.70 would shrink from 178 to 77, and what remains is release/CI/gate/lock/sweep work plus debt-ratchet slice 1 (coverage).
No milestone today (136), proposed destinations:
Decisions for the operator (named by more than one epic):
Findings:
apr qaexit 5 is judgedok#3898, ladder_serve_teardown reads 'not answering /health' as 'server gone', so a still-loading server is reported teardown: clean #3943, model_ladder.sh: serve_rc is a dead variable — every serve route can 503/500 and the row is still green #3886, model_ladder receipt: per-rowgreenis not the gate's verdict — 3 rows marked green that check_model_ladder FAILs (mirror of #3842) #3897, P0: chat/code judged by rc and serve by http — a verb emitting 'zombie zombie zombie' is recorded as working, and a GREEN row proves it #3921, CRUX judge floor (#3739): a cell judged on fewer than two engines is UNJUDGED, and the receipt states per cell WHICH engines ran and at WHICH versions #3832.Prepared by aprender-36 for aprender-cf (cop).
docs(roadmap): record the APPLIED milestone move and post-move counts…
docs(roadmap): 202 moves, not 201 (199 + 3 retried) (#3998)
noahgift commentedon Sep 23, 2026
Milestone triage APPLIED (#4024): 202 moved, 35 still pending
Operator approval 2026-09-23, relayed by aprender-cf. 202 issues were moved to the milestone #4024 proposes (97 out of 0.70.0, 105 from no milestone), each with a comment naming the table and its reason. No issue was closed.
Open issues per milestone now:
0.70.0 now holds 126:
close?rows;The milestone API's 140 also counts pull requests.
Still pending (untouched): 35
Operator decision (two or more epics name it): 8
Close candidates (stale or superseded): 7
Verify and close (the release tree implements it): 6
greenis not the gate's verdict — 3 rows marked green that check_model_ladder FAILs (mirror of #3842) #3897: model_ladder receipt: per-rowgreenis not the gate's verdict — 3 rows marked green thatapr qaexit 5 is judgedok#3898: RELEASE GATE HOLE: check_model_ladder ignores qa_rc / gates_failed — a Q4_K row with `apr0.69.1 in flight (closes at the tag, or carries): 13
Pinned thread: 1
chore(roadmap): aggregate the PMAT-3998 fragment (pmat roadmap aggreg…
chore(roadmap): PMAT-3998 row is kind:docs (the deliverable is a docs…
docs(roadmap): correct the triage after the quorum -- 3 moves reverte…
noahgift commentedon Sep 23, 2026
Correction to the applied triage (#4024), after its quorum (Opus 5.5 lane 1, FAIL, findings accepted).
Three applied moves broke the table's own rule that an issue named by two epics is an operator decision:
They are reverted to their pre-move state (#3421 and #3428 back to 0.70.0; #3997 back to no milestone), each with a comment. They are now pending decisions:
#3951 (must-carry in #3998 and #3994) is also a decision now: 0.70 or 0.71. It is likely resolved by #3990, so re-measure it first.
The two comments that had failed (#3917, #3532) are posted. #3483's 0.71 placement contradicts the table's own rule for FP8/batched paths (0.73); it is flagged for your re-check and was not re-moved.
Net moves: 199. Still pending: 38 (12 DECIDE, 7 close?, 6 verify-close, 12 in flight, 1 pinned). Nothing was closed. The doc is updated on #4024 @ 5e27b35.
docs(roadmap): recompute the totals from the table after the correcti…
docs(roadmap): 10 placement corrections pending approval + 6 duplicat…
docs(roadmap): record the 9 applied corrections, the #3513 withdrawal…
docs(roadmap): quorum round-5 non-blocking fixes -- #4008 row, closed…
142 remaining items