Repository navigation
yokoten: every gate that judges main's head at read time #5082
Description
Poka-yoke 4 (yokoten) from the nightly-train A3.
Defect found: the Nightly Train judged "main's head at the moment of the read", but its producers had run hours earlier on an older commit. Main moves during the night because the queue keeps merging, and the train ran 6–7 h after its 04:45Z cron (runs were created 10:47Z–11:49Z, 10-05 to 10-10). So every row read "no run on C".
Ask:
- List every train, gate and release-path script in this repo that reads "the head of main at read time" and then judges results produced earlier, by another workflow, on whatever commit was head then.
- For each, record the file:line, what it judges, and whether a merge between the producer and the read can change its verdict.
- Fix each hit the same way: judge the producers' own commit, never fall back to main's head, and add a late-start self-test case.
- Separately, measure why the scheduled train run is created 6–7 h after its cron, from the run timestamps. Do not guess.
Refs #5079
keep-open: tracks the sweep and one PR per hit
🤖 Generated with Claude Code
Activity
fix(release): lanes_pick_c reads tonight's run age without a date cal…
fix(release): gates and train judge the producer's commit, not main's…
noahgift commentedon Oct 14, 2026
Survey for asks 1, 2 and 4, read from main at 8c3b19c. Fixes (ask 3) carry to 0.73, one PR per hit.
Ask 1–2: readers whose verdict a merge between the producer and the read can still change (3 open hits)
.github/workflows/nightly.yml:101-106+scripts/nightly_manifest.py:106-157ref: maincheckout, thengit rev-parse HEAD.github/workflows/readiness-nightly.yml:54-66workflow_run.head_sha(on dispatch,github.sha)GITHUB_SHAfrom its own cron. After any merge between them, no sha carries both artifacts, so the lane is not_measured.github/workflows/qwen-story-daily.yml:150-158workflow_run.head_sha; on dispatch,github.shaAlready safe (judge a picked, pinned or own commit):
nightly_train.shnight_head (picks C, with late-night self-test cases),t2_preflight.sh,tag_coverage_gate.sh,lanes_pick_c.sh,dogfood_night_sha.sh,rehearse.sh --pick-c/--streak,prepare_bump.sh,autopilot.sh,cascade-publish.sh,ci_mg_reuse.sh/ci_mg_workspace_result.sh. By design:nightly_train.sh:401(stale-pin check of the judge itself) andrc_cut.sh(release branch; a later push is meant to supersede the cut).Producer-side exposure (not a reader, but it causes the train's "no run on C"): deep-, examples-, release-gates-, coverage-, models- and assets-rehearsal-nightly each build their own
github.shafrom crons between 20:30 and 00:37 UTC, and none picks night C. Only dogfood-nightly and release-lanes-nightly pin to C. The 0.73 fix list should include pinning these, not only the 3 readers.Ask 4: why the scheduled train starts 5–8 h late (measured, not guessed)
nightly-train.ymlcron is45 4 * * *. Scheduled runs, 10-05 to 10-14:createdAtequalsstartedAton every run, so the time is not queue wait for a runner: the schedule event itself is delivered late. The delay hits the whole repo, not just this workflow.nightly.yml(cron 23:00) was created at 01:40–03:25Z over 10-09 to 10-14 (2h40m–4h25m late), anddogfood-nightly.yml(23:05) at the same minutes. That is GitHub's scheduled-event delivery, which is outside our control. So the fix is a trigger that does not depend on it:workflow_runon the last producer, or a dispatch from that producer. Either one is a workflow change, so it goes to a quorum first.noahgift commentedon Oct 14, 2026
Folded into #2876 as a checklist row (TRIAGE-100, 2026-10-14): the open-ticket cap is 100, so this ticket is now tracked on that master and closed as not planned. Nothing is lost: reopen it when its work resumes, and an open PR for it reopens it.
Agent: aprender-clerk