Repository navigation
ci: fence fat_driver case-table annotations; section git fetch reads up to 3 times (P6) #4936
Description
Re-scoped 2026-10-08 (cop executor): two items. Guard-step timeouts are #4921, not this ticket.
A. Planted fat_driver self-test rows raise live ::error:: annotations
x86-main and every workspace-test shard carried section a: failure, section a: cancelled and external job 'workspace-test': not completed within 12000s as error annotations about a minute in. No section had failed. The lines came from fat_driver.py self-test, whose planted rows print real workflow commands (the table passed 60/60). x86-main runs no external-job wait at all, so its 12000 s line could only come from the table. When the job went red later on another section, the annotation was read as the cause.
- feat(release): nightly assets rehearsal — build every release asset on a schedule, unable to upload (#4806) #4813 head 6bf78ea, run 37784220425, job 113337037463: the annotation came 66 s in, and no
workspace-testcheck run exists on that head. - feat(embed): APR-EMBED-001 EG-0 — probe, M9, verb totality guard, embedding ops, forward contract, corpus #4913 head 964ae94, run 37781580426, job 113325724966: the same annotation, and that head's
workspace-testwent green.
Fix: the self-test runs inside GitHub's ::stop-commands::<token> fence, through a wrapper (scripts/ci/fence_workflow_commands.sh); fat_driver.py is unchanged. Falsifier: a real self-test failure still turns the step red, with one live ::error:: naming it. A wiring check is RED on an unfenced self-test and on a fenced real run/wait, whose deadline ::error:: must stay live.
B. Section git fetch is one read; a stalled read is a red section
Sections died on RPC failed; curl 92 HTTP/2 stream N was not closed cleanly: CANCEL (err 8) or curl 56 Recv failure: Connection reset by peer, on every runner host alike (the x86-main and shard reds of PRs 4931-4935 and 4923). One tier-step fetch of main at depth 1 ran 376 s and died, although main's objects were already local. The read did not answer; it was not too big. The egress path is with infra.
Change (operator C343 #9, P6): scripts/ci/fetch_p6.sh replaces all 11 git fetch call sites in ci/sections.yml and the merge_group deepen in scripts/lib/resolve_base.sh. This reduces section-fetch drops and does not clear them. actions/checkout already reads 3 times and still failed on #4927. Those drops, and the stalls behind every drop, need infra's network act.
- A read that did not answer is read at most 3 times. An answer (missing ref, not found, 401/403/404) is never re-read.
- Each re-read is one status line.
- Every read is cut when it runs below 1000 B/s for 60 s.
- 3 reads with no answer still fail the step.
- A wiring check is RED on any bare
git fetchleft inci/sections.yml.
Activity
fix(ci): section git fetch reads up to 3 times under P6, a stalled re…
[-]ci(x86-main): section a fails 'workspace-test not completed within 12000s' after 66s — false stop[/-][+]ci: fence fat_driver case-table annotations; section git fetch reads up to 3 times (P6)[/+]fix(ci): resolve_base's merge_group deepen reads under P6 too
feat(release): nightly train reads the assets rehearsal as its assets…
revert(release): the assets lane waits for three green rehearsal nigh…
fix(rehearsal): a failed tag fails its step — the tee steps run under…
fix(ci): the review findings on the fence and P6 halves (#4936)
fix(ci): two rows that could not fail, and the helper path found at s…
fix(ci): the live counter could count fewer commands than the runner …
16 remaining items
fix(p6): `fetch_p6.sh -C ''` fetched into the cwd's repo, and the roa…
fix(roadmap): the P6 by-sha read's -C was never reached by a row (#4936)
feat(release): nightly assets rehearsal — build every release asset o…
feat(release): nightly release rehearsal on one commit, with the thre…
fix(ci): section git fetches read up to 3 times through fetch_p6.sh (…
LAB guard batch (re-cut, supersedes paiml#5216): paiml#5025 paiml#5047 …
noahgift commentedon Oct 14, 2026
Folded into #5306 as a checklist row (TRIAGE-100, 2026-10-14): the open-ticket cap is 100, so this ticket is now tracked on that master and closed as not planned. Nothing is lost: reopen it when its work resumes, and an open PR for it reopens it.
Agent: aprender-clerk
noahgift commentedon Oct 14, 2026
Row moved: #5306 was split into named masters (TRIAGE-100 master naming), so this ticket's checklist row is now on #5324.
Agent: aprender-clerk