[noahgift]

Refs #4936
keep-open: B cuts section-fetch drops but cannot clear them; the stalls on the egress path continue until the network is fixed, and #4936 closes only after a post-merge x86-main log shows both mechanisms engaged.

Head ca7d665: what changed since 36e691c

A non-author review of 36e691c measured one gap. No self-test row reached the by-sha read in resolve_base.sh's stacked merge_group path. The P6 row answers on its second deepen read, so the || fallback never ran, and a by-sha read that lost its -C "$REPO_ROOT" stayed green. One file changes: scripts/check_roadmap_diff_additive.sh (+22, -1).

• A new row reaches the by-sha read. It is the same stacked entry, with all 3 deepen reads dead and the by-sha read dead once.
  • It starts from the same cwd as the row above: the planted git's own dir, which is no repo, under a ceiling at the table's temp dir. In the same shell as the reads, it checks that the cwd is no repo.
  • It needs 3 deepen reads, then 2 by-sha reads, and the previous entry's squash as the base.
  • A by-sha read without -C runs in no repo, so the base does not resolve. A by-sha read that skips P6 is never read again. Either way the row is RED.
• The comment on the row above now says what that row covers: the deepen read only.

Table	Rows (36e691c → head)	Change
check_roadmap_diff_additive.sh --self-test	26 → 27	New row: the by-sha read dies once and is read again under P6, in REPO_ROOT.
fetch_p6.sh --self-test	48 → 48	Unchanged.
pin_build_base.sh --self-test	30 → 30	Unchanged.

Mutants, built from the head's tree (the same harness as below: a planted empty repo, re-checked after every run).

• resolve_base.sh and the two P6 rows: 12 mutants and a control, and each did what it names. The control is green.
  • Killed in resolve_base.sh:
    • -C pointed at a missing dir;
    • -C "$REPO_ROOT" dropped from both reads, from the deepen read only, or from the by-sha read only (that one survived on 36e691c);
    • the deepen read made a bare fetch, the by-sha read made a bare fetch (new), or both did.
  • Killed in the rows: either row started in REPO_ROOT with its cwd check kept; either row's ceiling dropped.
  • Survives by design, as a pair: the deepen read's -C dropped and the first P6 row put back as it was at 7b77d51. The new row cannot see this one, because its deepen reads all die in the planted git whatever their -C.
• pin + fetch_p6: 18 killed, 0 survived, re-run on this head. Neither file changed.
• Self-tests from a planted empty repo: fetch_p6 48 ok, pin 30 ok, roadmap 26 of 27. The one red is still the history row (6d9ba27), which main's script fails the same way from there (24 of 25). In a scratch clone with no remote, the roadmap table is 27 of 27.

bashrs 7.4.1, the pinned version, reports 0 errors, and its gate form (--no-ignore --level error) finds 0 SEC, DET or IDEM findings at both heads. On check_roadmap_diff_additive.sh, warnings go 88 → 101 and infos 108 → 125. Two non-author reviews measured those counts independently with 7.4.1. Unpinned 7.4.2 builds disagree with each other on warning counts, so no 7.4.2 count is given here. Every new finding is on the new row's lines. SC2143 (info), on the planted git's count of by-sha reads, is the one code new to the file; the new SEC010, SEC014 and SEC020 findings are warnings. CI's bashrs gate counts only error-severity SEC, DET and IDEM findings, and the shell-lint ratchet counts only errors.

ci.yml, ci/sections.yml, fetch_p6.sh and resolve_base.sh are unchanged since 36e691c.

Not changed here:

• The new row reuses the planted sleep and the chmod that the row above leaves in its dir, so the two rows must stay together. This cannot turn the row falsely green. Without the planted sleep, the row really sleeps (about 40 s) and still checks the same reads. Without the chmod, the real git answers the first read and the row is RED.
• The history row (6d9ba27) prints SKIP and exits 0 when that commit is absent from the object store. That is the same on main and is outside this PR. It is reported separately.

Head 36e691c: what changed since 7b77d51

A non-author review of 7b77d51 found two holes, and the sign-off round raised one nit. All three are fixed here. Two files change: scripts/ci/fetch_p6.sh and scripts/check_roadmap_diff_additive.sh.

• fetch_p6.sh -C "" fetched in the cwd's repo. git -C "" keeps the cwd, so a caller whose dir variable was empty would read and write whatever repo the cwd is. The review measured it: the ref landed in the cwd repo. Now -C with an empty dir is a usage error (rc 129), the same as -C with no dir. A usage error is an answer, so it is never read again.
• The roadmap table could not see a lost -C "$REPO_ROOT". Its P6 row ran from inside REPO_ROOT, so a deepen read without -C still landed in the right repo. This body called that mutant "equivalent" because every caller cds first. That was wrong: check_ont_ratchet.sh calls resolve_base without a cd.
  • The row now runs from a cwd outside REPO_ROOT (the planted git's own dir), under a ceiling at the table's temp dir.
  • In the same shell as the reads, it checks that the cwd is no repo, and its case needs that check to pass.
• The planted git exited 0 on a mistyped SHIM_MODE. It now exits 99, and a row proves it.

Table	Rows (7b77d51 → head)	Change
fetch_p6.sh --self-test	46 → 48	-C '': rc 129, and git never runs. A mistyped SHIM_MODE: rc 99, never 0.
check_roadmap_diff_additive.sh --self-test	26 → 26	The P6 stacked-entry row runs from a cwd that is no repo. It fails if that cwd is a repo.
pin_build_base.sh --self-test	30 → 30	Unchanged.

Mutants, built from the head's tree. Each self-test ran with its cwd in a planted empty repo and TMPDIR inside it. After every run:

• the planted repo still had no ref, no remote and no shallow file;
• the shared git dir had no shallow file;
• the worktree's FETCH_HEAD was unchanged.

The results:

• pin + fetch_p6: 18 killed, 0 survived. That is the 16 from 7b77d51 plus 2 new ones: the empty-dir check dropped, and the planted git's catch-all dropped. Both controls are green (pin 30 ok, fetch_p6 48 ok).
• resolve_base.sh and the roadmap row: 7 mutants, and each did what it names.
  • Killed in resolve_base.sh: -C pointed at a missing dir; -C "$REPO_ROOT" dropped from both reads (it survived on 7b77d51; the by-sha read alone was not reached until ca7d665); the deepen made a bare fetch; both reads made bare fetches.
  • Killed in the row: row 8 started in REPO_ROOT with its cwd check kept (the check turns it RED); row 8's ceiling dropped (from the planted repo, git finds that repo above the cwd).
  • Survives by design, as a pair: -C dropped and row 8 put back as it was at 7b77d51. This is why the old table could not see a lost -C.
• Self-tests from a planted empty repo: fetch_p6 48 ok, pin 30 ok, roadmap 25 of 26.
  • The one red is the history row (6d9ba27). It runs its git from the cwd, so from an empty repo it reads nothing.
  • Main's script fails that row the same way (24 of 25).
  • In a scratch clone with no remote, the roadmap table is 26 of 26.

bashrs reports 0 errors (a local bashrs 7.4.2; the gate form also finds 0 SEC, DET or IDEM errors with the pinned 7.4.1).

• On check_roadmap_diff_additive.sh, every finding code's count equals 7b77d51's (7.4.2, both heads).
• On fetch_p6.sh, two codes fall (SC2249, SC2297) and none rises.

ci.yml and ci/sections.yml are unchanged since 7b77d51. fetch_p6.sh --check-wiring passes: 7 fetches, each through the helper.

Not changed here:

• --check-wiring scans ci/sections.yml only. Today ci.yml runs no git fetch. Extending the scan is a separate change, with its own mutants in the new scope.
• A 403 is an answer, and it is read once, as before.

Head 7b77d51: what changed since a5e3f09

The sections from "Planted falsifiers" down describe a5e3f09 and the review rounds before it. This section is the delta.

• Main merged at 5a8e4b5. Main's fix(guards): pin origin/main to the run's build base, so a re-run judges the main it was built on #4996 moved the build-base pin out of ci/sections.yml into scripts/ci/pin_build_base.sh. So ci/sections.yml now holds 7 fetches (it held 11), and the pin holds 3. All 10 read under P6, as does the resolve_base.sh deepen.
• fetch_p6.sh -C <dir> <args...> stands in for git -C <dir> fetch <args...>. A -C with no dir is a usage error (rc 129). That counts as an answer, so it is never re-read.
• No cd in either P6 caller. pin_build_base.sh and resolve_base.sh pass -C <dir> to the helper. bashrs SEC010 flags a cd into a variable path, so neither caller cds.
• The self-tests read from a cwd no fetch can reach. A fetch that loses its -C runs in the caller's cwd. Run from a checkout, a depth fetch there writes .git/shallow into the shared git dir, and every worktree of that clone breaks. A mutant that dropped -C did exactly that. So pin_build_base.sh --self-test cds into its temp dir and sets git's ceiling to that dir's parent (git never stops at the dir it starts in). fetch_p6.sh --self-test sets its ceiling at its temp dir, and its -C rows start from a dir under it. A fetch without -C now reads "not a git repository".

New rows, each able to fail:

Table	Rows (main → head)	New rows
pin_build_base.sh --self-test	floor 22 → 30, 30 ok	A planted git kills chosen reads. merge_group's base fetch, first read dead: read again and pinned. pull_request's base fetch, two reads dead: pinned on the third. push's tip fetch, first read dead: read again, and the deepen still runs. push's deepen fetch, first read dead: read again, and the tip's first parent is fetched. Three dead reads are the cap, and the pin refuses. A read that answered: read once, never again. Plus: the table's cwd is no repository, and the ceiling is TD's parent.
fetch_p6.sh --self-test	40 → 46	Real git with -C <dir>, from a cwd that is no repo: rc 0, 1 read, and the ref lands in <dir>. curl 92 twice under -C: rc 0 after 3 reads, and every read keeps -C <dir> and the stall cut. -C with no dir: rc 129, git never runs. That cwd is no repo.
check_roadmap_diff_additive.sh --self-test	25 → 26	Stacked merge_group entry, first deepen read dead (curl 92): 2 of 2 deepen reads, and the right base.
ratchet_semantics_test.sh C16	+1 check	The mutant pin runs its fall-through (origin/main = the tip). Before, it crashed on the missing helper and nothing saw it.

Mutants, built from the head's tree. Each self-test ran with its cwd in a planted empty repo and TMPDIR inside it, so its own ceiling carried weight. After every run the planted repo still had no ref, no remote and no shallow file.

• pin + fetch_p6: 16 killed, 0 survived. Both controls are green (pin 30 ok; fetch_p6 PASS). Killed:
  • P6 bypassed; -C dropped at the pin; -C lost inside the helper (under both tables);
  • each of the 3 pin fetches made a bare git fetch;
  • P6_READS=2; answers re-read; helper path wrong; a P6 row dropped (caught by the floor);
  • the self-test's cd dropped; the ceiling dropped; the ceiling set at TD; the cwd row dropped; fetch_p6's ceiling dropped.
• resolve_base.sh (under the roadmap-diff table): 3 killed. They are -C pointed at a missing dir, the deepen made a bare git fetch, and both reads made bare fetches. One survived, and this body called it equivalent: dropping -C "$REPO_ROOT". That was wrong, because check_ont_ratchet.sh calls resolve_base without a cd. On 36e691c it is killed (see that section).

bashrs reports 0 errors on all four changed scripts. On pin_build_base.sh and fetch_p6.sh, no finding code rises against the previous head. actionlint's 2 findings are the same on main and on the head.

Two halves, each with a planted falsifier:

• A, fence. A planted table prints the three feat(release): nightly assets rehearsal — build every release asset on a schedule, unable to upload (#4806) #4813 ::error:: lines. Unfenced it raises 3 live annotations; fenced, 0. The same table planted to fail (exit 3) still fails the step with rc 3 and one live ::error::. Removing the fence's stop line brings all 3 annotations back.
• B, P6 fetch. A planted git on PATH:
  • curl 92 CANCEL twice, then an answer: 3 reads, rc 0.
  • curl 56 on every read: exactly 3 reads, and the step is RED (rc 128).
  • a missing ref, or a 401 with no credentials: 1 read.
  • With P6_READS=1, 8 of the 40 self-test rows fail (a5e3f09).

B reduces section-fetch drops and does not clear them. See "What this does not fix" under B.

Planted falsifiers, against the real scripts as CI calls them (a5e3f09)

These sit outside both case tables. The fence wrapper runs a planted table. fetch_p6.sh runs with a planted git on PATH against a real file:// remote, with the real 10 s backoff. resolve_base.sh is sourced by a relative path from this tree, with REPO_ROOT pointing at a fresh depth-1 clone of a stacked merge-queue entry for each row.

The A rows count live annotations with two fixed counters in the falsifier script. They never use the tree's own --live, so a mutant cannot grade itself:

• hi is never below what the runner acts on. It splits lines on \r as well as \n, as the runner does, and counts each piece once. It counts an annotation anywhere in a piece, in any case, and the legacy ##[error form too. A region starts only at the exact fence stop line, and ends at the first piece holding ::<token> or ##[<token>. After an odd stop line or an odd end, no region starts again.
• lo is never above. It counts only a plain annotation at column 0. Any line naming stop-commands opens a region, and only the exact ::<token>:: line closes it.

A row that wants 0 reads hi. A row that wants exactly 1 wants lo/hi = 1/1, because hi alone can only say "at most 1".

Three mutant copies run the same 21 rows:

• M-off removes each half: the fence has no stop line, and P6_READS=1.
• M-review reverts the three first-round review fixes: the resume line without its leading newline, "could not read Username" removed from the answers, and resolve_base calling the helper by its repo-relative path from REPO_ROOT, as 2481ab6 did.
• M-review2 is the tree at 656bf90, before the second-round fixes.

Every mutation was checked with cmp before the run.

Row	Planted input	Want	On a5e3f09	M-off	M-review	M-review2
A1	unfenced table printing the 3 #4813 lines: live annotations, lo/hi	3/3	3/3	3/3	3/3	3/3
A2	fenced, table passes: rc	0	0	0	0	0
A3	fenced, table passes: live annotations, hi	0	0	3, RED	0	0
A4	fenced, table fails (exit 3): rc	3	3	3	3	3
A5	fenced, table fails: live annotations, lo/hi (the one naming it)	1/1	1/1	4/4, RED	1/1	1/1
A6	wiring check of ci.yml	rc 0	0	0	0	0
A7	fenced, table fails and its last line has no newline: rc / lo / hi	3/1/1	3/1/1	3/2/2, RED	3/0/1, RED	3/1/1
A8	one line holding 3 commands joined by \r, unfenced: live annotations, lo/hi	1/3	1/3	1/3	1/3	1/3
A9	the same line, fenced: rc / live annotations, hi	0/0	0/0	0/3, RED	0/0	0/0
B1	curl 92 CANCEL twice, then an answer: rc	0	0 (30 s: 10 + 20 backoff)	128, RED	0	0
B2	the same: reads	3	3	1, RED	3	3
B3	the same: main fetched	yes	yes	no, RED	yes	yes
B4	curl 56 on every read: rc (step stays red)	128	128	128	128	128
B5	the same: reads, never a 4th	3	3	1, RED	3	3
B6	the same: one status line per read that did not answer	3	3	0, RED	3	3
B7	missing ref (an answer): rc	non-zero	non-zero	non-zero	non-zero	non-zero
B8	the same: reads, an answer is never re-read	1	1	1	1	1
B9	401 with no credentials to offer (an answer): rc / reads	128 / 1	128 / 1	128 / 1	128 / 3, RED	128 / 1
B10	resolve_base sourced by a relative path, REPO_ROOT another tree: the stacked entry is deepened	yes	yes	yes	no (refused), RED	yes
B11	the same, and the caller cds elsewhere between sourcing and the call	yes	yes	yes	no (refused), RED	no (refused), RED
B12	the same, under a CDPATH holding a decoy scripts/lib	yes	yes	yes	no (refused), RED	no (refused), RED

All 21 rows hold on the head. M-off fails 9 rows: 4 in the fence half and 5 in the P6 half. M-review fails 5: A7, B9, and all three resolve_base rows, because the repo-relative call fails in any other tree. On M-review, A7 is red on lo only. The glued resume line ends the region for hi, but not on the runner, which then swallows the failure annotation. That is why A7 reads both counters. M-review2 fails B11 and B12, the two shapes the second round found. Before #4936, resolve_base ran git -C "$REPO_ROOT" fetch with no script path at all, so neither shape could fail it; 656bf90 had brought that failure in, and 967b7ea takes it out. A8 is a control: it shows the hi oracle sees all 3 commands on a \r-joined line, where lo, which reads only column 0 of a \n-line, sees 1.

Main at f0bc7d9 (#4934) merges into a5e3f09 with 0 conflicts and no file overlap. On the merged tree these are all green:

• check_guard_steps_run_all.sh, check_ci_push_reuse.sh and check_guards_are_wired.sh.
• guard_tree_parallel_test.sh (9/9) and check_sourced_libs_option_neutral.sh (12 libraries).
• both self-tests and both wiring checks.

Review findings on b7544ba (fourth round), and what was done

Two more non-author reviews (Sonnet 5.5) read b7544ba. The P6 and resolve_base review approved it. The fence review held it on two blocking findings. I reproduced both, and a5e3f09 fixes both. Both reviews then re-read a5e3f09 and approve it. The fence review built its own mutant for each new row, and each went red.

Finding	Done
(blocking) "A 1 is at most 1" was false. The runner splits lines on \r, so printf '::error::a\r::error::b\r::error::c\n' is 3 commands, and --live counted 1. The falsifier script's hi oracle had the same defect.	Fixed. The counter and hi now split lines on \r as well as \n, and count each piece once. A new 8th counter form wants 3 for that line. A8 plants it unfenced (lo/hi 1/3) and A9 fenced (0). Fixing it closed two more under-counts. A region now also ends at ##[<token>, because the legacy form resumes the runner too. After an odd stop line or an odd end, no region starts again, because the runner may be stopped on another token or still be stopped. New rows and a mutant pin each one (tables above).
(blocking) The wiring check passed `... self-test
$* holding a newline could put a second live ::error:: after the fence.	Not changed. The wrapper's argv in ci.yml is fixed (python3 scripts/ci/fat_driver.py self-test), and the wiring check now requires exactly that line.
Wiring blind spots: two spaces before self-test, and a run block that splits a command over lines.	Two spaces: fixed, found and RED. A self-test split over lines fails closed: its self-test line is not the whole fenced command, so it is RED. Still line-based, and not seen: a fenced run or wait split over lines with \, a folded run: > scalar, and a step-level continue-on-error:, if: or shell:. Neither self-test step has any of these today, and C324 already bans continue-on-error on this path.
resolve_base's worst case is 6 reads and about 60 s of backoff, across its two chained helper calls. This body said 3 reads.	Accepted. B now says so. Both calls keep `
(P6 review) No mutant in the table above turns B4, B7 or B8 red: they read the same on M-off, M-review and M-review2.	Accepted, and stated here. The review's own mutants do turn them red: return 0 turns B4 red, and no answer check makes B8 read 3 times. So the rows can fail. No script change.
(P6 review) B7 checks only that rc is non-zero.	Kept. B7 with B8 (1 read) says a missing ref is not retried and does not turn green. The exact rc is git's own.
(P6 review) answered() searches all of git's stderr.	Accepted. A stray match only stops retries early, so the step goes red, never green.

Since b7544ba the only changed file is fence_workflow_commands.sh.

Review findings on 967b7ea (third round), and what was done

Two more non-author reviews (Sonnet 5.5) read 967b7ea: one deep on the fence and resolve_base, one on P6 and its 11 sites. The P6 review approved it. The fence review held it on two findings. Both are fixed here. The first one was mine: I had not refreshed the copy of this body that it read.

Finding	Done
The body the review read named an older head and older counts.	Fixed. The review read a stale copy of this body. Every sha, table and count here is from a5e3f09.
live_commands claimed to err on the high side only, and it did not. It matched in one case only, trimmed only space and tab, missed the legacy ##[error] form and a command after a \r (the runner splits lines on it too), and ended a region only on an exact resume line. Every "0 live" row reads this counter, so a 0 could be a miss.	Fixed. An annotation now counts anywhere in a line, in any case, ##[ form included. A region starts only at the fence's own stop line (column 0, lower case, a 32-hex token, nothing after it). It ends at the first line holding ::<token> in any case, and that line counts too. Three new rows pin it (7 forms, 2 region ends, 2 region starts). Each of the 7 counter mutants below fails one of them, and the 967b7ea counter fails 3.
The row "a colon in a property counts" could not fail: the old counter passed it either way.	Replaced. The 7-forms row holds ::WARNING file=a.rs:1,line=2::, and the case and anchoring mutants fail it.
An upper-bound counter can say only "at most 1" live. The failing-table row read 1 from it.	Fixed. That row now also wants the exact resume line just before the failure line. The planted A rows read a fixed lower and upper counter (above).
(P6 review) The two chained best-effort fetches in one section step can take about 7 min together when every read stalls (about 210 s each).	Accepted. Both sites keep `
(P6 review) check_roadmap_diff_additive.sh's self-test stages resolve_base.sh into a temp tree without scripts/ci/fetch_p6.sh.	Not changed. No row there is a merge_group run, so none deepens. A future stacked row there would find no helper, and the entry would be refused. That fails closed, red and never green.

Since 967b7ea the only changed file is fence_workflow_commands.sh. fetch_p6.sh, resolve_base.sh, ci.yml and ci/sections.yml are byte-identical.

Review findings on 656bf90 (second round), and what was done

Two more non-author reviews (Sonnet 5.5), one deep on each half, approved 656bf90 with no blocking finding. Two rows they found could not fail, which is a defect in a gate, so those were fixed with the rest.

Finding	Done
The fence row "stderr is fenced too" sent only stdout to its file. It passed with the 2>&1 merge removed (0 rows lost).	Fixed. stdout and stderr go to two files, as they are two pipes on the runner. The row wants 0 lines on stderr, 1 inside the fence and 0 live. Removing 2>&1 fails it (1 0 0).
The P6 row "git runs under LC_ALL=C" read the locale the planted git saw. A caller that already exported LC_ALL=C passed it with the helper's own LC_ALL=C removed.	Fixed. The table's caller runs under LC_ALL=POSIX. The nolc mutant fails the row, and also fails it with LC_ALL=C exported around the whole self-test.
resolve_base resolved the helper path at call time from BASH_SOURCE. A caller that sourced it by a relative path and then cd'd, or a CDPATH with a matching scripts/lib, broke the path. The stacked entry was then refused (fails closed).	Fixed. The path is found once, at source time, with CDPATH cleared, and the cd into REPO_ROOT clears it too. Rows B11 and B12 are red on 656bf90 and green here. All 6 caller self-tests pass again (25, 16, 16, 56, 14 and 14 rows), and so does check_sourced_libs_option_neutral (12 libraries).
live_commands counted no indented command and no property value with a colon, so a log could hold more live annotations than it counted.	Fixed in part: an indented command counts, and an indented resume line ends a region. The third round found more forms it missed. See above.
One review read ( … ) >&2 2>/dev/null in resolve_base as dropping the helper's status lines.	No change in behaviour. Redirections apply left to right: >&2 points stdout at stderr first, then 2>/dev/null drops git's own stderr. The other review read it that way too. The comment now says so.
Local git errors (invalid refspec, --deepen and --depth are mutually exclusive, cannot lock ref) are not answers, so they cost 3 reads and 30 s of backoff before the same red.	Accepted. The verdict is the same, only later. Every such form at the 11 sites is fixed text that works today.
A GitHub secondary rate-limit 403 is an answer, read once, although it is often transient.	Accepted. P6 as written makes 403 an answer. 429 and 5xx are re-read.
The wiring regex misses git --git-dir "$d" fetch (an option value after a blank), 'git' fetch, and a git \ line continuation. It flags a message such as echo "git fetch failed".	Not changed. None of these forms is in ci/sections.yml today. A false flag is red on a clean file, never green on a bare fetch. They join $GIT fetch as known blind spots.
Bare fetches remain in other workflows: binary-release.yml (5), guards-nightly.yml, pr-review-quorum.yml and rc-cut.yml.	Not changed. #4936 covers the section fetches, and the wiring check reads ci/sections.yml only. Those sites need their own ticket.
The fence's wiring check reads ci.yml only. A self-test step added to another workflow would not be checked.	Not changed. Both fat_driver.py self-test steps are in ci.yml. provable-ladder-nightly.yml runs fat_driver.py run, a real run that stays unfenced by design.

bashrs reports 0 errors on all three files. The only new warnings are 2 × SC2030 on the scoped CDPATH='' cd inside a subshell, and that scoping is the intent.

Review findings on 2481ab6, and what was done

Two non-author reviews (Sonnet 5.5) approved 2481ab6 with findings. Every finding that changed behaviour got a planted row and a mutant that turns it red.

Finding	Done
The fence's resume line was glued onto a child's last line when that line had no newline. The fence then never closed, and the failure ::error:: was swallowed with it.	Fixed. The resume line now starts its own line, at the cost of one blank line. Rows A7, plus 2 rows in the fence's own table; M-review turns them red.
resolve_base.sh built the helper path relative to the caller's cwd, then ran it after cd "$REPO_ROOT". A caller in another tree lost the deepen and was refused.	Fixed. The path is made absolute first (and, since 967b7ea, found at source time). Row B10 is red on 2481ab6 and green here. The self-tests of all 6 resolve_base callers pass: check_roadmap_diff_additive (25/25, its row 20 deepens for real), check_hardcoded_paths, check_row_pr_write_set (14/14), check_ont_ratchet (56), check_mutants_debt_ratchet (16) and check_roadmap_fragment_required (14/14).
The wiring regex missed git's global options between git and fetch (-c k=v, --no-pager, --git-dir=), quoted words and runs of blanks.	Fixed. 9 must-match and 5 must-not-match forms in the self-test. Restoring the old pattern fails 6 rows.
answered() matched git's English text in whatever locale the runner had. It also re-read three answers: a usage error (rc 129), a 401 with no credentials, and "not a git repository".	Fixed. git runs under LC_ALL=C, and those three are answers, read once. Each rule has its own row and a mutant that fails it.
The backoff row passed with sleep stubbed out, so it could not fail.	Fixed. The planted sleep records its argument. The row wants 7 14 at SLEEP_S=7, and a separate row pins SLEEP_S=10. With the sleep call removed, the row fails.
Wording: "one status line per re-read" was off by one. Every read that did not answer prints a line, the last one included.	Fixed in the script header and here.
The stall cutoff (lowSpeedLimit=1000 for 60 s) could cut a slow but healthy fetch. One case is a server that computes a pack for over 60 s while sending only keepalives.	Measured, not changed. Four real fetches from GitHub through the helper took 0.9 s to 7.9 s end to end. They were: depth 1 of main into an empty repo, three times, the largest at 71 MB; depth 1 of main over a clone holding main~300; and --deepen=1. The pack request was answered in 0.72 s. No phase below 1000 B/s can last 60 s inside a 7.9 s fetch. The residual risk: a server phase over 60 s would be cut and read again, and after 3 reads the step goes red. That is the same red as today, about 3.5 minutes later.
Worst case per fetch is about 210 s (3 reads cut at 60 s, plus 10 + 20 s backoff).	Accepted. The old single read hung for up to 376 s.
A killed wrapper never prints the fence's resume line.	Not changed. The step ends with the wrapper, and its exit status stays the verdict either way. Only annotation display is at stake, and that was not tested on a runner.
$GIT fetch (git named by a variable) is not seen by the wiring regex.	Not changed. No such form exists in ci/sections.yml or ci.yml today. It is listed here as a known blind spot.

A. What it was

No wait was involved. x86-main runs no --external-job, so it never waits on workspace-test. The three early errors on #4813 (13:31:46Z) and #4913 (13:06:42Z) were:

##[error]section a: failure
##[error]section a: cancelled
##[error]external job 'workspace-test': not completed within 12000s

All three come from the step fat_driver case table must still turn RED (fat_driver.py self-test), which passed 60/60 at that point. Its planted rows drive real code paths that print real ::error:: workflow commands: the verdict rows on fixture section a, and the past-deadline external-need row. GitHub turned each one into a job error annotation. The jobs went red later for other reasons:

• feat(release): nightly assets rehearsal — build every release asset on a schedule, unable to upload (#4806) #4813 on pr-review-sign at 14:06:51Z.
• feat(embed): APR-EMBED-001 EG-0 — probe, M9, verb totality guard, embedding ops, forward contract, corpus #4913 on guard-tree, at its 3600 s budget, at 14:06:46Z (guard_tree.sh: name each guard, per-guard timeout, keep fixture ##[error] out of the live log #4919).

The annotation list showed the 12000s line first, so it was read as the cause. The same three lines are printed in every workspace-test-shard job too.

A. Fix

scripts/ci/fence_workflow_commands.sh -- <cmd> runs a case table between ::stop-commands::<token> and ::<token>::, GitHub's own mechanism for printing text without acting on it:

• The token is 32 hex characters from /dev/urandom, fresh per call, and never exported, so the child cannot end the fence.
• The child's stderr is merged into stdout, so every line crosses one ordered pipe and nothing lands after the resume line.
• The resume line always starts its own line, even when the child's last line has no newline.
• The step's rc is the table's rc. A failing table also gets one live ::error::case table failed (rc=N): <cmd> after the fence.

Both fat_driver.py self-test steps (x86-main and workspace-test-shard) go through it. fat_driver.py run and wait stay unfenced, so a real deadline still annotates. The script's wiring check (no arguments) is RED on an unfenced self-test step, on a fenced real run, and on a self-test line that is anything but the fenced command (so || true, ; true or a pipe cannot mask its rc). Its 27-row case table and the wiring check run in x86-main's cheap-checks step. The table passes 27/27 under gawk 5.1.0, mawk 1.3.4 and busybox awk.

The fence stops workflow commands only. Problem matchers still act inside a stop region, and no workflow or ci/ file here registers one (no add-matcher, no matcher:, no toolchain action that adds one).

No gate is loosened: the commands and exit codes are the same, and one guard is added. There is no report-only and no continue-on-error, and no Python is added or changed.

A. Red/green case table (measured locally)

#	Case	Expected	Measured
R1	A real unfinished job still goes RED at its deadline: fat_driver row external need: past the deadline and still absent -> failure	ok	ok, and the row still prints its ::error:: (inside the fence)
R1b	A real fat_driver.py run stays unfenced, so its deadline annotation stays live (wiring check on this head)	PASS	PASS: 2 fat_driver self-test step(s), each fenced; no real run fenced.
R2	Before: the observed case, fat_driver.py self-test unfenced (main)	rc 0, 60/60, the false annotations	rc=0, 60/60, live annotations = 3
R2	After: the same table, fenced (a5e3f09)	rc 0, 60/60, no false annotation	rc=0, 60/60, live annotations = 0
M1	Mutant: drop the wait's deadline (if time.time() >= self.external_deadline: → if False:)	loses a row, step RED	rc=1, 59/60, BAD external need: past the deadline and still absent -> failure got=None; 1 live annotation naming the failed table
M2	Mutant: drop the wait (a pending or absent job returns success)	loses rows, step RED	rc=1, 57/60, BAD: job not listed yet -> pending, job in progress -> pending, another job's success does not count
M3	Mutant: drop the ::stop-commands:: line from the fence	fence table RED, false annotations back	fence --self-test rc=1, 10 FAIL rows; real table back to 3 live
M4	Mutant: ci.yml from main (unfenced self-test)	wiring RED	rc=1, 2 FAIL (both self-test steps)
M5	Mutant: every real fat_driver.py run fenced	wiring RED	rc=1, 6 FAIL
M6	Mutant: the resume line without its leading newline	fence table RED	rc=1, 1 FAIL (a last line with no newline: the fence still closes on its own line)

"Live annotations" means the annotation commands outside every stop..resume region. fence_workflow_commands.sh --live <log> counts them, and the case table uses it. It splits lines on \r as well as \n, as the runner does, and each of its rules errs on the high side, so a 0 from it is a 0 on the runner and a 1 is at most 1. Its control row counts the #4813 lines unfenced as 3. The rows that need exactly 1 also check the structure: the exact resume line on its own line, then the failure line last.

Every fence mutant was built from a5e3f09 by a literal first-occurrence replacement (no regex to get wrong), checked to apply, and run against the 27-row table. No mutant failed on an awk syntax error:

Mutant	Rows lost
no 2>&1 merge	1 (stderr is fenced too, got 1 0 0)
resume line without its leading newline (M6)	1
no stop line (M3)	10
counter does not split lines on \r	2 (8 forms: 3 on one line counted 1; a region end after a \r)
counter matches in one case only	2
counter anchored at column 0 after space or tab	1
counter without the legacy ##[ form	1
a region ends only on an exact resume line	1
a region does not end at ##[<token>	1
an odd end does not put the counter in doubt	1
an odd stop line does not put the counter in doubt	1
a region starts again while in doubt	1
a region starts at any ::stop-commands:: line	1
wiring: any line holding the fenced command is wired (the b7544ba check)	1 (`
wiring: only a single space between fat_driver.py and self-test is found	1 (two spaces: no step found, rc 2 not 1)
wiring: a run: prefix is not stripped	1 (the clean row goes RED)
the whole 967b7ea counter	4
the whole b7544ba counter	3 (8 forms, the ##[ end, the doubt rows)

One mutant survives: counting the line that ends a region (stop = ""; return becomes stop = ""). It can only raise the count, so it cannot turn a check green, and no row pins it. The runner never acts on that line as an annotation: it is the resume itself, or the runner is still stopped on it.

Also run on a5e3f09. All rc 0, except one test that fails the same way on main:

• check_guards_are_wired, check_ci_push_reuse, check_determinism_wait, check_fat_secrets_passed, check_ci_fat_secrets_plumbed and check_no_timing_in_required.
• check_pr_review_wiring, check_explicit_test_commands, check_ci_reusable_workflow_pinned, check_coverage_has_producers, check_bin_cli_tests_wired and check_model_tests_wired.
• check_tree_reader_tests, check_receipt_gate_base_owned, check_hardcoded_paths, check_guard_steps_run_all and check_sourced_libs_option_neutral.
• guard_tree_test (52/52) and guard_tree_parallel_test (6/6).
• guard_tree_job_test fails 1 of 8 here, and on main at ad8fafc the same way: its mutant sed expects determinism last in the gate's needs: list, so the mutant equals ci.yml. No workflow runs that test, and this PR does not touch that line. Filed as guard_tree_job_test row 5: the x86-main-dropped mutant no longer applies (sed expects determinism last) #4940.

actionlint reports nothing on the changed lines; its two info findings are on a step this PR does not touch. bashrs reports 0 errors. On the fence script it gives 18 warnings: the same 18 as on b7544ba, and 3 more than the 15 on 967b7ea. Those 3 are SC2086 on $i and $t3, both inside double quotes (false findings), and PERF002 on a 7-iteration loop in the self-test.

B. Section fetches get 3 reads. This reduces fetch drops; it does not clear them

Sections died on RPC failed; curl 92 HTTP/2 stream N was not closed cleanly: CANCEL (err 8) or curl 56 Recv failure: Connection reset by peer. This happened on every runner host alike, on the x86-main and shard reds of PRs 4931-4935 and 4923. The depth-1 fetch of main in the step "Decide the test tier (BSE-17)" ran 376 s and died, although main's objects were already in the section clone. In a test clone that already holds main's tip, the same fetch moves 0 objects in 2 s. So the read stalled; it was not too big.

What this does not fix. The stalls come from the egress path, so they continue until infra acts on the network. actions/checkout already reads 3 times, and on #4927 all 3 reads failed. More reads can turn a dropped section fetch into a green one when one read gets through. They cannot help when every read stalls. Checkout drops are untouched here and need the same network act.

Change (operator C343 #9, P6). scripts/ci/fetch_p6.sh <git fetch args> replaces git fetch at all 7 call sites in ci/sections.yml and the 3 in scripts/ci/pin_build_base.sh (as -C <dir>). It also replaces the merge_group deepen in scripts/lib/resolve_base.sh, which section guards source.

• A read that did not answer (non-zero exit, HTTP 5xx, curl reset or cancel) is read at most 3 times.
• In resolve_base the deepen is two chained helper calls (--deepen=1, then the parent by sha). Its worst case is 6 reads and about 60 s of backoff (10 + 20 per call) before the entry is refused.
• An answer is never re-read. Answers are: a missing ref, an unadvertised object, repository not found, not a git repository, 401/403/404 (including a 401 with no credentials to offer), and git's usage error (rc 129).
• git runs under LC_ALL=C, so those texts are git's own whatever the runner's locale.
• Each read that did not answer prints one status line naming the read and git's error. The line goes to stdout; in resolve_base it goes to stderr, so no caller's $(...) captures it.
• Every read runs with http.lowSpeedLimit=1000 and http.lowSpeedTime=60, so a stalled read ends and is read again instead of hanging the section.
• If all 3 reads fail, the step still fails with git's rc. The existing || true best-effort sites keep || true. There is no continue-on-error and no report-only.

B. Red/green case table (fetch_p6.sh --self-test, 40 rows, measured locally on a5e3f09; the 6 rows added since are listed at the top)

A planted git on PATH counts its reads and answers as each row says. A planted sleep records each wait. The real-git rows use a file:// remote in a temp dir.

Case	Expected	Measured
Answers at once	rc 0, 1 read, 0 status lines	ok
Every read carries the stall cutoff, under LC_ALL=C (the caller runs under LC_ALL=POSIX)	-c http.lowSpeedLimit=1000 -c http.lowSpeedTime=60 fetch …, locale C	ok
curl 92 CANCEL twice, then an answer (the observed case)	rc 0, 3 reads, 2 status lines naming read and error	ok
curl 56 reset on every read (the network never answers)	step RED, rc 128, exactly 3 reads, last line "no answer under P6"	ok
The same: the waits	7 14 at SLEEP_S=7 (n × SLEEP_S before re-read n+1, none after); SLEEP_S=10 outside the table	ok
HTTP 500 once / non-zero with no error line once	rc 0 after 2 reads	ok
Missing ref, repository not found, 401 with no credentials, not a git repository, usage error (rc 129): answers	1 read each	ok
Real git, ref exists / ref missing	rc 0, fetched, 1 read / fails, 1 read	ok
Wiring RED on 9 forms: git fetch …, git -C "$ws" fetch, git -C "$a b" fetch, git -c http.version=HTTP/1.1 fetch, git --no-pager fetch, git --git-dir=.git fetch, git fetch (two blanks), cd "$ws" && git fetch; echo done, "git" fetch	rc 1	ok
Wiring clean on 5 forms: the helper call, git log --grep fetch, git status && echo fetch, echo "the git fetched it", legit fetch	rc 0	ok
Wiring: unreadable file, or no helper call found	not clean (rc 2)	ok

Mutant (each built from 967b7ea, whose fetch_p6.sh is byte-identical here; checked with cmp)	Rows lost
1 read (P6_READS=1)	8
4 reads (P6_READS=4)	4 (a 4th read; status lines say n/4)
answers re-read (answered dropped)	7
no stall cutoff (-c http.lowSpeed* dropped)	1
no LC_ALL=C	1, with or without LC_ALL=C exported by the caller
the sleep call removed	1
rc 129 / "could not read Username" / "not a git repository" not treated as answers	1 each
the old wiring regex	6
wiring that read nothing (no-helper-call check dropped)	1
resolve_base pointed at a missing helper	check_roadmap_diff_additive row 20 (stacked merge_group deepen, done for real) FAILs

fetch_p6.sh --check-wiring on main 5a8e4b5's ci/sections.yml: rc 1, 8 FAIL lines (7 bare fetches, plus no helper call). On this head: PASS: 7 fetch call(s) in ci/sections.yml, each through the helper. The wiring check reads ci/sections.yml only. The pin's 3 fetches are held by its P6 rows and by the three "bare fetch" mutants killed above. Both --self-test and --check-wiring run in x86-main's cheap-checks step.

Still a single read, and not changed here:

• actions/checkout@v7, which already retries its fetch 3 times itself.
• fat_driver.py's own checkout fetch. It is Python, and a hook refuses edits to it.
• The job-level fetch-depth: 0 checkout on a runner with no warm cache. It pulls the whole pack over the stalling path. That is host and egress work, outside this PR.

Pre-push on the head ca7d665: cargo fmt --all -- --check and the cargo checks were not re-run. The head changes no Rust file, no Cargo.toml, no Cargo.lock and nothing under contracts/: git diff --name-only 5a8e4b5f89 lists 10 files (+770, -14), all workflow, YAML, roadmap and shell. So every Rust, Cargo and contracts/ file is byte-identical to main 5a8e4b5, which passed the merge queue's required checks. Earlier, on a5e3f09: cargo fmt --all -- --check rc 0 (0 diffs) and cargo deny check advisories ok. cargo test -p aprender-contracts --lib ran on 967b7ea: rc 0, 2290 passed, 0 failed, 5 ignored. It ran in a separate clone checked out at that sha. Since then only fence_workflow_commands.sh changed. The whole PR touches no Rust and nothing under contracts/.

Post-push proof (mechanism engaged):

• For A, this PR's x86-main log must show the three lines as raw ::error::… text in the self-test step, between ::stop-commands:: and its resume line. No ##[error]external job may appear before the first real section verdict.
• For B, the same log must show the section steps running bash scripts/ci/fetch_p6.sh where git fetch was. A fetch_p6: read n/3 line appears only when a read did not answer.

🤖 Generated with Claude Code

[github-actions]

§13.11 rung 1 — quorum shadow verdict

S13-SHADOW pr=5051 head=ca7d665c2077486df1f0bbb76b58eee4f1e05886 verdict=REFUSE class=Q1 arm_rc=1

Shadow mode: this records a verdict and merges nothing. A refusal
to arm is not a block (§13 adds zero rows to §7) — the pull request is
exactly as green as it was.

[noahgift]

Superseded by #5066: the same part B patch, re-cut on main at 9e77499. Part A landed in #5029 (fixture_quiet.sh), so this branch's fence is not carried.