fix(pi): treat torn-down-task wake rows as main-only, not queue corruption - #3
Closed
frasdl wants to merge 2 commits into
Closed
fix(pi): treat torn-down-task wake rows as main-only, not queue corruption#3frasdl wants to merge 2 commits into
frasdl wants to merge 2 commits into
Conversation
added 2 commits
August 26, 2026 12:24
…uption A wake row can legitimately outlive its task's state files: the watcher queues a signal/stale row while the task still exists, the branch accepts the wake, and teardown deletes state/<id>.meta before the serialized branchChain reaches that wake. scopeForUnreadWake classified any unmapped signal/stale row as queue corruption (UNSAFE), poisoning the whole scan and producing a false 'Supervision branch unavailable, falling back to main: the unread wake queue could not be read safely' wake (2026-08-26 m365-pi-scout incident). Legible-but-unmapped rows now behave exactly like the existing check-kind class: excluded from the branch's eligible set, left queued for main, non-vetoing in ordinary scans, and still a heartbeat veto. Structural corruption (unknown kind, malformed line, non-numeric seq) still vetoes the whole scan. The classification contract stays owned by fm-branch-dispatch.ts's header; pi-supervision-branch.md documents the boundary. Also: bin/fm-wake-drain.sh no longer abandons an empty claim/consume scratch tmp file in state/ (the .main-eligible-rows.tmp.* and .wake-rows.consume.* litter), and the live Pi branch e2e fixture copies lib/fm-calm-visibility.ts its 0.84.x imports require, refreshing the dated runtime-backends.md evidence for pi-coding-agent 0.84.3. Regression: tests/fm-pi-branch-extension.test.sh (unmapped-row classification, mixed-queue eligibility, heartbeat veto) and tests/fm-wake-queue.test.sh (no scratch tmp left behind).
…vision-branch doc
Owner
Author
|
Abandoned per captain direction ("leave the fm branch completely"); closing without merge. Branch/work remains, unlanded. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Intent
Investigate and fix the "supervision branch silently swallows task wake rows without emitting an outcome" reliability bug in firstmate. Touches shared tracked material (bin/, .pi/extensions/, docs) - follow firstmate-coding-guidelines.
Part 1 root-cause (evidence-driven): the 2026-08-26 m365-pi-scout incident. Scout finished ~11:47-11:48 (report.md mtime 11:47, done: row queued 11:48:47 rows 53/54). Watcher absorbed signals as benign while the scout provably worked (correct). The branch accepted the done-wake at 11:48:48, handled it through 11:50:32 (outcome seq 22 at 11:50:19). Meanwhile a stale wake (row 55, queued 11:49:19) was accepted and chained behind the handling; when the chain re-scanned eligibility (~11:50:32), teardown (11:50:02) had deleted state/m365-pi-scout.meta, so the stale row's window could not be resolved and scopeForUnreadWake classified the queue as CORRUPTED, producing the false fallback "Supervision branch unavailable, falling back to main: the unread wake queue could not be read safely" (delivered 11:51:14, after main's drain at 11:51:17 already found the queue empty). Main also misread the outcome store ~1s before seq 22 landed and concluded the branch swallowed the rows (duplicate relay; maps to upstream open issues kunchenguid#2977 and kunchenguid#2984, which are NOT fixed here).
Part 2: GitHub issue search on the ORIGINAL repo kunchenguid/firstmate (open AND closed) for watcher/supervision wake absorption, missing outcomes, stuck wake queue, "queued wakes pending", stale swallowing, branch actor not reporting - summarize each with number/title/state/URL; state explicitly if none describe this incident (no issue describes the torn-down-row eligibility poisoning; closest are kunchenguid#2977 open, kunchenguid#2984 open, kunchenguid#2028 open, kunchenguid#2791 open, kunchenguid#2374 open).
Part 3 fix (implemented, EVALUATE AS ACCEPTED): in .pi/extensions/lib/fm-branch-dispatch.ts scopeForUnreadWake, a legible signal/stale row whose task metadata is gone (teardown raced the queued wake) is ordinary main-only content exactly like the check-kind class: excluded from eligibleSeqs, left queued for main, non-vetoing in ordinary scans, and still a heartbeat veto. Structural corruption (unknown kind, malformed line, non-numeric seq) still vetoes the whole scan. bin/fm-wake-drain.sh write_rows_file_locked no longer abandons an empty claim/consume scratch tmp file in state/. tests/fm-pi-branch-live-e2e.test.sh fixture copies lib/fm-calm-visibility.ts (required by 0.84.x imports); docs/verification/runtime-backends.md refreshed with dated 0.84.3 results. Regression tests added: tests/fm-pi-branch-extension.test.sh (unmapped classification, mixed-queue eligibility, heartbeat veto) and tests/fm-wake-queue.test.sh (no scratch tmp left behind). docs/pi-supervision-branch.md autonomy section updated; classification contract remains owned by fm-branch-dispatch.ts header. Constraints: keep the fix minimal and fail-closed for structural corruption; do NOT fix upstream issues kunchenguid#2977/kunchenguid#2984 (out of scope); no agent co-author; shellcheck-clean bin scripts; tests must exercise behavior through the public interface, never assert implementation source bytes; pre-existing local-only failure of the stock-render test in tests/fm-pi-branch-extension.test.sh (Pi 0.84.3 version skew, fails on clean main too) must not be attributed to this change.
What Changed
.pi/extensions/lib/fm-branch-dispatch.ts:scopeForUnreadWakenow classifies a legible signal/stale wake row whose task metadata is gone (teardown raced the still-queued wake) as ordinary main-only content instead of corruption - it is excluded fromeligibleSeqs, left queued for main, and no longer vetoes an ordinary scan, while structurally unreadable rows (unknown kind, malformed line) still veto the whole scan and an unmapped row still defers a heartbeat review to main. This removes the false "Supervision branch unavailable" fallback after a task teardown.bin/fm-wake-drain.sh:write_rows_file_lockednow removes the caller's owned scratch file along with the target when the source is empty, so empty main claims and branch consumes no longer abandon.main-eligible-rows.tmp.*/.wake-rows.consume.*litter instate/.tests/fm-pi-branch-extension.test.sh(unmapped-row main-only classification, mixed-queue eligibility, heartbeat veto) andtests/fm-wake-queue.test.sh(no scratch tmp left behind);tests/fm-pi-branch-live-e2e.test.shfixture now copieslib/fm-calm-visibility.ts(required by Pi 0.84.x imports);docs/verification/runtime-backends.mdrefreshed with dated 0.84.3 evidence anddocs/pi-supervision-branch.mdwording aligned.Risk Assessment
Testing
Validated the torn-down-task wake-row fix via the real public interfaces: new classification coverage passes on target (unmapped rows = ordinary main-only, excluded from eligibleSeqs, non-vetoing in mixed queues, still heartbeat vetoes; structural corruption still fail-closed) and reproduces the incident misclassification on base (whole queue corrupted/fallback-to-main, eligible lost), the mixed-queue eligible set [2,3] is published through the real grant bin, the fm-wake-queue suite (including the new no-scratch-tmp test) is green on target and fails on base, and the live e2e guard passes against the real Pi 0.84.3 SDK (fixture now copies fm-calm-visibility.ts). The only failure in the touched suites is the pre-existing Pi 0.84.3 stock-render skew in fm-pi-branch-extension.test.sh (byte-identical in base, explicitly not attributable per the intent), which unfortunately aborts that file before the new classification tests run in normal suite executions - the new tests were proven by direct execution of the same code paths instead.
Evidence: Target vs base scopeForUnreadWake classification of the torn-down (unmapped) wake row and mixed queue
Source: Target vs base scopeForUnreadWake classification of the torn-down (unmapped) wake row and mixed queue
TARGET: unmapped-only {status:unsafe, eligible:false, corrupted:false, eligibleSeqs:[]} / mixed {status:safe, eligible:true, corrupted:false, eligibleSeqs:["2","3"]} / heartbeat-with-unmapped {corrupted:true} / malformed-line {corrupted:true} / snapshot-publish: published rows ["2","3"] BASE: unmapped-only {corrupted:true} / mixed {corrupted:true, eligible:false}Evidence: Base (pre-fix) scopeForUnreadWake misclassification: the 2026-08-26 wake-loss reproduced
Source: Base (pre-fix) scopeForUnreadWake misclassification: the 2026-08-26 wake-loss reproduced
unmapped-only: {"status":"unsafe","eligible":false,"corrupted":true,...} mixed: {"status":"unsafe","eligible":false,"corrupted":true,...} heartbeat-with-unmapped: {"status":"unsafe","eligible":false,"corrupted":true,...}Evidence: Target bin/fm-wake-drain.sh leaves no empty claim/consume scratch tmp
Source: Target bin/fm-wake-drain.sh leaves no empty claim/consume scratch tmp
TARGET(post-fix): no scratch tmp files left in state/Evidence: Base bin/fm-wake-drain.sh abandons empty claim/consume scratch tmps in state/
Source: Base bin/fm-wake-drain.sh abandons empty claim/consume scratch tmps in state/
BASE(pre-fix): left scratch tmp file(s) in state/: .main-eligible-rows.tmp.P93d9M .wake-rows.consume.qL5HxIPipeline
Updates from git push no-mistakes
✅ **intent** - passed
✅ No issues found.
✅ **Rebase** - passed
✅ No issues found.
docs/pi-supervision-branch.md:38- Intent Part 2 (GitHub issue search on the ORIGINAL repo kunchenguid/firstmate, required: 'summarize each with number/title/state/URL; state explicitly if none describe this incident') has no trace in this change: the diff, the commit message, and the repo contain no summary of Pi supervision branch can duplicate one captain outcome and displace the next kunchenguid/firstmate#2977/Branch outcome is marked read at enqueue, so a crash before delivery leaves it durable but unannounced kunchenguid/firstmate#2984/Supervision looks stalled when it is actually thrashing on unread worker reports kunchenguid/firstmate#2028/A live second mate can silently stop consuming its durable wake queue kunchenguid/firstmate#2791/Turn-end wakes are never absorbed for harnesses without a verified busy source kunchenguid/firstmate#2374 or any other searched issue. The intent even supplies the expected conclusion ('no issue describes the torn-down-row eligibility poisoning; closest are Pi supervision branch can duplicate one captain outcome and displace the next kunchenguid/firstmate#2977 open, Branch outcome is marked read at enqueue, so a crash before delivery leaves it durable but unannounced kunchenguid/firstmate#2984 open, Supervision looks stalled when it is actually thrashing on unread worker reports kunchenguid/firstmate#2028 open, A live second mate can silently stop consuming its durable wake queue kunchenguid/firstmate#2791 open, Turn-end wakes are never absorbed for harnesses without a verified busy source kunchenguid/firstmate#2374 open'). If that summary was delivered in the authoring conversation or is destined for this run's PR description (a later pipeline phase), this is satisfied and no repo change is needed; if a durable repo record was intended, it is omitted here. Please confirm the intended delivery location..pi/extensions/lib/fm-branch-dispatch.ts:136- Residual (accepted, documented containment): when the branch's chained re-scan finds the queue reduced to only unmapped (torn-down) rows for an already-accepted wake, enqueueWake now returns without any fallback report, leaving the rows queued for main's next natural drain. A torn-down task's window/status files are gone, so the original re-fire source no longer exists; if no further fleet wake occurs the row sits durably queued (never destroyed, visible in state/.wake-queue). This matches the check-kind class exactly and is documented in docs/pi-supervision-branch.md, so it is informational: no fix requested, just noting the tradeoff versus the old behavior which at least produced an immediate (misleading) main fallback.tests/fm-pi-branch-extension.test.sh:1601- Pre-existing (not attributable to this change): the stock-render testtest_outcomes_tool_uses_stock_execution_and_export_consumersfails on the installed Pi 0.84.3 SDK (version skew; the test body is byte-identical to base commit d88c71f, verified via diff) and, because it is the first test invoked in the file andfail()exits the script, it aborts the entire suite before the newly added unmapped-row classification coverage at line 1603 runs. I verified the full-file run emits only that one not-ok (exit 1), and that all 17 remaining tests - includingtest_branch_dispatch_classifies_main_only_rows_and_writes_the_eligible_snapshotwith the new unmapped/mixed-queue/heartbeat assertions - pass when that invocation is removed. Consequence: the new regression tests added by this change are unreachable in normal suite execution until the version-skew test is repaired; the fix itself is proven by the direct-driver runs below, not by the in-file suite run.bash tests/fm-wake-queue.test.sh- full suite ok (exit 0), including the newtest_empty_actor_claims_leave_no_scratch_tmp_behinddriving the realbin/fm-wake-drain.sh+bin/fm-wake-grant.shwith an empty claim set and empty consume setbash tests/fm-pi-branch-extension.test.sh- aborts at the first test withnot ok - Pi outcomes rendering consumers must preserve stock behavior(Pi 0.84.3 version skew; test body verified byte-identical to base viagit diff d88c71f e7e46ff), so nothing else in the file ranSuite re-run with only the stock-render invocation removed (transient scratch copytests/.fm-pi-branch-ext-skip-stock.test.sh, deleted after): 17/17 ok includingtest_branch_dispatch_classifies_main_only_rows_and_writes_the_eligible_snapshot(new unmapped-only, unmapped+mixed-queue, heartbeat-veto assertions through the real lib + grant bin)FM_PI_BRANCH_LIVE_E2E=1 bash tests/fm-pi-branch-live-e2e.test.sh-ok - real Pi SDK 0.84.3 accepts the branch session construction and preserves an unpromptable wake, validating the fixture now copieslib/fm-calm-visibility.tsClassification driver (classify-driver.mjs) against targetfm-branch-dispatch.ts: unmapped row ->corrupted:falsemain-only; mixed queue ->eligible:true,eligibleSeqs ["2","3"], snapshotpublishedthrough the real grant bin; heartbeat with unmapped row ->corrupted:true; malformed line ->corrupted:true(fail-closed)Same driver against basefm-branch-dispatch.ts(git show d88c71f): unmapped row ->corrupted:trueand mixed queue ->corrupted:true, eligible:false- the 2026-08-26 m365-pi-scout wake-loss misclassification, i.e. the regression fails before the fix and passes afterDrain claims demo (drain-claims-demo.sh) against basebin/fm-wake-drain.sh: leaves.main-eligible-rows.tmp.*and.wake-rows.consume.*in state/ (exit 7); against targetbin/fm-wake-drain.sh:no scratch tmp files left in state/(exit 0)✅ **Document** - passed
✅ No issues found.
✅ **Lint** - passed
✅ No issues found.
✅ **Push** - passed
✅ No issues found.