fix(F192): restore evidence gate + guard fork-only patches against rebuild loss by mindfn · Pull Request #185 · mindfn/clowder-ai

mindfn · 2026-09-22T03:16:23Z

Restores the F192 evidence-prerequisite gate (lost to develop_base rebuilds three times) and fixes the mechanism that kept losing it.

Five-axis risk

Axis	Judgment	Reason
行为面	low-medium	Adds a pre-invocation gate to scheduled eval domains: when a telemetry-backed domain cannot produce evidence, it posts a SKIPPED notice instead of invoking the eval cat. Fail-closed. No user-facing surface.
数据	none	No schema, migration, or persisted-state change. Writes one system-thread notice on skip.
安全	none	No auth, secrets, or permission surface. The removed `EVAL_BASE_URL` parsing reduced env-derived branching.
契约	low	`check:sources` gains `check:fork-only-patches` as its first item. New CI workflow scoped to `develop_base`. No published API/type change.
不可逆	none	Revert is one commit. No destructive operation, no data boundary.

Gate choice: the full canonical pnpm check chain, because this PR modifies the check execution chain itself. pnpm gate was deliberately not used: it is hardwired to origin/main (script line 4: "先冻结一次 origin/main"; line 751), so on a PR targeting develop_base it rebases the branch onto upstream main and may auto-commit hygiene fixes — which would rewrite the reviewed HEAD. That is what rewrote this branch on 2026-09-22 (reflog rebase (start): checkout b2927441, a main-lineage commit). See Structural findings.

What this fixes

Lost three times: PR #91 (2026-08-11), PR #137 (2026-08-28), commit 70d79a2f2 (2026-09-01) — the third unnoticed for 21 days, caused by a commit titled "recover fork-only development layer lost in develop_base rebuild". Each loss lets the scheduler burn a full LLM session per daily eval fire; between 2026-09-22 and 2026-10-11 that happened daily, including after three API restarts.

Root cause, from 70d79a2f2's own body: the rebuild is reset to origin/main + fork/optimizations, and "443 files modified by BOTH main and fork: NOT restored" — the class the gate's wiring lives in.

The durability mechanism

scripts/fork-only-patches.json — registry of patches upstream does not carry
scripts/check-fork-only-patches.mjs — two checks, both fail-closed:
1. tree — each registered patch present now (files, anchors, call-site ordering anchors)
2. rebuild survival — everything the baseline claimed is still present, including the claim set (a rebuild can revert the registry rather than delete it)
Explicit vs default baseline — an explicit --baseline-ref (CI: github.event.before / pull_request.base.sha) that resolves to HEAD fails closed (mis-wired witness). The registry default resolving to HEAD — the normal state of a clean develop_base — uses committed HEAD as the witness for the working tree, so canonical pnpm check passes on its own target branch while uncommitted reversions are still caught. The CI wiring is itself a registered anchor, so dropping --baseline-ref turns the tree check red.
.github/workflows/fork-only-patches.yml — runs on develop_base, where no other workflow does.
docs/fork-only-patches.md — the why, and what is not covered.

Known residual gap (not claimed closed)

An in-tree workflow cannot survive its own deletion: if a rebuild drops workflow + guard + registry together, nothing runs. The authoritative control is branch protection on develop_base requiring Fork-only patch guard — documented, needs repo-admin, not configured by this PR. While repository Actions are not running (below), that check would also block every develop_base merge.

Review history

Round	HEAD	Verdict	Substance
1	`1766db9`	changes_requested	mis-wired `EVAL_BASE_URL` gate; guard could not report its own erasure; bare-substring anchors
2	`00be3f7`	changes_requested	CI fetched the baseline after the push (witness = accused)
3	`bf13816`	approved	—
—	rebase	—	`develop_base` rebuilt; `--onto c29dba70ad aa409203b` (4 commits only)
4	`6f98939`	changes_requested	default baseline == HEAD would make canonical `pnpm check` red on clean `develop_base`
5	`f62a9fc`	approved	local review fact `0001791689009970-000082-dfcc0036`
—	`4744926`	continuity (no re-review)	`develop_base` rebuilt again (1398 via #225/#228); `--onto 176694ebe9 c29dba70ad`, zero authored delta — see manifest
—	`7bcd70c`	continuity (no re-review)	`develop_base` + `fork/optimizations` rebuilt as main + replayed carrier (`8a90e142`); `--onto 8a90e142 176694ebe9`; authored lines byte-identical — see manifest

Merge-gate evidence manifest

Field	Value
head	`7bcd70cf6f2e483d2077a933c1fe7efa5be1229e`
localPeerReviewSha	`f62a9fce91…` (codex, approved) — bridged to head by continuityProof below
cloudReviewSha	none
headChangeCause	`rebase`
stale	false — `old APPROVE + continuityProof(C1,C2,C3)`; `skip=rebase-rereview`
continuityProof	reviewedHead `f62a9fce91` (approved) → rebase 1 `47449262de` (zero delta) → rebase 2: oldBase `176694ebe9` / newBase `8a90e142` / currentHead `7bcd70cf6f`. C1 commits 3–5 stable patch-ids identical; commits 1–2 patch-ids differ only in `package.json` context lines (the base added `build:app-assist`, `check:mcp-surface-governance`, an f212 test). Per commit, the authored `+/-` lines are byte-identical (788 / 857 / 134 / 315 / 164 lines). All 5 messages byte-identical. `postReviewDeltaPaths=[package.json (context only)]`, a lossless union with the new base. C2 base delta ∩ PR paths = {`index.ts`, `package.json`}; `index.ts` lost 2301 lines to the C1 plugin migration, but none of the wiring's identifiers were touched and all are still defined → `relation=none`. C3 see E5.

Accepted-Source-Reack: docs/features/F192-socio-technical-harness-eval.md@22385b60e01aee9d8691b6a867836ff0e94fa77f

Check	Result
E1 head == PR head	pass (local / remote / PR all `7bcd70cf6f`)
E2 stale == false	pass
E3 provenance + accepted source	pass — local fact covers exact HEAD; fence differed (fact used bare HEAD after a thread-id validation bug blocked the thread-source form), reconciled by the re-ack line above
E4 verdict	approved
E5 gate	canonical chain run item by item on exact HEAD, no skips: 42 items, 42 pass, 0 fail, including the base's new `check:mcp-surface-governance` (confirms the `check:sources` edit did not clobber base items). HEAD unchanged, tracked tree clean. Plus: deps reinstalled for the changed lockfile, shared + api build, 70/70 targeted tests, guard on the CI path (`--baseline-ref 8a90e142`) exit 0, both mutation tests re-run with identical results (gate broken → 3 red; explicit branch opened → 2 red).

gate_commands: per-item execution of check:sources / check:installed / check:artifacts; node --test scripts/check-fork-only-patches.test.mjs (28/28); evidence-gate + daily + prereq suites (66/66); pnpm --filter @cat-cafe/shared build, pnpm --filter @cat-cafe/api build; pnpm biome check; git diff --check. Mutation tests: disabling the gate branch turns 3 targeted tests red; opening the explicit-baseline branch turns 2 red.

CI — not green, and not claimed

statusCheckRollup is empty. Repository Actions have created zero check suites since 2026-09-28 (still true at 7bcd70cf6f) — for this PR's new heads and for main@3e70e1d680 alike — although Actions are enabled, the repo is public and the workflow is active. Previous heads of this PR did run (bf13816 → success).

Structural findings (outside this PR)

pnpm gate (scripts/pre-merge-check.sh) is hardwired to origin/main: on any PR targeting develop_base it rebases onto upstream main and may auto-commit.
check:capability-tips diffs origin/main...HEAD, so a develop_base branch is charged with the whole fork delta. On the previous base (c29dba70ad) that made canonical pnpm check red on develop_base itself; on the current base (≈ main + 1398) the delta is small and it passes. It will return whenever develop_base diverges from main again.

Acceptance tail (open)

co-creator 阶段 4 acceptance + merge (hotfix rule: no self-merge)
After merge + operator restart: a live OTel-disabled fire emits exactly one SKIPPED notice and zero eval-cat invocations
develop_base branch protection requiring Fork-only patch guard (operator; blocked in practice until Actions run again)
Land in the carrier, not only develop_base. The rebuild rule is now latest main + replayed fork/optimizations (both refs at 8a90e142, and this HEAD fast-forwards both). Anything merged only to develop_base is dropped by the next rebuild — the mechanism behind all three F192 losses. After merging, fast-forward fork/optimizations to the same commit.
Rebuild procedure: before pushing a rebuilt develop_base, run node scripts/check-fork-only-patches.mjs --baseline-ref <pre-reset sha> — with Actions down, nothing else catches a drop.

🐾 Agent: @Opus (claude-opus-5-5)

🤖 Generated with Claude Code

…from outside the tree Why: 砚砚's review of PR #185 found two P1 defects. Both were real, and verifying them showed each fix was aimed at the wrong thing. ## P1 - the co-location gate was wired to unrelated configuration Confirmed all three claims: - `EVAL_BASE_URL` is read only by the manual CLI scripts/run-f168-external-case-eval.mjs. The scheduled path never consults it. - The port compare was wrong in the documented local case: with `EVAL_BASE_URL=http://localhost:3004` it evaluated `"3004" === "3002"` (and `"3004" === ""` with no override) - false. It fell CLOSED on exactly the local topology it was meant to admit. - `new URL('http://[::1]:3002').hostname` is `"[::1]"`, so the `::1` branch was dead code. Repaired by removal, not by fixing the comparison. The scheduled path has no remote evidence target to check: f167-runtime-eval evidence is a snapshot+attribution pair ON DISK under the harness-feedback root (publish-verdict/a2a-generator-adapter.ts resolves it filesystem-relative), written by this process's own OTel pipeline. The local handle is authoritative because the producer IS this process - structurally, not accidentally. The four old tests injected the boolean closure directly, so they passed while the wiring was broken. Replaced with regressions that pin the real contract, including the default-port and `[::1]` cases that used to fail closed. ## P1 - the guard could not report its own erasure Also confirmed: base aa40920 has neither the checker, the registry, nor the package.json wiring. A tree-only check reads its registry FROM the tree, so the rebuild that drops the patches drops the guard too, and nothing goes red. That is how three losses stayed silent. Root cause, from commit 70d79a2's own body: the rebuild is "reset to origin/main + fork/optimizations", and "443 files modified by BOTH main and fork: NOT restored" - which is precisely the requiredAnchors class. Carrier membership is NOT the fix: fork/optimizations is an orphan snapshot (7 commits, no merge-base with upstream main, a July full-tree copy), so demanding current code be committed onto a stale unrelated history is incoherent. Verified before building on it. What does survive a reset is the remote-tracking ref. `origin/develop_base` still holds the previous tree's registry - and this script - after the rebuild has erased both from disk. The guard now compares what the previous tree claimed to protect against what the new tree has, and stays runnable via `git show origin/develop_base:scripts/check-fork-only-patches.mjs`. Proven with a rebuild fixture that wipes registry + guard + patch and asserts the loss is still named, including the guard's own erasure. Also: every workflow in this repo triggers on `main`, so nothing ran on develop_base - the branch that loses the patches. Added a dedicated workflow there (no paths-ignore: a rebuild can drop a patch whatever its diff touches). ## P2 - anchors could be satisfied by imports and comments Added `order` anchors pinned to CALL SITES, so a gate moved after the step it must precede is caught; a bare symbol would have matched the import. Hollow registry entries (no id, or asserting nothing) and an emptied registry now fail closed. Verified: 57/57 tests (guard 16, evidence-gate 20, daily + prereq 21), api build, biome clean repo-wide, git diff --check clean. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

@Opus

…build loss Why: the F192 evidence-prerequisite gate has now been silently dropped by develop_base rebuilds THREE times (PR #91, PR #137, commit 70d79a2). The third loss went unnoticed for 21 days, and was caused by a commit whose own title was "recover fork-only development layer lost in develop_base rebuild" - a rebuild explicitly trying to recover fork-only code still missed it. Each loss lets the scheduler burn a full LLM session per daily eval fire, re-concluding a telemetry gap it cannot act on. Re-applying the patch a third time would not stop a fourth loss, so this change fixes the mechanism, not just the symptom. ## Durability (the actual root cause) Maintainer direction when closing upstream PR zts212653#1352 (2026-08-25): "this is fork-specific harness logic that doesn't belong in upstream. The rebuild-loss issue should be solved by improving our fork's develop_base rebuild SOP (rebase fork-only patches)." Implements exactly that, as a mechanism rather than a checklist: - scripts/fork-only-patches.json - registry of patches upstream does not carry - scripts/check-fork-only-patches.mjs - fails closed on missing files AND on lost in-file anchors (the silent case: shared file survives, patch reverted) - wired FIRST in `pnpm check`, so a rebuild that drops a patch goes red immediately instead of failing silently in production - docs/fork-only-patches.md - why the boundary exists, so it is not re-litigated Verified red: run against the current unfixed develop_base tree it reports all 11 violations with actionable paths; green on this branch. ## Gate restoration, with the upstream review findings fixed Upstream review of zts212653#1352 surfaced two real defects. Both are fixed here rather than carried forward blindly: 1. The probe read the SCHEDULER process's telemetry handle while evidence is fetched from the adapter target (EVAL_BASE_URL). These coincide in this fork's single-process deployment, but only accidentally. Added `evidenceTargetIsLocal`, asserted at bootstrap from EVAL_BASE_URL vs API_SERVER_PORT; an unprovable target now fails closed instead of letting a local handle speak for a remote source. 2. The skip notice always advised configuring TELEMETRY_HMAC_SALT even when the detected cause was a deliberate OTEL_SDK_DISABLED=true. Next-step text is now derived from the detected cause. ## Test-validity fix The restored suite asserted `invokeTrigger.trigger` call counts, but RFC 5.4 replaced that positional call with the deliver envelope (targetCatId + privateContent). Those assertions were passing vacuously - `callCount() === 0` is trivially true on every path now, so "cat never invoked" proved nothing. Migrated all 6 sites to count real wakes on the live seam; the happy-path test asserts 1 wake, so the invariant is falsifiable in both directions. Tests: 20/20 evidence gate, 29/29 daily+nday, biome + tsc clean. `pnpm check` remaining failures (check:capability-tips) reproduce identically on unmodified develop_base - pre-existing, unrelated. Agent: @Opus (claude-opus-5) Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

…from outside the tree Why: 砚砚's review of PR #185 found two P1 defects. Both were real, and verifying them showed each fix was aimed at the wrong thing. ## P1 - the co-location gate was wired to unrelated configuration Confirmed all three claims: - `EVAL_BASE_URL` is read only by the manual CLI scripts/run-f168-external-case-eval.mjs. The scheduled path never consults it. - The port compare was wrong in the documented local case: with `EVAL_BASE_URL=http://localhost:3004` it evaluated `"3004" === "3002"` (and `"3004" === ""` with no override) - false. It fell CLOSED on exactly the local topology it was meant to admit. - `new URL('http://[::1]:3002').hostname` is `"[::1]"`, so the `::1` branch was dead code. Repaired by removal, not by fixing the comparison. The scheduled path has no remote evidence target to check: f167-runtime-eval evidence is a snapshot+attribution pair ON DISK under the harness-feedback root (publish-verdict/a2a-generator-adapter.ts resolves it filesystem-relative), written by this process's own OTel pipeline. The local handle is authoritative because the producer IS this process - structurally, not accidentally. The four old tests injected the boolean closure directly, so they passed while the wiring was broken. Replaced with regressions that pin the real contract, including the default-port and `[::1]` cases that used to fail closed. ## P1 - the guard could not report its own erasure Also confirmed: base aa40920 has neither the checker, the registry, nor the package.json wiring. A tree-only check reads its registry FROM the tree, so the rebuild that drops the patches drops the guard too, and nothing goes red. That is how three losses stayed silent. Root cause, from commit 70d79a2's own body: the rebuild is "reset to origin/main + fork/optimizations", and "443 files modified by BOTH main and fork: NOT restored" - which is precisely the requiredAnchors class. Carrier membership is NOT the fix: fork/optimizations is an orphan snapshot (7 commits, no merge-base with upstream main, a July full-tree copy), so demanding current code be committed onto a stale unrelated history is incoherent. Verified before building on it. What does survive a reset is the remote-tracking ref. `origin/develop_base` still holds the previous tree's registry - and this script - after the rebuild has erased both from disk. The guard now compares what the previous tree claimed to protect against what the new tree has, and stays runnable via `git show origin/develop_base:scripts/check-fork-only-patches.mjs`. Proven with a rebuild fixture that wipes registry + guard + patch and asserts the loss is still named, including the guard's own erasure. Also: every workflow in this repo triggers on `main`, so nothing ran on develop_base - the branch that loses the patches. Added a dedicated workflow there (no paths-ignore: a rebuild can drop a patch whatever its diff touches). ## P2 - anchors could be satisfied by imports and comments Added `order` anchors pinned to CALL SITES, so a gate moved after the step it must precede is caught; a bare symbol would have matched the import. Hollow registry entries (no id, or asserting nothing) and an emptied registry now fail closed. Verified: 57/57 tests (guard 16, evidence-gate 20, daily + prereq 21), api build, biome clean repo-wide, git diff --check clean. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

…nked path Why: testing the recovery procedure I had just documented showed it did not work, and failed in the worst possible way. `docs/fork-only-patches.md` tells you to rescue the guard out of the surviving ref after a rebuild deleted it: git show origin/develop_base:scripts/check-fork-only-patches.mjs > /tmp/guard.mjs node /tmp/guard.mjs That printed nothing and exited 0. Two defects, both of which make the guard report clean while checking nothing: 1. Entrypoint detection compared `process.argv[1]` with `fileURLToPath(import.meta.url)` via resolve(). The first is the path as typed, the second is the REAL path, and resolve() does not follow symlinks. On macOS `/tmp` -> `/private/tmp`, so the comparison was false and main() never ran. Any invocation through a symlinked path silently no-ops. Now compared by realpath. 2. The repo root was script-relative, so a rescued copy in /tmp resolved the root to `/` and would have found no registry. The CLI now asks git for the toplevel from the working directory, with an explicit `--repo-root` override and the old script-relative path as last resort. A guard that exits 0 having checked nothing is exactly the failure this file exists to remove - the same shape as the three silent losses it is meant to catch - so all three paths are pinned by tests: invocation through a symlink, a rescued out-of-tree copy, and `--repo-root`. `--repo-root` also must not become a way to opt out of the survival half, so an explicit tree with no resolvable baseline still fails closed. Verified: 61/61 tests, biome clean repo-wide. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

…ding its own reversion Why: 砚砚's round-2 review was right that the durability path was still broken, and reproducing it found a worse case than the one reported. ## The reported defect, confirmed CI fetched `origin/develop_base` AFTER the push, so the "baseline" was the very tree being validated - a witness that is the accused. The fixture only ever proved a locally preserved old ref, never the post-push lifecycle. ## The worse case found while reproducing it The registry is itself a shared file, so a rebuild can REVERT it instead of deleting it. Reproduced end to end: a rebuild that rolls the registry back and drops the F192 gate leaves both trees internally consistent, and the guard reports OK - 1 fork-only patch(es) present; 1 claimed by origin/develop_base still present exit 0. It confirms its own amnesia as healthy. That is the exact disease this file exists to cure, so it is now the primary thing the check tests for. ## Fixes - Refuse to compare a commit with itself. A baseline that resolves to HEAD fails closed naming `--baseline-ref`, instead of reporting green. - Compare claim SETS, not just files: any patch id the baseline registered that is no longer registered is a loss, reported as a reverted registry. - `--baseline-ref` is now actually honoured (it was parsed for the repo root but never passed to the survival check). - CI passes the immutable pre-push commit - `github.event.before` for a push (survives force-push), `pull_request.base.sha` for a PR - and fetches it, since a force-pushed rebuild leaves the old tip unreachable. - Removed the `|| retry-without-baseline` fallback. A guard that quietly downgrades itself when its input is missing is the disease. `--no-baseline` now makes that skip an explicit, printed intent for the one honest case (branch creation). ## Scope stated honestly An in-tree workflow cannot survive its own deletion: if a rebuild drops workflow + guard + registry together, GitHub runs the workflow from the pushed commit where it no longer exists, so nothing runs. The doc no longer claims otherwise. It now names the authoritative external enforcement - branch protection on `develop_base` requiring the `Fork-only patch guard` check, so a check that never reports BLOCKS rather than passes - and records that this needs repo-admin rights and is not configured by this PR. Tests: post-push advanced baseline, reverted registry, `--baseline-ref`, `--no-baseline`. 65/65 total, biome clean repo-wide, workflow YAML parses. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Why: 砚砚's round-4 review found a merge-time regression in approved code. Once this PR lands, a clean develop_base checkout has HEAD == origin/develop_base. The survival check failed closed whenever its baseline resolved to HEAD, and the guard is the FIRST step of `check:sources` - so canonical `pnpm check` would be red on the very branch the guard protects, and nothing after it would run. The fail-closed rule was right for the caller it was written for and wrong for the other one. - An EXPLICIT --baseline-ref that resolves to HEAD means the caller mis-wired the witness. That is the round-2 CI bug, so it still fails closed. - The registry DEFAULT resolving to HEAD means there is no older tree in this checkout. HEAD now witnesses the WORKING TREE instead, so an uncommitted registry or patch reversion is still caught rather than skipped. The default path cannot see a loss that was already committed, and now says so in its output instead of implying a verification. That is why CI passes the immutable pre-reset commit, and why the registry now anchors that wiring in the workflow (`--baseline-ref`, `github.event.before`, `pull_request.base.sha`): deleting it turns the TREE check red rather than letting CI quietly fall back to the default. `--no-baseline` is untouched and still reserved for branch creation. Tests, red before the change (4 failing, the explicit-HEAD one already green): - clean develop_base tip: default invocation passes - explicit --baseline-ref HEAD: still fails closed - at the tip, HEAD catches an uncommitted registry reversion and a dropped file - the workflow keeps passing an explicit immutable baseline (registry anchor) - the existing "honours --baseline-ref" test now asserts the default is honest ("no older baseline") instead of asserting it fails Mutation: forcing the explicit branch open turns exactly two tests red, the round-2 regression test and the new explicit-HEAD test, so the CI defence still bites. 28/28 guard tests, biome clean, git diff --check clean. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

mindfn mentioned this pull request Sep 28, 2026

F117 baseline B：流式时 effect 不再每次都排更新（修复 Maximum update depth） #203

Merged

mindfn force-pushed the develop_base branch from d35f3b8 to 3e70e1d Compare October 9, 2026 08:55

This was referenced Oct 9, 2026

A2A acceptance: original F117 lifecycle branch into develop_base #217

Merged

fix(F117): target-based guidance and post-restart delivery feedback #218

Merged

mindfn force-pushed the fix/f192-evidence-gate-durable branch from bf13816 to 6f98939 Compare October 11, 2026 03:08

mindfn force-pushed the develop_base branch from 7adb57d to dc582b8 Compare October 13, 2026 06:39

mindfn force-pushed the fix/f192-evidence-gate-durable branch from f62a9fc to 4744926 Compare October 14, 2026 03:11

mindfn force-pushed the develop_base branch from 73a1009 to 8a90e14 Compare October 20, 2026 08:24

mindfn and others added 5 commits October 20, 2026 01:35

mindfn force-pushed the fix/f192-evidence-gate-durable branch from 4744926 to 7bcd70c Compare October 20, 2026 08:45

Navigation Menu

Repository navigation

fix(F192): restore evidence gate + guard fork-only patches against rebuild loss - #185

fix(F192): restore evidence gate + guard fork-only patches against rebuild loss#185
mindfn wants to merge 5 commits into
develop_basefrom
fix/f192-evidence-gate-durable

mindfn commented Sep 22, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

mindfn commented Sep 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Five-axis risk

What this fixes

The durability mechanism

Known residual gap (not claimed closed)

Review history

Merge-gate evidence manifest

CI — not green, and not claimed

Structural findings (outside this PR)

Acceptance tail (open)

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

mindfn commented Sep 22, 2026 •

edited

Loading