proof: sparse strict reads crash; dangling derives_from edges commit and vanish

[toeknee-figma]

Describe the bug

Two independent @flatbread/proof@1.0.0 / flatbread@1.0.0 consumer failures block strict provenance/closure use:

1. A verified sparse Proof accepts valid writes, but its immediate strict read exits with a raw ENOENT while scanning the first untouched collection directory.
2. WriteDecision accepts a valid-shaped nonexistent derives_from target, advances the durable generation, and the bounded relations read then silently omits the stored dangling edge.

Both require consumers to duplicate storage or relation-integrity machinery that the Proof writer/read surface is expected to own.

Reproduced on Node 22.22.0 with the npm release tagged v1.0.0:

• @flatbread/proof@1.0.0
  • SHA-512: sha512-azPZYIP0JF1vLUb+t/BkBDiIcT1rohd9Fy3/vpMe5nIUhi96HPvgggh7jeRlfv0bGJnlFUOQqZE3DSrUOWFFvg==
  • SHA-1: e586b9a12f39be9ba3d11736f55350a2ff92f06b
• flatbread@1.0.0
  • SHA-512: sha512-PvD0RdU/aef9BHHPzvL7XEAMq00NUeJcS1wkC8x0G7Lpx9mOZnSwRESBDAmIybI10XTt3EJuinh78NJ1/zhvJA==
  • SHA-1: 2de572960defbf382254bc86d9aaafcd8e2e1f66
• exact transitive @flatbread/core@1.0.0
  • SHA-512: sha512-hpQN4/lhpzNl5MBE3BwkK5hcs/xL1Xy/p+ZRuXS7QIZpHSHXF3sS3eP4c/ZG/xLBEJMJKcVtQTNIROApadua2w==

Reproduction 1: strict read fails on a valid sparse Proof

In a new Git project, install the versions above and create the documented configuration:

// flatbread.config.js
import {
  defineConfig,
  sourceFilesystem,
  transformerMarkdown,
  proofContent,
} from "flatbread";

export default defineConfig({
  source: sourceFilesystem(),
  transformer: transformerMarkdown(),
  content: [...proofContent(".flatbread-proof")],
});

Add the documented ignore rules:

**/.flatbread-proof/.journal/
**/.flatbread/proof/read-cache/

Then run:

npx flatbread proof bootstrap --verify

npx flatbread proof write \
  '{"type":"CreateEffort","title":"Critical path","body":"Exercise storage and consistency.","produced_in":"gate0-critical","created_by":"harnessflow-gate0"}'

npx flatbread proof write \
  '{"type":"WriteIssue","effort":"<created-effort-id>","title":"Need a decision","body":"A blocker for the strict-read path.","kind":"blocker"}'

npx flatbread proof records <created-effort-id> \
  --kinds issue \
  --strict-min-generation <issue-write-generation> \
  --timeout-ms 3000

bootstrap --verify reports status: "ready"; both writes exit 0 and advance the generation. The strict read exits 1 with a raw Node stack trace ending in:

ENOENT: no such file or directory, scandir '<root>/.flatbread-proof/findings'

Only collections touched by a write have directories. The reader nevertheless scans all eight configured collection paths. Empty directories also cannot survive Git, so a fresh clone remains vulnerable even if local setup once created them.

Expected: verified bootstrap plus valid writes produce a readable sparse graph. Untouched collections are treated as empty or initialized through a supported lifecycle operation. Errors use the structured CLI envelope rather than a raw ENOENT stack.

Reproduction 2: dangling derives_from commits, then vanishes

To isolate this from reproduction 1, create all eight collection directories first. Create an Effort, then run:

npx flatbread proof write \
  '{"type":"WriteDecision","effort":"<existing-effort-id>","title":"Dangling derives-from","body":"This must have failed closed.","derives_from":["fnd-does-not-exist--0000000000000000"]}'

The command exits 0 and advances the durable generation. The public API snapshot retains the exact stored edge:

const snapshot = await buildProofSnapshot(root);
snapshot.getRecord(decisionId).frontmatter.derives_from;
// ["fnd-does-not-exist--0000000000000000"]

But this strict relation read also exits 0 while returning page.returned: 0, without an integrity anomaly:

npx flatbread proof relations \
  <existing-effort-id> <created-decision-id> \
  --relations derives_from \
  --strict-min-generation <decision-write-generation>

Expected: the missing forward target is rejected before commit. If legacy or corrupt data already contains a dangling edge, relation recall fails closed or explicitly reports it instead of silently presenting incomplete provenance.

Related context: #207 describes committed-index target validation, and #214 implements strict-generation synchronization; neither covers these reproductions.

Logs

# Reproduction 1 terminal failure
Error: ENOENT: no such file or directory, scandir '<root>/.flatbread-proof/findings'

# Reproduction 2 observed result
WriteDecision: exit 0, generation advanced
buildProofSnapshot(...).getRecord(decisionId).frontmatter.derives_from:
  ["fnd-does-not-exist--0000000000000000"]
proof relations ... --relations derives_from:
  exit 0, page.returned = 0

The full local characterization contains 13 executable assertions covering these failures plus every documented mutation, locking, recovery, paging/caps, Git revert, roots/config, and promotion limits. It passes because it asserts the observed blockers; its machine gate verdict is blocked.

System Info

System:
  OS: Linux 6.17 Ubuntu 22.04.5 LTS (Jammy Jellyfish)
  CPU: (16) x64 AMD EPYC 7R13 Processor
  Container: Yes
  Shell: 5.8.1 - /bin/zsh
Binaries:
  Node: 22.22.0
  npm: 10.9.4
  pnpm: 10.27.0
npmPackages:
  @flatbread/proof: 1.0.0
  flatbread: 1.0.0

Proposed acceptance criteria

• A fresh verified Proof containing only an Effort and Issue completes the strict records query without manually creating unrelated collection directories.
• Missing collection directories are handled as empty or created by a supported lifecycle operation.
• CLI failures retain the documented structured error shape; no raw ENOENT stack is exposed.
• WriteDecision rejects a nonexistent derives_from target before commit.
• Relation recall never silently omits a stored dangling edge; legacy/corrupt data yields an explicit integrity error or anomaly.
• Regression tests cover both sequences.
• Fixes ship in exact-pinned Proof and CLI releases so consumers can rerun their characterization suite without directory scaffolding or relation prevalidation.

Severity

blocking an upgrade — specifically, blocking adoption of Proof 1.0.0 for strict provenance and evidence-closure workflows.