Merge pull request #4408 from huangruiteng/codex/bounded-local-authority-profile

refactor(authority): bound the retained sqlite state log

Author: huangruiteng

.github/workflows/python-tests.yml

@@ -231,7 +231,7 @@ jobs:
           tests=()
           for test in tests/control_plane_ts/*.test.ts; do
             case "$test" in
-              */sqlite_authority_store.test.ts|*/local_authority_provider.test.ts|*/authority_provider_parity.test.ts|*/sqlite_capacity.test.ts) continue ;;
+              */sqlite_authority_store.test.ts|*/sqlite_authority_bounded_profile.test.ts|*/sqlite_authority_migration.test.ts|*/local_authority_provider.test.ts|*/authority_provider_parity.test.ts|*/sqlite_capacity.test.ts) continue ;;
             esac
             tests+=("$test")
           done

docs/architecture/rfcs/shared-goal-authority-state-provider-v0.md

@@ -1231,18 +1231,29 @@ is gated by evidence below, not by calendar dates or this PR's merge status.
 | New-Goal default decision (F) | Maintainers accept the qualified profile and canary results, operational diagnostics, backup/restore procedure, release instructions and default-disable path. Ship the default change in a separate disclosed release change. | Apply only to newly created eligible local Goals. Existing explicit file selections remain pinned. Unsupported runtimes/filesystems require an explicit supported choice; no silent backend switch on open failure. |
 | Existing-Goal migration and file retirement | Migrate opt-in cohorts using the reviewed fenced workflow; reconcile receipts, history, projections and rollback after each cohort. Inventory the last file-primary callers and compatibility windows before removing any path. | Each Goal needs explicit migration authority. Retire file as the ordinary primary only after that evidence; retain reference/import/export support until its own callers and retention duties end. |
 
-**Current evidence position (rechecked 2026-09-13).** #4121 merged as
+**Current evidence position (rechecked 2026-09-15).** #4121 merged as
 `bde1632bb6f29aeb9a8b4ac23ead3e98ba2f2f55`, delivering the first candidate
-milestone. It remains subject to profile qualification and promotion; it is
-not completion of lane L. Its head pointer is bounded and
-operation/cursor lookups are indexed, but it retains full historical projections
-and counts a covering index for continuity. That count grows with history;
-current/accessed-row digests are checked, not every historical payload per read.
-The qualification entrypoint now separates a small rehearsal from an explicit
-64-KiB 10k/100k storage axis, with p99/counts, cold CLI, RSS and a
-passed/failed/missing ledger. Unavailable logical/WAL traffic, full-domain,
-large-history recovery and elapsed-soak evidence remain holds; runner completion
-cannot claim the <=2 growth budget or ten-day qualification. See the
+milestone. The bounded retained-storage half of lane L now has a reviewed
+candidate: `loopx_sqlite_authority_store_v2` keeps one checkpoint per 64-commit
+window plus one exact state delta per commit instead of one full projection copy
+per row, proves the live head from the head row, its retained transaction and
+the cursor bounds, rebuilds at most one window per historical read, and proves
+the complete delta chain through `verifyAuthorityHistory`. Cursors, operation
+IDs, commit digests, provider revisions, receipts, events and scan pages are
+unchanged, and the shipped version-1 databases migrate through the reviewed
+`examples/coordination/sqlite-authority-migration.ts` entry point. Rehearsal
+evidence at 1,000 commits/64 KiB now reports 16 checkpoints, a 63-commit replay
+budget, one checkpoint history read and 1,048,576 retained projection bytes plus
+126,714 delta bytes against 65,536,000 bytes for one copy per commit.
+
+This is still not completion of lane L. File and NoKV continue to retain and
+decode their complete journal on every load, so bounded recovery is a property
+of the embedded candidate rather than cross-provider parity; the SQLite profile
+also still retains receipts and events without pruning. Unavailable logical/WAL
+traffic and the <=15x cumulative write-growth budget, 1 MiB and 300k headroom,
+full-domain workload, large-history recovery, fenced backup/restore, supported
+upgrade/rollback, OS/runtime coverage and the >=10-day elapsed soak remain
+holds, and runner completion cannot claim them. See the
 [SQLite qualification commands](../../reference/sqlite-authority-store.md#reproduce-validation).
 The public minimum remains Node 22.18 for File; SQLite additionally requires
 synchronous finalization and the WAL-reset fix, with Node 22.22.3/SQLite 3.51.3

docs/architecture/rfcs/shared-goal-authority-state-provider-v0.zh-CN.md

@@ -970,15 +970,23 @@ fencing/export 演练与 maintainer review 都通过才可晋升。发布紧凑
 | 新 Goal 默认决策（F） | 维护者接受合格 profile、canary 结果、运维诊断、backup/restore 流程、发布操作说明和关闭默认的路径；在独立且明确披露的发布改动中切默认。 | 仅适用于新建且符合条件的本地 Goal；已有显式 file 选择保持固定。不受支持的 runtime/filesystem 需显式选择支持方案，打开失败不能静默切 backend。 |
 | 已有 Goal 迁移与 file 退役 | 按已评审的 fenced workflow 逐批 opt-in 迁移，每批核对 receipt、历史、投影和回滚；删除路径前列清最后的 file-primary caller 与兼容窗口。 | 每个 Goal 需要明确迁移权限；证据满足后才退役常规 primary 角色。参考／导入／导出支持保留到其 caller 与保留责任分别结束。 |
 
-**当前证据位置（2026-09-13 复核）。** #4121 已合并为
-`bde1632bb6f29aeb9a8b4ac23ead3e98ba2f2f55`，交付第一个候选节点；
-仍需 profile 资格化与晋级，不代表 lane L 完成。
-其 head pointer 有界，operation/cursor 查询有索引，但保留完整历史 projection，连续性
-校验还会统计覆盖索引，因此该成本随历史增长。它验证当前及访问到的 row digest，
-不是每次读取都审计全部历史 payload。资格入口现在区分小型 rehearsal 与显式
-64 KiB 10k/100k 存储轴，记录 p99/样本数、cold CLI、RSS 和 passed/failed/missing
-账本。逻辑/WAL 流量、完整领域负载、大历史恢复和自然时间 soak 的缺口仍阻止晋升，
-工具跑完不等于通过 <=2 增长预算或十天资格。参见
+**当前证据位置（2026-09-15 复核）。** #4121 已合并为
+`bde1632bb6f29aeb9a8b4ac23ead3e98ba2f2f55`，交付第一个候选节点。
+Lane L 中“保留存储有界”的那一半现在有可评审候选：`loopx_sqlite_authority_store_v2`
+改为每 64 次提交一个 checkpoint、每次提交一条精确 delta，不再为每一行保留一份完整
+projection；活跃头由 head 行、对应保留事务和游标连续性自证；单次历史读取最多重建一个
+窗口；完整 delta 链由 `verifyAuthorityHistory` 线性证明。cursor、operation id、commit
+digest、provider revision、receipt、event 与 scan 页面字节均未改变，已发布的版本 1
+数据库通过评审过的 `examples/coordination/sqlite-authority-migration.ts` 入口迁移。
+rehearsal 证据（1000 次提交／64 KiB）现为 16 个 checkpoint、63 次 replay 预算、
+单次历史读取最多重建一个 64 次提交窗口，保留 1,048,576 字节 projection 与
+126,714 字节 delta，而“每提交一份完整拷贝”为 65,536,000 字节。
+
+这仍不代表 lane L 完成。File 与 NoKV 依旧在每次 load 时保留并解码完整 journal，
+因此“有界恢复”是内嵌候选 provider 的性质，不是跨 provider 等价；SQLite profile
+也仍不裁剪 receipt 与 event。逻辑/WAL 流量与 <=15x 累计写入预算、1 MiB 与 300k
+headroom、完整领域负载、大历史恢复、fenced backup/restore、受支持升级/回滚、
+OS/runtime 覆盖和 >=10 天自然时间 soak 仍是 hold，工具跑完不能声称已满足。参见
 [SQLite 验证命令](../../reference/sqlite-authority-store.md#reproduce-validation)。
 公开 Node 最低版本 22.18 继续用于 File；SQLite 另需同步 finalization 与 WAL-reset
 修复，参考组合为 Node 22.22.3／SQLite 3.51.3。Node 24 主 runtime 与 Node 26

docs/reference/sqlite-authority-store.md

@@ -2,8 +2,9 @@
 
 SQLite is an **opt-in local conformance candidate**, behind the existing
 TypeScript `AuthorityStore` interface. File remains the default. This slice
-does not promote a goal, migrate existing authority, enable cross-host writes,
-or qualify ten elapsed days of operation.
+does not promote a goal, run a live cutover, enable cross-host writes, or
+qualify ten elapsed days of operation. It does provide the explicit
+version-1 to version-2 database migration described below.
 
 ## Placement and persistence
 
@@ -18,24 +19,37 @@ directory. Metadata binds the goal, schema version and random database
 incarnation. Provider revisions combine that incarnation with a monotonic
 integer sequence; they are not authority revisions or lease epochs.
 
-The version-1 schema contains:
+The version-2 schema contains:
 
 | Table | Contract |
 | --- | --- |
 | `metadata` | Version and database/goal identity |
-| `head` | One bounded pointer to the current committed projection |
-| `commits` | Unique operation ID, canonical commit digest, ordered cursor, original receipts, events and full projection |
+| `head` | The live committed projection, its state digest and one cursor |
+| `commits` | Unique operation ID, canonical commit digest, ordered cursor, original receipts and events, one exact state delta, its state digest and parent state digest |
+| `checkpoints` | One full projection and its digest per bounded window |
 
 `commits` also serves as the durable projection outbox used by
 `scanCommitted`. There is no independent ACK or second receipt authority.
 Existing consumers resume by cursor. A unique operation index makes receipt
-lookup and cursor paging indexed. The common continuity check counts the compact
-covering index, so total read/write cost is not independent of history length.
-It does not deserialize the complete retained payload history. Historical receipts and full projections are retained without
-pruning. Fixed live state therefore produces linear database growth, not
-bounded total disk use. Growing application projections require separate
+lookup and cursor paging indexed.
+
+Retention is bounded by window instead of by history: every commit keeps one
+exact delta, and one full projection is retained per checkpoint window
+(`authority_state_log.ts`, 64 commits per window). A live read resolves the head
+from the head row, its retained transaction and the cursor bounds; a historical
+read rebuilds at most one window from the covering checkpoint. Retained deltas
+are therefore the only part that still grows with history, and their size is
+proportional to what each commit changed. Original receipts and events are
+retained without pruning, so fixed live state with large receipts still grows
+with history. Growing application projections require separate
 retention/compaction work.
 
+The rehearsal profile measures this profile directly: at 1,000 commits with a
+64 KiB projection, 16 checkpoints retain 1,048,576 projection bytes and 126,714
+delta bytes, against 65,536,000 bytes for one full copy per retained commit.
+Formal 10k/100k evidence still requires the separately authorized matched
+profile.
+
 Writes use `BEGIN IMMEDIATE`, a five-second busy timeout, WAL and
 `synchronous=FULL`. The head, receipt, events and outbox row commit together.
 Before-COMMIT failures roll back; a COMMIT error reports an ambiguous outcome
@@ -49,18 +63,29 @@ rotation, corruption repair, or network-filesystem sharing is supported.
 
 ## Read integrity
 
-Authority reads share one SQLite snapshot for metadata, head and requested rows.
-The same check runs inside the write transaction before any new commit row:
-positive unique integer cursors must have `min=1` and `count=max=head`. Thus a
-missing head, rolled-back head, or internal cursor gap is rejected as
+Authority reads share one SQLite snapshot, and writes run the same live proof
+inside their transaction before publishing a new commit row. The proof is
+layered so that each layer pays only for what it returns:
+
+| Layer | Proves | Cost |
+| --- | --- | --- |
+| Live head (`loadAuthority`, `commitAuthority`) | Head row digest over the live projection, the retained transaction at that cursor reproducing its exact commit digest, parent linkage, `min=1`/`count=max=head` cursor continuity, and the presence of the checkpoint that covers the head | One head row, one retained row, one parent digest and index lookups; independent of retained history |
+| Materialized history (`scanCommitted`, `readReceipt`) | Every row from the covering checkpoint through the requested span, including each delta, state digest and parent lineage; paged scans also prove the lookahead row used for `has_more` | At most one checkpoint window plus the requested span |
+| Archive audit (`verifyAuthorityHistory`) | The complete delta chain from the empty root, every checkpoint against retained history, and the final state against the head | Linear in retained history; qualification and recovery only |
+
+A missing head, rolled-back head, internal cursor gap, rewritten receipt/event,
+orphaned parent digest or mismatched state digest is rejected as
 `provider_protocol_violation` before returning authority or accepting a write.
+Preparing a commit also re-applies its own delta and requires byte equality with
+the committed projection before anything is written.
 
-The newest row's canonical commit digest is recomputed on every authority read
-and write. Historical receipt reads additionally validate their selected row;
-paged scans validate each returned row and the lookahead row used for
-`has_more`. The digest includes the operation ID, projection, events, receipts
-and expected predecessor revision, reconstructed from the unchanged v0 sequence
-contract. No schema migration or alternate digest format is introduced.
+Two shapes are deliberately outside the live proof because the live head never
+reads them: the delta of the newest retained row, and the projection of the
+checkpoint the live head resumes from. Both are refused by every read that
+materializes their span and by the archive audit, and neither can change the
+authority value a live read returns. The commit digest is unchanged from v0
+(operation ID, projection, events, receipts, expected predecessor revision), so
+cursors, provider revisions and stored digests stay comparable.
 
 This is integrity validation of the current and accessed evidence, not a full
 cryptographic audit of every historical payload on each call. Unaccessed older
@@ -192,6 +217,10 @@ npm ci --ignore-scripts
 npm run typecheck:control-plane
 node --no-warnings --experimental-sqlite --experimental-strip-types --test \
   tests/control_plane_ts/sqlite_authority_store.test.ts \
+  tests/control_plane_ts/sqlite_authority_bounded_profile.test.ts \
+  tests/control_plane_ts/sqlite_authority_migration.test.ts \
+  tests/control_plane_ts/authority_state_log.test.ts \
+  tests/control_plane_ts/authority_provider_parity.test.ts \
   tests/control_plane_ts/local_authority_provider.test.ts \
   tests/control_plane_ts/sqlite_runtime_admission.test.ts \
   tests/control_plane_ts/sqlite_capacity.test.ts
@@ -251,8 +280,42 @@ The real-process regressions exercise SIGKILL before and after business COMMIT,
 lost-response receipt readback, exact head/event/receipt/scan equivalence and
 SQLite `max_page_count` exhaustion. These are small disposable-database tests,
 not power-loss, operating-system ENOSPC or large-history recovery qualification.
-The source uses the shared retained-journal snapshot contract. No checkpoint,
-retention deletion, restore-incarnation change or migration format is added.
+Retention deletion, restore-incarnation change and cross-host sharing are still
+absent from this slice.
+
+## Version-1 migration
+
+The shipped version-1 database keeps one full projection per retained row, so it
+cannot be read by the version-2 provider. `sqlite_authority_migration.ts`
+migrates one Goal database in place: it reads the frozen version-1 rows, proves
+every stored commit digest, writes the checkpoint/delta log, proves that each
+written delta reconstructs its projection, requires the commit count to match,
+swaps tables and updates the schema version inside a single
+`BEGIN IMMEDIATE` transaction. Any failure rolls back and leaves version 1
+untouched; a second run reports `already_current`; a rewritten proof, a
+mismatched goal/incarnation or an existing swap target fails closed. Cursors,
+operation IDs, commit digests, provider revisions, receipts, events and scan
+pages are byte-identical after the migration.
+
+A version-1 database that published only its schema and metadata — the state a
+goal leaves behind when it selected the provider and never committed — migrates
+to an equally empty version-2 database instead of failing, so the operator is
+never left with a database that neither provider accepts.
+
+Run it from the repository checkout, with `--execute` omitted for a safe plan:
+
+```sh
+node --no-warnings --experimental-sqlite --experimental-strip-types \
+  examples/coordination/sqlite-authority-migration.ts \
+  --directory "$RUNTIME_ROOT/authority/sqlite-v0" --goal-id example
+```
+
+Add `--execute` (optionally with `--expected-identity`, which refuses a database
+whose stored incarnation is not the one the operator names) to migrate. The
+entry point rewrites only that Goal's database; it does not change provider
+selection, promote a goal, or enable cross-host writes. Keep the pre-migration
+database copy until the promoted Goal has been validated. A production cutover
+command, migration manifest and reverse export remain separate deliverables.
 
 ### Qualification holds / 资格保留项
 
@@ -264,6 +327,12 @@ backup/restore, supported upgrades/rollback, OS/runtime coverage and a real
 >=10-day soak. Those holds still block profile promotion. Accelerated volume
 never substitutes for elapsed time, and running this command starts no soak.
 
+Retained state is measured where the formal profile runs: an axis reports its
+checkpoint count, replay budget, recovery tail and retained projection/delta
+bytes, and `bounded_retained_state` compares them against one full copy per
+retained commit. That row stays `missing` for a rehearsal, exactly like every
+other formal budget.
+
 SQLite 资格参考使用 Node 22.22.3／SQLite 3.51.3；打开前同时检查实际 WAL 修复版本
 和 statement 关闭行为。公开 Node 最低版本 22.18 继续用于默认 File 路径。显式
 SQLite 选择遇到不合格 runtime 会拒绝，不会改默认 provider 或静默回退。
@@ -278,3 +347,12 @@ RSS 和文件大小；没有量到的累计 WAL／逻辑写入和纯锁等待保
 耗尽、长期 consumer backlog 或完整恢复验证。首批测量允许保留 failed/missing；
 >=10 天自然时间 soak、迁移和晋升分别评审与授权。本入口不改变持久格式、Todo
 语义、默认 provider 或任何活跃 Goal。
+
+版本 2 把“每个提交都保留一份完整投影”改成“每个窗口一个检查点 + 每提交一条精确
+delta”：活跃头读取只用自己的行、对应提交和游标连续性自证，历史读取最多重建一个
+窗口，完整归档由 `verifyAuthorityHistory` 线性审计。版本 1 数据库需要显式迁移
+（`examples/coordination/sqlite-authority-migration.ts`，默认只做 plan，`--execute`
+才写入，`--expected-identity` 可拒绝并非操作者所指的 incarnation，失败保持 v1
+原样）。只发布过 schema 与 metadata、从未提交的 v1 库会迁移成同样为空的 v2 库，
+不会让操作者落在两个 provider 都不接受的状态。迁移不改 cursor、operation id、
+commit digest、provider revision、receipt、event 或 scan 页面字节。