Skip to content

perf(authority): reuse File proofs and bound quota observation output - #5222

Open
huangruiteng wants to merge 3 commits into
mainfrom
codex/goal-read-cost-0928
Open

huangruiteng wants to merge 3 commits into
mainfrom
codex/goal-read-cost-0928

Conversation

@huangruiteng

Copy link
Copy Markdown
Collaborator

Repeated observations of two File-backed Goals were evicting each other's single cached history proof. Quota status/plan also emitted complete Todo lists even though should-run already had bounded display helpers. This change addresses those two measured costs within the local-authority qualification work in the retirement checkpoint.

  • File now retains up to four verified stores in LRU order under one 128 MiB encoded-data budget, retaining only a bounded head/receipt view for large journals. Every hit still reads/hashes the bytes and checks store identity. Failed publication invalidates only that store. The TS authority contract, on-disk format and original receipts are unchanged.
  • Intentional CLI default change: quota status/plan JSON uses the existing Todo summary projections after complete planning. Counts, quota ordering/decisions, health and omission markers remain available. Consumers needing complete item metadata use --include-detail agent-todos, user-todos, or all. Markdown is unchanged. A shared command/section map owns validation and all expansion; combining all with an unsupported section now fails before collection.
  • The CLI presentation adapter remains Python; File proof ownership remains TypeScript. The display hook is CLI-only, so App/Lark and shared semantic API payloads need no companion change. No new decision owner, provider default, migration or legacy-writer removal is introduced. This is independent of fix(quota): scope read-only plans to the selected Goal #5220's Goal-selector fix.

On detached copies retaining 379 original commits, alternating two File stores took 6.30–6.49 seconds per read before; after each store's first proof, the candidate took 9.8–11.2 milliseconds with matching hashes/cursors. First proof remains about 6 seconds. Three fresh-process samples measured File head reads at 5.98–6.32 seconds and SQLite at 34.5–36.0 milliseconds; these are different integrity workloads, not full-history throughput parity. A captured single Goal JSON row shrank from 1,252,747 to 78,688 UTF-8 bytes, and explicit detail restores it. Display size does not measure collection latency.

Validation on 8b1e25849bdadccd366cd67732378e7f1489f56a:

  • 17 Python tests: real File/SQLite CLI, exact Unicode Todo metadata, selective/full detail, unchanged Markdown, negative selectors and existing projections.
  • 310 File/journal tests and 347 SQLite/snapshot/runtime tests pass, including corruption, identity, eviction and original-receipt recovery.
  • A 148-second isolated run performed 12 fresh-process writes per provider across a checkpoint, checking receipt replay, changed-intent rejection and projection/hash parity at each step. No active store was modified. This is not the ten-day D2 soak.
  • TS typecheck, repository-configured mypy, changed-path Ruff, whitespace and public-boundary checks pass. An exploratory expanded mypy run reports the same 4,179 diagnostics on base and head (zero new diagnostics); the configured 19-file gate is clean.
  • Exact-scope change-quality receipt is valid; standard premerge is ready (19 selected checks, zero blocking failures). It preserves one inherited advisory: loopx.status has 119 compatibility reexports against a 117 ceiling on the pinned base; newer main fixes that in 8444845. No budget was raised here. Full PR CI remains required.

Remaining qualification: cold File verification, aggregate RSS, sustained workload/capacity/platform coverage, and consumer/onboarding upgrade acceptance. SQLite is a stronger long-history candidate, but these observations do not change the release default or authorize legacy-writer deletion. Raw snapshots and one-off diagnostic scripts are excluded.

Signed-off-by: huangruiteng <14976749+huangruiteng@users.noreply.github.com>
Signed-off-by: huangruiteng <14976749+huangruiteng@users.noreply.github.com>
Signed-off-by: huangruiteng <14976749+huangruiteng@users.noreply.github.com>

@huangruiteng huangruiteng left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approval conclusion (author-owned PR; GitHub blocks formal self-approval)
Reviewed exact head: 8b1e25849bdadccd366cd67732378e7f1489f56a
Comparison merge-base: 82dfe2e2c8c85d3fd63490313597886eba709aa0

动机

这份 PR 对应 shared-goal authority B 本地档的两个可量化观察成本:多 Goal 交替使用 File store 时,单项缓存不断失效;quota status/plan 默认 JSON 带回完整 Todo 数组,输出随项目规模膨胀。它是有界性能修复,不把 File 改成默认提供者,也不宣称完成 D2 十天稳定性验收。既有 checkpoint 的后续多日比较仍待完成。

改动思路

沿用两个既有 owner:quota planner 先对完整权威状态作决策,最后才生成有界 JSON 摘要;需要旧完整数组的 caller 可用 --include-detail agent-todos|user-todos|all 取回。FileAuthorityStore 把单项已验证读视图改成至多四库、128 MiB 编码数据预算的 LRU;命中仍重读文件、计算 digest 并核对 store identity,绝不让缓存决定写入或替代持久 journal。没有新增第二个资格/权限 owner。

具体改动

完整 PR 是 13 文件、+346/-37:现有 CLI/投影与 File store 小幅修改;新增 7 项 quota 观察测试、两项 File 交替/LRU 测试;文档同时说明旧/新默认、full-detail 冷路径、缓存界限和 D2 未闭合。CLI 参数按命令白名单校验,all 不能夹带不属于该命令的 selector;Markdown 路径保持原样。File 缓存超过四库或预算即逐出最久未用项,篡改或身份不符会重新验证并拒绝。

新增测试未改动公共 journal 格式或 quota 决策逻辑。

对主干的风险

最重要的兼容风险是默认 JSON 少了逐条 Todo,不能把它称为完全无变化。真实 File/SQLite CLI 测试证实完整 detail 保留原 note/status,摘要保留 count/plan,Markdown 输出相同;公开 quota 文档和 help 明示这项默认变化。独立复核又用同一个 12-Todo synthetic File Goal 跑旧/新 CLI:旧版 status/plan 完整输出分别 115,667/115,665 字节,新版 --include-detail all 与旧版逐条 Todo、quota 和 summary 一致;新版默认分别 16,298/16,292 字节。未发现决策或持久状态变化。

缓存方面,我把 head 新增的两项测试原样放到一次性的不可变基线测试位置运行:旧单项缓存分别出现验证计数 6(期望 2)与 8(期望 5),精确 head 两项通过;篡改一个 File store 不影响另一个,五库场景证实逐出。基线/head 原有 quota suite 各 26 passed,File store suite 302/304 passed;head 新 quota 测试 7 passed、完整 TS 控制面 3359 passed/30 skipped/0 failed、Ruff 与 TS typecheck 通过。最初隔离 worktree 缺 pg 时的八个导入失败在基线/head 同形,npm ci 后消失,不计为 PR 缺陷。未查询或等待远端 CI;本机无 PostgreSQL 测试 URL,30 项跳过及多日 D2 均不外推成通过。

我的整体评价

APPROVE(author-owned PR,故以 COMMENTED review 记录正式结论,不伪造 GitHub self-approval)。此变更改善本地档反复观察成本,同时保留权威读验证与完整详情回退;用基线反例和真实 CLI 双后端验证了核心承诺。Future-facing pass 看过相邻 owner:命令/section 映射与既有摘要 helper 已复用,继续提取抽象没有当前价值;D2 sustained File/SQLite 对照仍由既有 checkpoint 负责。它更动 runtime/control-plane 行为,按仓库规则应留给维护者合并,我不自合并。

English verdict: APPROVE - 8b1e258. The bounded display and four-store verified-view cache retain decision/authority semantics. Same-fixture CLI and base-vs-head counterfactuals passed; prior missing-pg worktree failures were setup-only. D2 soak and unconfigured PostgreSQL integration are not certified; no remote CI or merge action was used.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant