Repository navigation
ci(dogfood): prune the per-sha Turbo seed to the run's tasks before saving it - #30241
Conversation
The dogfood job's "Save Turbo cache (main only)" saved the whole restored .turbo/cache per sha, so each seed was every seed before it plus one run (~3.69 GB per Linux-turbo-dogfood-1-main-* seed, 95% of the Actions cache pool in the latest reading). Wire it behind scripts/ci/prune-turbo-cache.mjs as Test Core and Build Core are: the test step now writes a run summary (--summarize, no task hash moves), the new prune step keeps the entries that summary names (the closure's build tasks and dogfood#test), and the save runs only when the prune says save=true. Restore keys are unchanged. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017ihL2ebbZkr2QAEG1x7UC8
|
domain:devx#2 · marchtian · session_017ihL2ebbZkr2QAEG1x7UC8 Contract reviewServed-tier: Inputs read: card #25452 body and all 25 comments; PR #30241 body, file list (one file, Check-runs on ① Derived judgmentsThe diff is three edits inside the
② Semver levelThe diff publishes nothing: one workflow file, no package, no spec surface, no skill, no ADR. ③ Boundary flags(a) No (b) The pruned seed must still serve (c) ④ Design judgment
Against the ruling and the card. Ruling 6458731814 letter A names the Build Core, Type Check and pack-smoke seeds, which #27067 already landed; the dogfood seed surfaced only in pool reading 6647111495. The PR applies A's principle to it — "each seed-save step is wired behind Attempts to break it:
Findings:
Note, not a finding: the save step still has no Implemented-by: VERDICT: PASS |
|
domain:devx#2 · marchtian · session_017ihL2ebbZkr2QAEG1x7UC8 席位核对 — ACCEPT ·
|
Requested by 田騏 · project thread
Part of #25452 (the card stays open for its after-reading: the pool after ≥ 5 merges, read by an account that can read
actions/cache/*)Clause-②: no
Metadata-protocol: none
维护者速读
prune-turbo-cache.mjs后面。测试步骤的 turbo 命令加--summarize写运行摘要,新增的修剪步骤只留本次运行用到的任务条目,保存只在修剪答save=true时进行。不加脚本、不加门禁、不改恢复键、不抬池子上限。Linux-turbo-dogfood-1-main-*每个 sha 约 3.69 GB,两份占池子 95%。整份保存会把之前每份种子都累积进来,把其他种子挤出 10 GB 池。裁决 6458731814 选 A「修剪,不删」。.github/workflows/ci.yml的 dogfood 作业。dogfood 只在 main 的 push/定时运行上跑,PR 上看不到效果。最坏情况是某次 push 不保存种子(没有运行摘要就是save=false),下次继续恢复上一份种子。回滚 = revert 本 PR。.github/workflows/ci.ymldogfood 作业新增的「Prune the Turbo cache to this run's tasks (main only)」步骤,以及其后保存步骤的if:;同一作业测试步骤 turbo 命令上的--summarize。node scripts/ci/prune-turbo-cache.mjs --self-test,预期最后一行prune-turbo-cache self-test: OK (25 case(s) across 5 batteries).What changes
Ruling 6458731814 (#26957, letter A): "prune, do not drop", each seed-save step "wired behind
scripts/ci/prune-turbo-cache.mjs --site …exactly as Test Core … [is] today, so a seed holds only the task entries its own run used". "⛔ The pool limit is not raised here" stands.One file,
.github/workflows/ci.yml, jobdogfood(Dogfood Regression Gate (N/3)), +23 / -2:pnpm turbo run test --filter=@objectstack/dogfood --log-order=streamgains--summarize, so the run writes.turbo/runs/(one JSON file per run). The job had no run summary before: neither of its two turbo invocations (the closure build and the test run) passed--summarize, and the job sets noTURBO_RUN_SUMMARY.id: prune-turbo-cache,if: always() && github.event_name == 'push':node scripts/ci/prune-turbo-cache.mjs --site 'Dogfood Regression Gate (N/3)' --cache .turbo/cache --run-summaries .turbo/runs. These are Test Core's step and condition, without--extra-summaries(next bullet).if:ANDssteps.prune-turbo-cache.outputs.save == 'true'. The path and key are unchanged.upgrade-harness-optin.yml(its restore at L96), the job'sif:, its timeout, and the two attestation steps, which stay last and unconditioned.Why one summary is enough, and why there is no
--extra-summaries: the test run plans the whole^buildclosure and replays it, so its summary names every closurebuildtask. Measured at basef2a9007e93withturbo run build --filter=@objectstack/dogfood --dry=jsonagainstOS_TEST_SHARD=1/3 turbo run test --filter=@objectstack/dogfood --dry=json: 71 shared tasks, 71 hash-identical, 0 differing. The only closure node missing from the test plan is@objectstack/dogfood#build, which has no command and no cache entry. Test Core needs--extra-summariesonly because its closure summary is moved out of.turbo/runs/by the remote-cache assertion, and the dogfood job runs no such assertion.No
--siteregistry, self-test or gate lists sites.git grep prune-turbo-cacheoutside the workflows finds only the script itself and twoui/scripts/__tests__files about the ui jobs. So, as with #27067, nothing else needs to change.Tests
Local seed-size estimate (closure only), at 4f58391 = base + this diff
The setup stands in for CI. A private
--cache-dirwas seeded with a copy of this container's shared.turbo/cache(260 MB, 1059 files) as the "restored seed". Then the job's closure command ran (--concurrency=2):Tasks: 69 successful, 69 total,Cached: 2 cached,6m39s, on a shared 4-vCPU box. Then came the test step exactly as this PR spells it, but with an out-of-rangeOS_TEST_SHARD=9/3, so vitest refused at once. That run reported69 cached, 70 total,Failed: @objectstack/dogfood#test, and wrote its summary anyway, so a red test step still leaves a summary to prune by. Then the prune ran exactly as the new step does:dogfood#testentry (outputs: [], a log archive) adds a small amount: NOT MEASURED, because a full dogfood shard was not run here.--summarizemoves no hash: the summary's 69 build hashes equal the dry-run plan taken without it (69 identical, 0 differing).upgrade-harness-optin.ymlrestoresturbo-dogfood-1-main-*and builds--filter=@objectstack/dogfood): every one of the 69 closure tasks that has a command still has its.tar.zstin the pruned seed (69 of 69, 0 missing).node scripts/ci/prune-turbo-cache.mjs --self-test(script unchanged):prune-turbo-cache self-test: OK (25 case(s) across 5 batteries)., exit 0.dogfoodjob's steps, parsed with theyamlpackage: ... 8Upload stall diagnostic reports, 9Prune the Turbo cache to this run's tasks (main only)(id=prune-turbo-cache,if="always() && github.event_name == 'push'"), 10Save Turbo cache (main only)(if="always() && github.event_name == 'push' && steps.prune-turbo-cache.outputs.save == 'true'"), 11Attest this shard ran and passed, 12Publish this shard's attestation.Acceptance notes
474c030b13):Linux-turbo-dogfood-1-main-*seeds are 3,685,866,556 B and 3,687,370,253 B, together 7.37 GB, or 95.2 % of the second read.*-turbo-*keys overall are 99.8 %.upgrade-harness-optin.yml:96as the source, but that step isactions/cache/restoreand never saves. The writer is this job's save step:github.jobisdogfood, so its key isLinux-turbo-dogfood-SHARD-main-SHA. The harness only restores shard 1's seed.main. It is a maintainer pool reading (actions/cache/usageplus the fullactions/cacheslisting, in the shares of 6647111495), because the seat token gets 403 onactions/cache/*. Until then, the first push run's threeprune-turbo-cache: Dogfood Regression Gate (N/3): …lines are the measured post-prune seed size.dogfoodjob does not run onpull_requestormerge_group(THE TWO SPEEDS), so this PR's own CI cannot exercise the new step. The first run is the first push run onmainafter merge.Verification
Derived gates (
node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack), run on2a737765df(this branch after mergingorigin/mainff12aed37; the derived list was byte-identical before the merge). Each exit code was captured before any pipe.53 derived commands: 53 exit 0, plus the --ran line
nodegates (closing-keyword parity, comment-mask corpus, self-test-wired, self-test-workflow-commands, step-collectors, whole-set-label-write, assert-turbo-remote-cache / plan-test-core-shards / prune-turbo-cache / scheduled-full-run self-tests, drift-comment, ci-failure self-test, dts-emitted self-test): exit 0.pnpm check:*, includingcheck:ci-filter-parity,check:required-contexts,check:shard-attestation,check:stall-guard-budget,check:stall-guard-headroom,check:workflow-status-functions,check:workflow-step-name-quoting,check:agent-test-spelling,check:aggregator-roster,check:nul-bytes,check:pm-dispatch-gates(fast tier, 340 s) andcheck:type-check-debt(under the verify lock, re-measure 366 s, 0 raw tsc errors): exit 0.check:dual-build-cjs-loadsfirst exited 3 (PREREQUISITE NOT MET: 7 packages outside the dogfood closure had nodist/). It was re-run aftercheck:type-check-debtbuilt every package and then exited 0: 116 require entry points across 71 packages load.actionlintis not derived and not installed, so it was not run. CI's Lint & Repo Gates runs the workflow lints.Session: https://claude.ai/code/session_017ihL2ebbZkr2QAEG1x7UC8
🤖 Generated with Claude Code