Repository navigation
Commit 5065db8
ci: retire Spec Coverage and Engine Split Metric; run the showcase smoke in the Nightly Tiers live tier (#26529)
<!-- claude-projects-attribution: {"github_login":"os-steve"} -->
_Requested by **Steve** · [project
thread](https://claude.ai/code/project/chan_01FdL1XeUyvVNxLreh3F4Qi9?thread=cmsg_01FdL1XeUyvVNxLreh3F4Qi988VUoRMhEaekZbB638HuLE)_
Fixes #26445
Clause-②: no
Metadata-protocol: none
Ruling-ref: 6433804956
## 维护者速读
- 改了什么:删掉两条没人读的定时工作流(Spec Coverage 每天、Engine Split Metric 每周);Showcase
Smoke 不再是独立工作流,它的整套 Playwright 冒烟改由夜跑 `Nightly Tiers` 里的一个
`*.live.test.ts` 运行。
- 为什么改:裁决 B(6433804956)。三条工作流红了都不开卡、没人看;并进夜跑后,showcase 冒烟红了会由夜跑的 report
步骤开卡。
- 风险与回滚:夜跑分片 1 多约 8 分钟(构建 Console + 装 Chromium + 冒烟本身),仍在 45 分钟上限内;回滚即
revert 本 PR,三条工作流原样回来。
-
先看:`.github/workflows/test-nightly-tiers.yml`(新增准备步骤、诊断上传、分片腿顺序)、`examples/app-showcase/test/showcase-smoke.live.test.ts`、`examples/app-showcase/vitest.config.ts`。
- 两分钟验证:本 PR 改了 `test-nightly-tiers.yml`,所以它自己会在 PR 上跑一次 Nightly
Tiers(分支上不写看板);看 `Nightly tiers (1/2)` 里 showcase 那一腿的 `53 passed`。
- 需要你做的:这是 Tier H(`.github/workflows/**`、`scripts/pm/**`),请批准后由席位落地。
## What changes
**Retired (deleted):**
- `.github/workflows/coverage-nightly.yml` (Spec Coverage, nightly). Its
only product was a `coverage-report` artifact; nothing reads it.
`packages/spec`'s `test:coverage` script stays as an on-demand developer
command (it is not a `check:*` gate and no workflow calls it after this
PR).
- `.github/workflows/engine-split-metric.yml` (Engine Split Metric,
weekly, report-only). **The ADR-0076 D7 metric script stays in
`scripts/`, unwired from any schedule**:
`scripts/check-engine-split-ratio.mjs` runs on demand (its header now
says so), and its `--self-test` stays wired in `lint.yml`, so the
instrument stays correct while unrun. Keeping it means ADR-0076 still
names a script that exists; its sentence naming the deleted workflow
needs a one-line note, which is a separate governed PR (this one does
not touch `docs/adr/**`).
- `.github/workflows/showcase-smoke.yml` (Showcase Smoke, nightly).
**Showcase smoke moved under Nightly Tiers:**
- New `examples/app-showcase/test/showcase-smoke.live.test.ts`. The
`*.live.test.*` name puts it in the nightly `live` tier
(`scripts/nightly-tiers.mjs`), so no pull request or merge-queue run
collects it. It does not restate any assertion: it runs `playwright
test` with the package's own `playwright.config.ts` (all three specs
under `e2e/`: the 49 nav surfaces, the ADR-0085 detail shapes, the bulk
capability gate), streams the `list` reporter to stdout for the stall
guard, and passes only when Playwright's JSON report has no failing
test, no run-level error, and at least one test ran. A red names each
failing spec (`file › title`) and, for a whole-run red, says to read it
as environment first.
- `examples/app-showcase/vitest.config.ts` reads `OS_TEST_TIERS` through
`scripts/nightly-tiers.mjs` (`readTierMode`, `isNightlyTierFile`,
imported, not re-spelled): under `queue` the tier files are excluded by
exact path and the default include is unchanged; under `nightly` they
are the include. `turbo.json`'s `@objectstack/example-showcase#test`
already hashed `OS_TEST_TIERS`; it now also lists
`scripts/nightly-tiers.mjs` (+ `.d.mts`) as inputs, as
`@objectstack/cli#test` does.
- `.github/workflows/test-nightly-tiers.yml`:
- new step "Prepare the showcase browser smoke": on the shard carrying
`@objectstack/example-showcase` only, `pnpm console:build` and
`playwright install --with-deps chromium` (the two prerequisites the
retired workflow had). `continue-on-error`, so a failed Console build
cannot take the shard's CLI slice down with it; the smoke then fails as
a whole-run red and the night still files a card.
- new step "Publish the showcase smoke's Playwright diagnostics"
(`examples/app-showcase/test-results/`, `always()`, `if-no-files-found:
ignore`), replacing the retired workflow's two diagnostics uploads.
- the leg loop runs the file-level slices first and the whole-package
leg last (`for LEG in $SLICES __whole__`), with a comment on why it
differs from ci.yml here: the loop stops at the first red leg, and the
cheap whole package (the smoke) must not stop the shard's slice of the
CLI e2e tier.
- header comments that named the two retired workflows are reworded.
**Pins and readers updated:**
- `scripts/check-stall-guard-budget.mjs`: self-test battery 10 registers
one case per real (workflow, job) with a guarded step;
`coverage-nightly`'s job was one, so its floor goes 13 → 12 with a
comment saying why.
- `scripts/pm/dispatch-gates.self-test.mjs`: the live-tree case that
required `scripts/check-engine-split-ratio.mjs --days 90` in workflow
text drops that key (same treatment as `check-prerelease-pin-watch.mjs
--verbose` when its workflow left); the fixture row quoting it stays.
- `docs/qa/platform-checklist/areas/platform-core.json`: the showcase
item's `source` names the live test and `test-nightly-tiers.yml` instead
of the deleted workflow.
- Comment pointers in `half-state-patrol.yml`,
`shard-timings-refresh.yml` and
`examples/app-showcase/playwright.config.ts`.
Readers grepped (file names, workflow names, job names, artifact names,
`test:smoke`, `test:coverage`): `dependabot.yml`, the required-context
set, the aggregator roster, `check:self-test-wired` and
`scripts/ci/scheduled-full-run.mjs` hold none of the three.
`scheduled-full-run.mjs` uses `'Spec Coverage'` only as an example of a
name outside its map, which stays true.
## Tests
Local, on this branch (worktree `objectstack-issue-26445`), closure
built with `pnpm turbo run build
--filter=@objectstack/example-showcase^...` and `pnpm console:build`:
- Live tier, real stack (`OS_TEST_TIERS=nightly vitest run` in
`examples/app-showcase`, Chromium from `/opt/pw-browsers`): `53 passed
(2.3m)` from Playwright, `Test Files 1 passed (1)`, `Tests 1 passed
(1)`, at `0d08883a`.
- Negative control: `scripts/ablation-replace.mjs` injected a failure
into the Task Board surface only (anchor 1 → 0, blob `cf29f07379cd` →
`26b53aa6a54e`). Result: Playwright `52 passed`, Task Board failed twice
(with its retry), and the vitest test failed with `1 of 53 showcase
smoke test(s) failed, 0 run-level error(s)` naming
`showcase-smoke.spec.ts › surface renders cleanly: Task Board`.
Restored: blob equals HEAD, `git diff HEAD` empty.
- Two reds on the way, both fixed in this PR and kept in the test's
comments: vitest's worker sets `NODE_ENV=test`, which disarms the
dev-admin seed, so global setup's sign-in got 401 (the wrapper now drops
vitest's worker env); a randomly picked port that was busy made `serve
--dev` shift one port up while Playwright waited on the original until
its 180 s timeout (the wrapper now takes a port from the OS).
- Queue tier: `pnpm --filter @objectstack/example-showcase test` → `Test
Files 39 passed (39)`, `Tests 484 passed (484)` (the live file is not
collected). `pnpm --filter @objectstack/example-showcase typecheck` exit
0.
- `node scripts/nightly-tiers.mjs --check`:
`@objectstack/example-showcase owns 1 nightly-tier file(s); vitest
collects 39 under queue, 1 under nightly`; CLI unchanged (512 / 85).
`OS_TEST_TIERS=bogus` is refused by the config.
- Partition of the nightly document (`--packages` →
`partition-test-shards.mjs --shard k/2`): shard 1 = `@objectstack/cli
1/2` + `@objectstack/example-showcase` (kept whole, "not the run's
floor"), shard 2 = `@objectstack/cli 2/2`. `nightly-tiers.mjs
--self-test` and `partition-test-shards.mjs --self-test` exit 0.
## Acceptance notes
- **Measured on a runner by this PR itself:** `test-nightly-tiers.yml`
is in its own `pull_request` paths filter, so this PR runs Nightly Tiers
on a hosted runner (branch runs never write to the board). That run is
the first reading of the prepare step and the smoke under turbo's strict
env.
- Cost: shard 1 took ~24.6 min on the last scheduled night (run
40275583120). The retired workflow spent ~4.6 min on the Console build
and ~1.4 min on the smoke (run 40286891669), so shard 1 should land near
33 min against its 45-minute limit. The partitioner weighs the showcase
by its queue-tier timing (48 s) and does not know the live run's cost;
if shard 1 gets tight, the lever is the matrix, not this file.
- Not changed, read-only for this card: `.github/workflows/ci.yml` L810
still says "coverage-nightly workflow" in a comment. Left as dated
readings: `docs-deploy.yml` (cron-hour survey "on the day this landed"),
`lint.yml` L1495 (a 2026-08-21 measurement),
`scripts/docs-audit/README.md`, `check-pnpm-acquisition.mjs`,
`check-node-version.mjs`, `check-stall-guard-budget.mjs`'s "measured
when this landed" line, and `dispatch-gates.mjs` comments that cite the
old workflows as history.
- Follow-up owed (separate governed PR): ADR-0076's D7 line names
`.github/workflows/engine-split-metric.yml`; a one-line note that the
script now runs on demand.
- Root `test:smoke` (turbo) and the package's `test:smoke` stay: `pnpm
test:smoke` is still how a person runs the suite by hand, and the QA
checklist items cite it.
## Verification
Gate list derived with `node scripts/pm/dispatch-gates.mjs --commands
--repo objectstack-ai/objectstack` at the final head: 86 commands at
`57f8caa3`, each run with its exit code captured before any pipe: **86
exit 0**. `dispatch-gates --ran`: "86 derived famil(ies) accounted for —
86 run, 0 NOT-MEASURED (a DERIVED zero)". Two were red on the way and
are fixed in this PR: `check:stall-guard-budget` (battery 10 floor, see
above) and `check:pm-dispatch-gates` (1 of 1935 fast-tier cases, the
engine split live key). The dispatch-gates slow tier (`node
scripts/pm/check-dispatch-gates.mjs --slow`, push/schedule only in CI)
also ran: `✓ dispatch-gates self-test: 2200 cases pass`, exit 0. After
merging `origin/main` (`2094edeb`, which brought the shard-matrix reader
into `partition-test-shards.mjs`) the derivation is unchanged (86) and
the overlapping gates were re-run: `partition-test-shards.mjs
--self-test`, `nightly-tiers.mjs --self-test`, `nightly-tiers.mjs
--check` all exit 0, and the nightly partition is unchanged. Families CI
runs (Lint & Repo Gates, TypeScript Type Check, Test Core, Build Core,
the UI lanes, Nightly Tiers on this PR) are read from this PR's
check-runs.
Changeset: none. `skip-changeset` — workflows, root scripts,
`turbo.json` and the private `@objectstack/example-showcase` publish
nothing.
## Human Minimum
- [x] 1. Schemas start as Zod; `packages/spec` holds no runtime logic.
(not touched)
- [x] 2. Every name I added uses the casing for its kind.
- [x] 3. No formatter ran; `pnpm lint` passes (CI).
- [x] 4. One issue, one branch off an up-to-date `main`, one pull
request.
- [x] 5. Nothing publishes; `skip-changeset`.
- [x] 6. `packages/spec` not touched.
- [x] 7. No hand edits to `content/docs/references/`,
`packages/*/CHANGELOG.md` or `content/docs/releases/`.
- [x] 8. Title, body and comments are in English (the maintainer brief
is Chinese by convention).
- [x] The *Before Submitting* list in CONTRIBUTING.md is done.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
https://claude.ai/code/session_01ADHkmTHHuvZqTM7XKWfv8Q
---
_Generated by [Claude
Code](https://claude.ai/code/session_01ADHkmTHHuvZqTM7XKWfv8Q)_
---------
Co-authored-by: Claude <noreply@anthropic.com>1 parent 16ca320 commit 5065db8
15 files changed
Lines changed: 323 additions & 201 deletions
File tree
- .github/workflows
- docs/qa/platform-checklist/areas
- examples/app-showcase
- test
- types
- scripts
- pm
This file was deleted.
This file was deleted.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
178 | 178 | | |
179 | 179 | | |
180 | 180 | | |
181 | | - | |
| 181 | + | |
182 | 182 | | |
183 | 183 | | |
184 | 184 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
271 | 271 | | |
272 | 272 | | |
273 | 273 | | |
274 | | - | |
| 274 | + | |
275 | 275 | | |
276 | 276 | | |
277 | 277 | | |
| |||
This file was deleted.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
13 | 13 | | |
14 | 14 | | |
15 | 15 | | |
16 | | - | |
| 16 | + | |
17 | 17 | | |
18 | 18 | | |
19 | 19 | | |
| |||
41 | 41 | | |
42 | 42 | | |
43 | 43 | | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
44 | 52 | | |
45 | 53 | | |
46 | 54 | | |
| |||
75 | 83 | | |
76 | 84 | | |
77 | 85 | | |
78 | | - | |
79 | | - | |
80 | | - | |
| 86 | + | |
81 | 87 | | |
82 | 88 | | |
83 | 89 | | |
| |||
205 | 211 | | |
206 | 212 | | |
207 | 213 | | |
| 214 | + | |
| 215 | + | |
| 216 | + | |
| 217 | + | |
| 218 | + | |
| 219 | + | |
| 220 | + | |
| 221 | + | |
| 222 | + | |
| 223 | + | |
| 224 | + | |
| 225 | + | |
| 226 | + | |
| 227 | + | |
| 228 | + | |
| 229 | + | |
| 230 | + | |
| 231 | + | |
| 232 | + | |
| 233 | + | |
| 234 | + | |
| 235 | + | |
| 236 | + | |
| 237 | + | |
| 238 | + | |
208 | 239 | | |
209 | 240 | | |
210 | 241 | | |
| |||
220 | 251 | | |
221 | 252 | | |
222 | 253 | | |
| 254 | + | |
| 255 | + | |
| 256 | + | |
| 257 | + | |
| 258 | + | |
| 259 | + | |
| 260 | + | |
223 | 261 | | |
224 | 262 | | |
225 | 263 | | |
| |||
246 | 284 | | |
247 | 285 | | |
248 | 286 | | |
249 | | - | |
| 287 | + | |
250 | 288 | | |
251 | 289 | | |
252 | 290 | | |
| |||
299 | 337 | | |
300 | 338 | | |
301 | 339 | | |
| 340 | + | |
| 341 | + | |
| 342 | + | |
| 343 | + | |
| 344 | + | |
| 345 | + | |
| 346 | + | |
| 347 | + | |
| 348 | + | |
| 349 | + | |
| 350 | + | |
| 351 | + | |
| 352 | + | |
| 353 | + | |
| 354 | + | |
302 | 355 | | |
303 | 356 | | |
304 | 357 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
347 | 347 | | |
348 | 348 | | |
349 | 349 | | |
350 | | - | |
| 350 | + | |
| 351 | + | |
351 | 352 | | |
352 | 353 | | |
353 | 354 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
12 | 12 | | |
13 | 13 | | |
14 | 14 | | |
15 | | - | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
16 | 19 | | |
17 | 20 | | |
18 | 21 | | |
| |||
0 commit comments