[objectstack-fleet]

Fixes #30289

Clause-②: no
Metadata-protocol: none

维护者速读

• 改了什么: 新增私有包 @objectstack/scenarios(packages/qa/scenarios,不发布、无构建步骤、不调用任何模型),内含场景卡的 zod schema 与加载器;在 docs/qa/scenarios/ 落下开放场景库的头 9 张卡(5 张来自 packages/cli/src/lint/corpus.ts 的 golden case,4 张通用构建场景)。每张卡六层:提示词、结构期望 expect、能力标签与分类、问数(种子数据 + 声明式真值 + 角色)、写操作与五项护栏、一次迭代请求。问题的答案由代码从种子数据算出,不手填。
• 为什么改: cloud#3600 D1 裁决(「同意」):开放场景库放本仓 docs/qa/scenarios/,schema 与加载器放私有 packages/qa/* 包,不进 packages/spec。这是给评分器(姊妹卡,expect 形状共用)和后续本地 runner 打地基。
• 风险与回滚: 不触 packages/spec、不改任何公开面、包为 private: true。仓级改动只有三处登记:scripts/cross-package-test-inputs.mjs 与 turbo.json 声明本包测试读 docs/qa/scenarios/*.json 和清单 area 文件(否则改卡不会触发测试、缓存会回放旧绿),以及 scripts/check-ci-filter-parity.mjs 自测里按名记下这条新 glob(计数 20 → 21)。回滚 = revert 本 PR。
• 先看: ① packages/qa/scenarios/src/loader.ts 的 checkCard(交叉引用与真值计算的拒收点);② packages/qa/scenarios/src/truth.ts 的 evaluateTruth 与 topN(平局截断、空集平均的拒收);③ docs/qa/scenarios/invoice_with_line_items.json(一张卡的完整样子)。
• 两分钟验证: pnpm --filter @objectstack/scenarios test,预期 Test Files 3 passed、Tests 42 passed;把任一张卡的 category 改成 retail 再跑,library 用例转红并点名该文件与 category。
• 需要你做的: 无,席位落地。

What changes

New private package @objectstack/scenarios (packages/qa/scenarios), named after its siblings (@objectstack/dogfood, @objectstack/http-conformance, @objectstack/vitest-filter-preflight: private: true, exports at src, no build step). Its only runtime dependency is zod.

• src/schema.ts: the card shape. Every object is strict, so an unknown key is refused rather than dropped. The six layers:
  1. prompt: { short, detailed }, each { zh?, en? } with at least one language.
  2. expect: field-for-field the shape the scorer card fixes (objects, minFieldsPerObject?, selectFields?, lookups?, dashboards?, appNavTargets?). Nothing renamed, no required key added, and no optional key added.
  3. capabilities (AREA.NAME) and category. The category is a closed enum: app, crm, sales, marketing, finance, hr, operations, project, productivity, data, integrations, devtools, utility, other.
  4. run.seed, run.questions (at least 3), run.personas (at least 2). A truth spec is count | sum | avg | min | max | topN with filter (eq ne gt gte lt lte in), groupBy and order.
  5. run.operations (create, update or delete; at least one create or update) and run.guardrails: all five checks, permission_parity, write_validation, confirm_irreversible, audit_actor and injection_as_data. Each check names the persona, operation or seed cell it is about.
  6. iterate: one change sentence plus an additive expectDelta.
• src/truth.ts: evaluates a truth spec over the seed rows. It refuses any truth the seed does not determine: an avg, min or max over no values, a numeric aggregate over text, a mixed number/string comparison, a topN whose cut-off falls inside a tie, or a topN asking for more entries than exist.
• src/loader.ts: loadScenarios(dir, { areas }) returns typed cards with their computed truths, or a list of { file, path, message } refusals. A card with any refusal is not returned at all. The second pass holds the card together: id equals the file's basename; lookups, selects and nav targets resolve inside expect; the capability area is in the closed set; every expected object is seeded, and only those; each seeded object carries minFieldsPerObject fields; lookup values name seeded rows; question, persona and operation references resolve against the seed; guardrail references resolve; the iterate delta is additive and consistent. readCapabilityAreas(dir) reads the area closed set as the basenames of docs/qa/platform-checklist/areas/*.json.

Nine seed cards in docs/qa/scenarios/, plus a README.md covering the format, the six layers, how truth is computed, and the rule that this set holds generic scenarios only.

• The five corpus cards derive expect mechanically from their fixtures. Every object is listed. minFieldsPerObject is the smallest object's field count, because the scorer applies it to every object. Each select field becomes a selectFields row whose minOptions is its option count, and each lookup or master_detail field becomes a lookups row. The derivation ran over a copy of corpus.ts, and its output is identical to what the cards carry.
• The four generic cards carry the prompts and expect values exactly as the card states them.

Repo wiring that an existing gate requires (outside the claimed file surface; see Acceptance notes):

• scripts/cross-package-test-inputs.mjs: an @objectstack/scenarios entry declaring the two globs this suite reads, with a heldBy witness for the area directory, which is listed with readdirSync.
• turbo.json: an @objectstack/scenarios#test task whose inputs carry both globs, so the cache hash moves when a card changes.
• scripts/check-ci-filter-parity.mjs: its self-test pins how many declared globs a pre-ci.yml's core filter misses ten more declared cross-package test inputs — content/ (7), docs/, skills/, .github/workflows/ #10015 crosspkg rollback leaves uncovered. docs/qa/scenarios/*.json is a new member of that set, so the pin moves from 20 to 21 and names the glob. That pin's own comment prescribes exactly this edit.

Tests

• test/library.test.ts loads every card under docs/qa/scenarios/ and asserts zero refusals and exactly the nine ids. It then asserts every question's computed truth (44 questions) against values I worked out by hand from the seed rows, not against a snapshot of the evaluator's output.
• test/refusals.test.ts holds 21 refusal cases plus one control. Each case starts from a real library card, changes exactly one thing, and asserts the refusal's file and exact field path. The control shows the unmutated card loads clean.
  • The three classes the card names: unknown capability area (capabilities[0]), a category outside the enum (category), and a truth kind outside the enum (run.questions[0].truth.kind).
  • Other cases: unknown key, id/basename mismatch, invalid JSON, a prompt with no language, too few questions, a dangling lookup target, a dangling nav target, a seed thinner than minFieldsPerObject, a field typo in a question, a tied topN cut-off, an avg over zero rows, a sum over text, a broken lookup value in the seed, a persona that hides nothing, an operation on a missing row, a missing guardrail check, an injection cell that is not text, and a delta that re-adds an existing object.
• test/truth.test.ts: the evaluator's semantics (null skipping, an empty sum, string and date comparison, in and eq null, group ordering with a null key and numeric keys, order without groupBy refused, ranking, oversized topN).

Acceptance notes

• File surface extended, as recorded above. scripts/cross-package-test-inputs.mjs and turbo.json are what pnpm check:cross-package-test-inputs requires of any package whose tests read outside it. Without them, a card-only edit would not schedule this suite in CI (Layer A) and turbo would replay a cached green (Layer B). Adding that declaration in turn moves the check:ci-filter-parity self-test count. No workflow edit was needed, because docs/** is already in ci.yml's crosspkg filter.
• Premises measured: (1) The checklist area basenames equal each file's own area field for all 16 files. No other closed area source exists, so the PM's suggested source is used. (2) Cards as JSON beside the docs/qa/platform-checklist/ JSON ledgers fit the repo's precedent. (3) No existing schema to extend. The only scenario schema in the tree is packages/spec/src/qa TestScenario (the os test suite format), which is a different concept, and packages/cli MetadataEvalCase is a prompt plus a fixture with no expect or Run half.
• Not wired to CLI or scorer. That is sibling work, as the card scopes it. The corpus expect is derived once and is not held equal to corpus.ts by a test. A drift pin would make a private qa package import CLI internals (corpus.ts is not exported), so that direction is left to the CLI-wiring card.
• minFieldsPerObject for invoice_with_line_items is 1, because the fixture's account object has one field. The derivation is mechanical, and the scorer applies the bound to every object.
• No customer-derived content anywhere. Nothing calls a model or the network.

Verification

Package suite and typecheck (final head 038f4be)

pnpm --filter @objectstack/scenarios test       Test Files  3 passed (3) · Tests  42 passed (42)
pnpm --filter @objectstack/scenarios typecheck  exit 0
os-verify-lock: VERDICT command-exit 0

Ablation (one, on a tested behaviour; restored byte-identical)

truth.ts avg divided by rows.length instead of the non-null value count, applied through scripts/ablation-replace.mjs (anchor 1 → 0, blob 0afa46f9 → a6a44d3e). test/truth.test.ts turned red on "skips null values in a numeric aggregate". After the restore, git hash-object equals the HEAD blob 0afa46f9 and git diff HEAD is empty. The library cards hold no null in an averaged field, so the library suite stayed green under this mutation, as expected. The null-skipping semantics are pinned by truth.test.ts.

Derived gates (dispatch-gates --commands, reconciled with --ran)

Tree: objectstack-ai/objectstack at 038f4be (merge base 4be9734). Reconciliation: dispatch-gates --ran → 92 derived famil(ies) accounted for — 84 run, 8 NOT-MEASURED (6 DERIVED from a recorded exit 3, 2 claimed), exit 0.

• 84 run, every one exit 0, among them check:cross-package-test-inputs, check:ci-filter-parity (red first on the rollback pin, 1 of 139 assertions; after the pin edit 140 assertions, exit 0), check:pm-dispatch-gates, check:type-check-coverage, check:test-source-alias, check:console-intercept-disarm, check:nul-bytes, check:doc-authoring, check:issue-citations, check:changeset-fixed, check-changeset-no-major, check-empty-changeset, check:turbo-task-graph, check:undeclared-dep-imports, check:manifest-repository-directory, check:pnpm-filter-targets.
• NOT MEASURED, exit 3 PREREQUISITE NOT MET (they read built dist/ this worktree never built; this diff builds no package): check:dts-closure, check:dual-build-cjs-loads, check:lean-entry-closure, check:sourcemap-no-sources-content, @objectstack/lint check:doc-formula-expressions; check-plugin-teardown-shape --self-test needs an unshallowed clone (it exits 3 on the shared checkout's own tree too).
• NOT MEASURED, timed out under the local 300 s wrapper (repo-wide scans CI owns): check:query-options-erasure, check:type-check-debt.

---

Generated by Claude Code

[github-actions]

📓 Docs Drift Check

This PR changes 1 package(s): @objectstack/scenarios, touching 75 documentable anchor(s). ⚠️ 5 changed file(s) yielded no anchor (packages/qa/scenarios/README.md, packages/qa/scenarios/package.json, packages/qa/scenarios/src/index.ts, …), so the pages documenting them are NOT COVERED by this run — this is not a clean bill of health for those files.

19 hand-written doc(s) name something this change touched — list omitted above 15 rows. Re-derive on the tree named below: node scripts/docs-audit/affected-docs.mjs --json bfca0efb79ac5df2049f392cdecd3bbeeea9ec61.

⛔ 5 release-owned page(s) also affected — read-only, see AGENTS.md Documentation Guardrails.

What this run could not see

• 5 changed file(s) yielded no anchor (packages/qa/scenarios/README.md, packages/qa/scenarios/package.json, packages/qa/scenarios/src/index.ts, …) — pages documenting those are invisible to this run
• 41 name(s) were too generic to anchor anything (single lowercase words)
• the SDK route bridge reached 61 of 219 client-bound route-ledger rows — the other 158 have no registrar path: tail to select them, so pages documenting THEIR client methods cannot appear above, on this or any run. Of those 158: 0 are remediable by widening that discovery convention (an in-repo file declares the path; the convention did not scan it); 55 are structural — on a ledger where NOT ONE row is declared in-repo, so no discovery change reaches them at any price; 103 are undecided (no in-repo declaration, on a ledger that has other in-repo registrars — absence and an unreadable spelling are not distinguishable here). The rows themselves: node scripts/docs-audit/affected-docs.mjs --bridge-coverage
• a page that states a rule by its inputs shares no identifier with the emitter that implements the rule, so an emitter-only diff cannot list it — not on this run and not on any run. Measured on fix(driver-sql): emit varchar(maxLength) for a text field a declared index keys on #11430: content/docs/protocol/objectql/types.mdx documents the text-family column mapping by the ObjectQL type names it maps FROM (text / textarea / html) while the diff changed createColumn; it went unlisted, and it was the page that diff falsified, in four places. No shared token exists to detect this on, so a rule your change carries has to be re-read by hand in the pages that restate it.
• a key NAME is not a key, so the hand re-read the line above prescribes can land on the wrong schema. The same spelling is authorable on one governed type and a [REMOVED] tombstone on another for each of active, aria, joins, objects, template, tools and version (censused on [finding] tools is a key on BOTH AgentSchema (tombstoned, dead) and SkillSchema (live, cloud-attested), so a name-based search attributes skill examples to the agent key — it produced a false stop-the-line alarm on PR #19059 #19093 over the liveness ledger's governed types, top-level keys); nothing in a search result distinguishes the two, so a grep hit on a LIVE example reads as evidence about the DEAD key. Measured on fix(spec): the agent.tools liveness row says dead — it claimed live on a key the schema tombstoned #19059: content/docs/ai/agents.mdx was reported as contradicting the agent.tools tombstone over its tools: example at :161, which is inside the defineSkill({ block opened at :155 — the page was already correct. Settle ownership by PARSING the value against both schemas, never by the name: that literal PASSES SkillSchema, and as an AgentSchema it FAILS at tools with the tombstone prescription. ⛔ These names are not the whole class — a key retired through a .strict() guidance map leaves no tombstone in the walked shape and none of them here (tool.category, live as AIToolDefinition.category).

Coarse fallback — 0 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json bfca0efb79ac5df2049f392cdecd3bbeeea9ec61 → packageMentionDocs.

Which tree this was computed on

This run read content/docs from 387c9368badb5155825e5d368973f3c78ed7a434 — the merge of head 038f4be0d81bfaff184a620879a88c96421b46ae into base bfca0efb79ac5df2049f392cdecd3bbeeea9ec61, which is what actions/checkout gives a pull_request run. Not the PR head.

A worktree cut from an older main holds a different content/docs, so re-deriving there can legitimately return a different list — that is a different tree, not a wrong row. To answer on the same tree:

# while this PR is open — GitHub drops the merge commit once it closes
git fetch origin 387c9368badb5155825e5d368973f3c78ed7a434 && git checkout 387c9368badb5155825e5d368973f3c78ed7a434
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin bfca0efb79ac5df2049f392cdecd3bbeeea9ec61 038f4be0d81bfaff184a620879a88c96421b46ae && git checkout -B drift-repro bfca0efb79ac5df2049f392cdecd3bbeeea9ec61 && git merge --no-ff 038f4be0d81bfaff184a620879a88c96421b46ae

node scripts/docs-audit/affected-docs.mjs --json bfca0efb79ac5df2049f392cdecd3bbeeea9ec61

⚠️ That checkout carried uncommitted changes, so the commit above does not fully identify what was read.

Advisory only, and a precision-first one (#9192): a page is listed because it names a
symbol, wire route or SDK method this diff touched — not because it mentions a changed
package. Each row says which anchor put it there, so a wrong row is reportable rather than
merely annoying. To re-verify, run the docs-accuracy-audit workflow scoped to these files:
node scripts/docs-audit/affected-docs.mjs bfca0efb79ac5df2049f392cdecd3bbeeea9ec61 → pass the list as
args.docs, on the commit named under Which tree this was computed on.

[objectstack-fleet]

epic:#3600#1 · hotlong · session_01Q4EjjPXwX1YfHyKJUh4agu

席位核对

Verdict: ACCEPT at head 038f4be0d81bfaff184a620879a88c96421b46ae, landing HELD.

• Waits-on: objectstack-ai/cloud#3600 decision comment 6651166095, Q1 (the maintainer confirms the 9 seed scenarios before they land; maintainer verbatim: 「这个场景的清单是需要我人工审核和确认的」).
• Even after Q1 is ruled, landing also waits for every check-run on the then-current head to be green.

1. Shape.
   • Draft PR → main. Line 1 is Fixes #30289. Clause-②: no and Metadata-protocol: none sit at line start in the body and the changeset.
   • 26 files, +2835/−3, under the 3000-line human-merge threshold.
2. Scope (read via the PR files endpoint).
   • The new private package @objectstack/scenarios (private: true, exports at src, only zod as a dependency) and 9 cards plus a README under docs/qa/scenarios/.
   • The changeset is right for a private package: .changeset/config.json has privatePackages: { version: true, tag: false }.
   • The three extra files (scripts/cross-package-test-inputs.mjs, turbo.json, scripts/check-ci-filter-parity.mjs) are an accepted deviation: an existing gate forced them, the dev reported them, and the claim surface is amended on the card. None is governed.
   • No packages/spec change; no public surface.
3. Ruling compliance.
   • The expect shape matches the card field for field: nothing renamed, no optional key added.
   • The 9 ids are exactly the card's list (5 corpus, 4 generic).
   • Spot-checked book_library.json. It has all six layers. Capabilities are drawn from checklist areas, and the category is from the closed enum. There are 5 questions and 2 bilingual personas.
   • No customer-derived content; no model call.
4. Tests and gates.
   • Dev side, with commands and output: 3 files / 42 tests, all 44 computed truths equal hand-worked values; 21 refusal cases each assert file + exact path; one ablation turns the truth suite red and restoring returns it to green.
   • dispatch-gates: 92 derived, 84 run with exit 0, 8 NOT MEASURED (6 for missing prerequisites, 2 timed out locally; all go to CI).
   • Check-runs on 038f4be0 at review time: 11 success, 7 skipped, 16 in_progress, 0 failure.
5. Out-of-scope findings. Corpus ↔ card drift has no pin yet. Carrier: objectstack#30291, whose CLI wiring is the natural owner of a drift check. The shallow-clone teardown self-test is environment-only, with no carrier.
6. Landing. After Q1 is ruled A (or rework lands per Q1 = B) and every check is green, the seat flips this PR to ready and enables auto-merge through the relay.