Skip to content

test(qa-scenarios): add the leave_balances mid-tier scenario card - #30690

Merged
objectstack-fleet[bot] merged 2 commits into
mainfrom
claude/project-thread-oih0ft
Nov 6, 2026
Merged

objectstack-fleet[bot] merged 2 commits into
mainfrom
claude/project-thread-oih0ft

Conversation

@objectstack-fleet

Copy link
Copy Markdown
Contributor

Requested by Elon · project thread
Fixes #30479
Clause-②: no
Metadata-protocol: none

维护者速读

  • 改了什么:新增「请假与假期余额」中档场景卡:7 个对象(部门、员工、假期类型、假期余额、请假单、请假明细、审批步骤),3 天及以内只需经理审批、超过 3 天再加 HR 审批,批准后自动扣减余额;3 个角色其中 2 个按行限定;1 个请假天数报表;120 张请假单跨 7–10 月;10 道问数、6 个写操作、4 个 agent 动作。另配一份只属于这张卡的真值测试。
  • 为什么改:落实 cloud#3600 九个场景中的 HR 场景,让 AI 建应用、问数、代办写操作都有一张能用代码判分的请假与余额卡。
  • 风险与回滚:只加测试数据和测试,所在包私有、不发布,运行时零改动;回滚即删两个新文件。
  • 先看:① docs/qa/scenarios/leave_balances.json 的 run.operations(六个写操作及预期结果);② 同文件的 run.questions 与 run.personas;③ packages/qa/scenarios/test/leave-balances.test.ts 的 EXPECTED_TRUTHS。
  • 两分钟验证:pnpm --filter @objectstack/scenarios test,预期 5 个测试文件、73 个用例全绿。
  • 需要你做的:无,席位落地。

What changes

  • docs/qa/scenarios/leave_balances.json (new): the 请假与假期余额 leave-and-balances mid-tier card.
    • Seven objects: department, employee, leave_type (annual, sick, personal), leave_balance (one per employee, tracked type and leave year), leave_request, leave_day (one line per date, a full day or a half day) and approval_step.
    • The route follows the days: up to 3 days goes to the department manager only, more than 3 days to the manager and then HR (e13), and a manager's own request goes to the HR manager (e14) at step 1. On the last approval the balance's used_days (a roll-up of the approved requests) rises and remaining_days (entitlement + carry-over - used) falls.
    • Two validation rules: a request cannot be submitted for more days than its balance has left, and a sick request over 2 days needs a certificate.
    • Three personas: employee (own rows) and dept_manager (own department) carry a row scope; hr_officer sees everything except the medical note, which only the requester sees.
    • Seed: 120 requests starting July to October 2026, with 307 day lines, 141 approval steps and 28 balances. The rule that generated it is stated in the card's $comment, so every truth can be worked by hand.
    • Run half: 10 questions, 6 operations and 4 agent actions (one confirm-gated); iterate adds time off in lieu.
  • packages/qa/scenarios/test/leave-balances.test.ts (new): this card's own test file.
    • It pins all 10 computed truths and the 9 per-persona truths against hand-worked values, with the derivations beside them.
    • It asserts the mid-tier floor and checks the seed against the rules the card states: a request's days equal its weekday lines, a balance equals its approved requests and the tenure rule, each request's steps match its days and status, and both validation rules hold on every submitted request. It also checks that LR-0091 really exceeds its balance, so the refused submission is a real refusal.
    • It checks that every where in an applied post-state matches no seeded row, so it can only name the row the write creates.
    • It shows that a wrong seed cell and a wrong truth spec in a fixture copy go red against those pins.

Why

#30479 is one of the six mid-tier cards of the open scenario library. This one covers HR: leave requests whose approval route depends on their length, and balances that move when a request is approved.

Tests

pnpm --filter @objectstack/scenarios test: 5 files, 73 tests passed. pnpm --filter @objectstack/scenarios typecheck: exit 0, after building the closure with pnpm --filter '@objectstack/scenarios^...' build. eslint --no-inline-config on the new test file: exit 0.

Acceptance notes

  • File names. The card is leave_balances.json, not leave-balances.json as the issue spells it: a card's id is a snake_case machine name that must equal the file's basename. The test file follows the model's spelling, leave-balances.test.ts.
  • No shared-file edit. Since travel_expense landed, library.test.ts checks only that the three control cards are present, and the README's "Adding a card" step points at a per-card test file. So this PR touches neither, and the sibling cards no longer conflict on a shared line.
  • No schema field for some floor items. As with travel_expense, the approval flow, "declarative preferred" and a report as distinct from a dashboard have no field in the schema. The card states them in prompt.detailed (zh and en) and in capabilities, seeds approval_step as an expected object, and declares the report under expect.dashboards. Nothing in the loader grades them.
  • Capability tags. Where a tag in the closed tag table of the open scorer PR feat(scenarios): the scenario scorer moves into @objectstack/scenarios and judges capability tags, agent-action exposure, write outcomes and per-persona answers #30594 fits, the card uses that spelling (approvals.approval-flow, access-security.row-scope, dashboards.widget-measures, records-forms.lookup), so those tags are judged once the scorer lands. Tags outside that table read unscored there, not failed.
  • Seven operation kinds, six operations. file_and_submit_from_text covers two of them: entry from free text with lines, and the multi-step chain (file the request, add a full day and a morning, submit, route to the manager).
  • Routed approval step not asserted. The post-state of file_and_submit_from_text does not assert the routed step-1 row. A where cannot name the new request's id, and approver e05 · pending also matches seeded rows.
  • Dates. asOf is 2026-09-30, so October's requests are the upcoming ones and every draft or pending request is for future leave; "last month" is August. Balances belong to leave year 2026/27, which starts on 1 July, so the seed is the whole leave year so far. Days count Monday to Friday; public holidays are out of scope and the $comment says so.
  • Reference. objectstack-ai/templates is not attached to this session, so the domain is designed from the card text alone.

Verification

scenarios test + typecheck
 Test Files  5 passed (5)
      Tests  73 passed (73)
os-verify-lock: VERDICT command-exit 0
> tsc --noEmit
os-verify-lock: VERDICT command-exit 0
one-time check: a wrong cell in the real card goes red

scripts/ablation-replace.mjs set LR-0098's days from 5 to 6 in docs/qa/scenarios/leave_balances.json and ran this card's test file: 4 of 9 tests failed (the pinned truths, the lines check, the balance check, and the fixture-copy pin, which reads the real card). It then restored the file: blob equal to HEAD, git diff HEAD empty.

dispatch-gates --commands (50 commands, local, head 42b3e57)

All 50 derived commands exit 0 on head 42b3e57a (the commit before the main merge). The --ran reconciliation reads: 50 derived, 50 run, 0 NOT-MEASURED, 0 UNRUN, a derived zero because every line recorded its exit code.

Four of them first exited 3 (PREREQUISITE NOT MET). Each exited 0 after the step it names:

  • check-plugin-teardown-shape.mjs --self-test, after git fetch --unshallow origin main (the clone was shallow);
  • check:doc-formula-expressions, check:lean-entry-closure and check:dual-build-cjs-loads, after a full turbo run build (119 of 119 tasks).

The merge of main (9 commits) touches neither docs/qa/scenarios/ nor packages/qa/scenarios/, and the derivation on the merged head 9bd3d3c9 names the same 50 commands. On that head the scenarios tests (5 files, 73 tests) and typecheck pass again.

Changed lines: 1342 (+1342 / -0, 2 files), under the 3000 threshold.

Human Minimum

  • 1. Schemas start as Zod; packages/spec holds no runtime logic. (Not touched.)
  • 2. Every name I added uses the casing for its kind.
  • 3. No formatter ran; pnpm lint passes. (eslint on the new test file locally; the repo-wide run is CI's.)
  • 4. One issue, one branch off an up-to-date main, one pull request.
  • 5. A change that publishes carries a changeset, and both declaration lines are filled in here and in the changeset. (@objectstack/scenarios is private: true and publishes nothing, so no changeset, skip-changeset, as travel_expense landed.)
  • 6. packages/spec touched: its generated artifacts are regenerated and committed. (Not touched.)
  • 7. No hand edits to content/docs/references/, packages/*/CHANGELOG.md or content/docs/releases/.
  • 8. Title, body and comments are in English.
  • The Before Submitting list in CONTRIBUTING.md is done.

Generated by Claude Code

The 请假与假期余额 (leave and balances) card: seven objects (department,
employee, leave_type, leave_balance, leave_request, leave_day,
approval_step), a route set by the request's days (manager only up to
3 days, manager then HR above), the balance reduced on approval, three
personas (two row-scoped), 120 requests from July to October 2026,
10 questions, 6 operations and 4 agent actions.

Its own test file pins every computed truth, over all rows and per
persona, against hand-worked values, checks the seed against the rules
the card states, and shows a wrong seed cell and a wrong truth spec in a
fixture copy go red.

Claude-Session: https://claude.ai/code/session_016WV4cw5kGUbtDLQqKwTH7E
Co-authored-by: Claude <noreply@anthropic.com>
@objectstack-fleet objectstack-fleet Bot added the skip-changeset PR has no user-facing published change; bypasses the changeset gate label Nov 5, 2026
@github-actions

github-actions Bot commented Nov 5, 2026

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

Nothing in this diff resolved to a documentable surface (no symbol, route or SDK anchor derived from 0 changed package(s)), so this run has no opinion about the docs.

What this run could not see

Coarse fallback — 0 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json a2437b560bd8872feece6ec8b57c6fdf21cff1f4 → packageMentionDocs.

@github-actions github-actions Bot added the tests label Nov 5, 2026
@objectstack-fleet

Copy link
Copy Markdown
Contributor Author

domain:devx#3 · os-elon-musk · session_016WV4cz3RTQjLh7hZBLzgtZ

席位核对 · ACCEPT · 9bd3d3c9 · 2026-11-05T23:01Z

Head-sha: 9bd3d3c99f989426a61ee20cd1463888e6716d02

  • Shape: 2 files, +1,342/−0 (< 3,000), both on the amended claim surface (docs/qa/scenarios/leave_balances.json, packages/qa/scenarios/test/leave-balances.test.ts). Not governed. Fixes #30479, Clause-②: no, Metadata-protocol: none at line start.
  • Premise measured: schema from feat(scenarios): scenario-card schema + loader (private packages/qa package) and nine open seed cards #30340 on main; test(qa-scenarios): add the travel_expense mid-tier scenario card #30600 left library.test.ts and the README card-agnostic, so no shared line is touched.
  • Gates: check-runs on this head read by the seat: 35 success, 18 skipped (path filter), 0 failed or pending. Dev's local dispatch-gates 50/50 exit 0, --ran 0 NOT-MEASURED.
  • Spot checks (seat's own arithmetic against the pinned truths): remaining_annual_by_department Engineering (15+3−4) + (10+5−13.5) + (5−3) + (10+2−9) = 20.5 ✓, Operations 9 + 2.5 + 3.5 + 3 = 18 ✓, Sales 17 + 1 + 8 + 5.5 = 31.5 ✓, HR 12 + 20 = 32 ✓; lowest_annual_remaining e02 1, e06 1.5, e07 2 ✓. The acceptance "a deliberately wrong truth goes red" is two cases (wrong seed cell, wrong truth spec).
  • Accepted deviations: snake_case basename (id = basename, as test(qa-scenarios): add the travel_expense mid-tier scenario card #30600); the routed step-1 row of file_and_submit_from_text is not asserted in the post-state (the routing rule itself is pinned by the "routes each request by its days" case).
  • Out of scope: none. Skills-docs: n/a.
  • Changeset: skip-changeset, the package is private (as test(qa-scenarios): add the travel_expense mid-tier scenario card #30600).
  • Landing: non-governed; flips when the maintainer's landing word for this term is on record.

@github-actions

github-actions Bot commented Nov 6, 2026

Copy link
Copy Markdown
Contributor

⛔ merge queue 构建失败 — 先分诊,再决定要不要重排

队列构建 42159595597 红了。队列跑的是全量套件(PR 侧 CI 只跑 affected 子集),
所以失败的测试可能在本 PR 没碰过的包里 —— 那不是重排能修的。每次盲目重排都会让排在后面的所有 PR 重建一轮。

分类:failure —— 按下面的日志分诊。

失败的 job(日志抽取,best effort):

  • Test Core (1/8) — 失败步骤: Run this shard's tests

    @objectstack/spec:test:  FAIL   local  scripts/liveness/check-liveness.test.ts > check:liveness — evidence pointers (#5623) > is green against a verbatim copy of the shipped ledgers
      ↳ 失败原因: @objectstack/spec:test: AssertionError: Spec liveness gate (registry-rooted) — governed types: object, field, flow, action, hook, permission, position, agent, tool, skill, dataset, page, view, report,
    @objectstack/spec:test:  FAIL   local  scripts/liveness/check-liveness.test.ts > check:liveness — evidence pointers (#5623) > stays green when the missing path is attributed to ANOTHER repo
      ↳ 失败原因: @objectstack/spec:test: AssertionError: Spec liveness gate (registry-rooted) — governed types: object, field, flow, action, hook, permission, position, agent, tool, skill, dataset, page, view, report,
    @objectstack/spec:test:  FAIL   local  scripts/liveness/check-liveness.test.ts > check:liveness — evidence pointers (#5623) > never bounds a citation attributed to ANOTHER repo
      ↳ 失败原因: @objectstack/spec:test: AssertionError: Spec liveness gate (registry-rooted) — governed types: object, field, flow, action, hook, permission, position, agent, tool, skill, dataset, page, view, report,
    @objectstack/spec:test:  FAIL   local  scripts/liveness/check-liveness.test.ts > check:liveness — evidence pointers (#5623) > prints the citation count and how many are in range, in the documented two
      ↳ 失败原因: @objectstack/spec:test: AssertionError: Spec liveness gate (registry-rooted) — governed types: object, field, flow, action, hook, permission, position, agent, tool, skill, dataset, page, view, report,
    @objectstack/spec:test:  FAIL   local  scripts/liveness/check-liveness.test.ts > check:liveness — symbol anchors (#12516) > stays GREEN on the drifted BEFORE-state — the honest residual this grammar e
      ↳ 失败原因: @objectstack/spec:test: AssertionError: Spec liveness gate (registry-rooted) — governed types: object, field, flow, action, hook, permission, position, agent, tool, skill, dataset, page, view, report,
    @objectstack/spec:test:  FAIL   local  scripts/liveness/check-liveness.test.ts > check:liveness — symbol anchors (#12516) > prints the anchor count and how many resolve, equal on a green run
      ↳ 失败原因: @objectstack/spec:test: AssertionError: Spec liveness gate (registry-rooted) — governed types: object, field, flow, action, hook, permission, position, agent, tool, skill, dataset, page, view, report,
    @objectstack/spec:test:  FAIL   local  scripts/liveness/check-liveness.test.ts > check:liveness — the evidence-scan population (#13041) > stays GREEN when a `dead` entry carries the SAME rotted pointe
      ↳ 失败原因: @objectstack/spec:test: AssertionError: Spec liveness gate (registry-rooted) — governed types: object, field, flow, action, hook, permission, position, agent, tool, skill, dataset, page, view, report,
    @objectstack/spec:test:  FAIL   local  scripts/liveness/check-liveness.test.ts > check:liveness — the evidence-scan population (#13041) > declares every status either scanned or explicitly unscanned, 
      ↳ 失败原因: @objectstack/spec:test: AssertionError: Spec liveness gate (registry-rooted) — governed types: object, field, flow, action, hook, permission, position, agent, tool, skill, dataset, page, view, report,
    @objectstack/spec:test:  FAIL   local  scripts/liveness/check-liveness.test.ts > check:liveness — the live-elsewhere criteria (#13483) > is green on the shipped ledgers and publishes the population be
      ↳ 失败原因: @objectstack/spec:test: AssertionError: Spec liveness gate (registry-rooted) — governed types: object, field, flow, action, hook, permission, position, agent, tool, skill, dataset, page, view, report,
    @objectstack/spec:test:  FAIL   local  scripts/liveness/check-liveness.test.ts > check:liveness — the live-elsewhere criteria (#13483) > is green on a planted live-elsewhere row and counts it as carry
      ↳ 失败原因: @objectstack/spec:test: AssertionError: Spec liveness gate (registry-rooted) — governed types: object, field, flow, action, hook, permission, position, agent, tool, skill, dataset, page, view, report,
    @objectstack/spec:test:  FAIL   local  scripts/liveness/check-liveness.test.ts > check:liveness — the README state table (#7257) > is green against a verbatim copy, and says how many rows it checked
      ↳ 失败原因: @objectstack/spec:test: AssertionError: Spec liveness gate (registry-rooted) — governed types: object, field, flow, action, hook, permission, position, agent, tool, skill, dataset, page, view, report,
    @objectstack/spec:test:  FAIL   local  scripts/liveness/check-liveness.test.ts > check:liveness — the generated count artifact (#7377, sharded #20361) > is green against a verbatim copy, says the shar
      ↳ 失败原因: @objectstack/spec:test: AssertionError: Spec liveness gate (registry-rooted) — governed types: object, field, flow, action, hook, permission, position, agent, tool, skill, dataset, page, view, report,
    

↳ 失败原因 是判读的关键:超时(Test timed out in … / Hook timed out in …)多半是负载/时序,不是本 PR 的回归;
断言(AssertionError: …)才指向真实的行为改变。两者的 FAIL 行长得一模一样,只有这一行能区分。

⚠️ 断言这一侧有一类例外,判据是断言在测什么,不是它是不是 AssertionError。 断言的对象是产品行为(一个值、一个形状、一次拒收)⇒ 照上面读:真实的行为改变,去查,⛔ 不要重排掉;
断言的对象是这次实验自身的有效性前提(跑完的耗时、负载下的先后、任何只在时间预算内才成立的条件)⇒ 它跟超时是同一类,同样对负载敏感,重排一次是合法的判别手段。
识别是机械的:断言的消息或它比较的值本身点名了一段时长、一个时间戳、一个耗时计数。实测过的一对 —— AssertionError: SecurityPlugin.init() ran: expected false to be true 测的是产品行为(真回归);
AssertionError: this run took over a second, so second-precision stamps could have differed too: expected 1006 to be less than 1000 测的是实验前提:它守护的那条不变式当时是绿的,同一个 head 原样重排一次即成功。
穿着 AssertionError 外衣的时间测量,仍然是时间测量。(⛔ 这只改「怎么读一次红」,不改「哪些测试可以重排」——后者由别处管。)

跨 PR 相同签名(24h,按失败测试文件聚合):

  • scripts/liveness/check-liveness.test.ts — 24h 窗口内只有本 PR 撞到过,暂不汇总(再有一个不同 PR 撞到就会自动开汇总 issue)。
  • ⚠️ 24h 评论账本没读完(超过 5 页仍未读到窗口尽头),所以上面的「不同 PR 数」是下界,不是全量。

历史信号:

  • 本 PR 过去 24h 无队列失败记录(首次)。
  • 过去 24h 队列共有 2 个失败构建(不含本次)。

分诊清单:

  1. 失败测试在本 PR 改动的包里 → 真回归,修 PR。
  2. 失败测试与本 PR 无关 → 看上面的「跨 PR 相同签名」;已有汇总 issue ⇒ flaky/环境问题实锤,去那张 issue 上谈,修好前重排只会再烧一轮全队列。
  3. 两者都不是 → 可能与同组 PR 语义冲突;等前面的 PR 落地或失败出队后再重排一次即可,不要连续重排。

Generated by Claude Code · merge-queue-triage workflow (#4859)

@github-actions

github-actions Bot commented Nov 6, 2026

Copy link
Copy Markdown
Contributor

⛔ merge queue 构建失败 — 先分诊,再决定要不要重排

队列构建 42161970075 红了。队列跑的是全量套件(PR 侧 CI 只跑 affected 子集),
所以失败的测试可能在本 PR 没碰过的包里 —— 那不是重排能修的。每次盲目重排都会让排在后面的所有 PR 重建一轮。

分类:failure —— 按下面的日志分诊。

失败的 job(日志抽取,best effort):

  • Test Core (1/8) — 失败步骤: Run this shard's tests

    @objectstack/spec:test:      ✓ inverts the card's measured control: the three bogus keys now FAIL parse instead of adding zero warnings  343ms
      ↳ 失败原因: (这条 FAIL 之后 12 行内没有可识别的原因行 —— 点进 job 看)
    @objectstack/spec:test:  FAIL   local  scripts/liveness/check-liveness.test.ts > check:liveness — evidence pointers (#5623) > is green against a verbatim copy of the shipped ledgers
      ↳ 失败原因: @objectstack/spec:test: AssertionError: Spec liveness gate (registry-rooted) — governed types: object, field, flow, action, hook, permission, position, agent, tool, skill, dataset, page, view, report,
    @objectstack/spec:test:  FAIL   local  scripts/liveness/check-liveness.test.ts > check:liveness — evidence pointers (#5623) > stays green when the missing path is attributed to ANOTHER repo
      ↳ 失败原因: @objectstack/spec:test: AssertionError: Spec liveness gate (registry-rooted) — governed types: object, field, flow, action, hook, permission, position, agent, tool, skill, dataset, page, view, report,
    @objectstack/spec:test:  FAIL   local  scripts/liveness/check-liveness.test.ts > check:liveness — evidence pointers (#5623) > never bounds a citation attributed to ANOTHER repo
      ↳ 失败原因: @objectstack/spec:test: AssertionError: Spec liveness gate (registry-rooted) — governed types: object, field, flow, action, hook, permission, position, agent, tool, skill, dataset, page, view, report,
    @objectstack/spec:test:  FAIL   local  scripts/liveness/check-liveness.test.ts > check:liveness — evidence pointers (#5623) > prints the citation count and how many are in range, in the documented two
      ↳ 失败原因: @objectstack/spec:test: AssertionError: Spec liveness gate (registry-rooted) — governed types: object, field, flow, action, hook, permission, position, agent, tool, skill, dataset, page, view, report,
    @objectstack/spec:test:  FAIL   local  scripts/liveness/check-liveness.test.ts > check:liveness — symbol anchors (#12516) > stays GREEN on the drifted BEFORE-state — the honest residual this grammar e
      ↳ 失败原因: @objectstack/spec:test: AssertionError: Spec liveness gate (registry-rooted) — governed types: object, field, flow, action, hook, permission, position, agent, tool, skill, dataset, page, view, report,
    @objectstack/spec:test:  FAIL   local  scripts/liveness/check-liveness.test.ts > check:liveness — symbol anchors (#12516) > prints the anchor count and how many resolve, equal on a green run
      ↳ 失败原因: @objectstack/spec:test: AssertionError: Spec liveness gate (registry-rooted) — governed types: object, field, flow, action, hook, permission, position, agent, tool, skill, dataset, page, view, report,
    @objectstack/spec:test:  FAIL   local  scripts/liveness/check-liveness.test.ts > check:liveness — the evidence-scan population (#13041) > stays GREEN when a `dead` entry carries the SAME rotted pointe
      ↳ 失败原因: @objectstack/spec:test: AssertionError: Spec liveness gate (registry-rooted) — governed types: object, field, flow, action, hook, permission, position, agent, tool, skill, dataset, page, view, report,
    @objectstack/spec:test:  FAIL   local  scripts/liveness/check-liveness.test.ts > check:liveness — the evidence-scan population (#13041) > declares every status either scanned or explicitly unscanned, 
      ↳ 失败原因: @objectstack/spec:test: AssertionError: Spec liveness gate (registry-rooted) — governed types: object, field, flow, action, hook, permission, position, agent, tool, skill, dataset, page, view, report,
    @objectstack/spec:test:  FAIL   local  scripts/liveness/check-liveness.test.ts > check:liveness — the live-elsewhere criteria (#13483) > is green on the shipped ledgers and publishes the population be
      ↳ 失败原因: @objectstack/spec:test: AssertionError: Spec liveness gate (registry-rooted) — governed types: object, field, flow, action, hook, permission, position, agent, tool, skill, dataset, page, view, report,
    @objectstack/spec:test:  FAIL   local  scripts/liveness/check-liveness.test.ts > check:liveness — the live-elsewhere criteria (#13483) > is green on a planted live-elsewhere row and counts it as carry
      ↳ 失败原因: @objectstack/spec:test: AssertionError: Spec liveness gate (registry-rooted) — governed types: object, field, flow, action, hook, permission, position, agent, tool, skill, dataset, page, view, report,
    @objectstack/spec:test:  FAIL   local  scripts/liveness/check-liveness.test.ts > check:liveness — the README state table (#7257) > is green against a verbatim copy, and says how many rows it checked
      ↳ 失败原因: @objectstack/spec:test: AssertionError: Spec liveness gate (registry-rooted) — governed types: object, field, flow, action, hook, permission, position, agent, tool, skill, dataset, page, view, report,
    

↳ 失败原因 是判读的关键:超时(Test timed out in … / Hook timed out in …)多半是负载/时序,不是本 PR 的回归;
断言(AssertionError: …)才指向真实的行为改变。两者的 FAIL 行长得一模一样,只有这一行能区分。

⚠️ 断言这一侧有一类例外,判据是断言在测什么,不是它是不是 AssertionError。 断言的对象是产品行为(一个值、一个形状、一次拒收)⇒ 照上面读:真实的行为改变,去查,⛔ 不要重排掉;
断言的对象是这次实验自身的有效性前提(跑完的耗时、负载下的先后、任何只在时间预算内才成立的条件)⇒ 它跟超时是同一类,同样对负载敏感,重排一次是合法的判别手段。
识别是机械的:断言的消息或它比较的值本身点名了一段时长、一个时间戳、一个耗时计数。实测过的一对 —— AssertionError: SecurityPlugin.init() ran: expected false to be true 测的是产品行为(真回归);
AssertionError: this run took over a second, so second-precision stamps could have differed too: expected 1006 to be less than 1000 测的是实验前提:它守护的那条不变式当时是绿的,同一个 head 原样重排一次即成功。
穿着 AssertionError 外衣的时间测量,仍然是时间测量。(⛔ 这只改「怎么读一次红」,不改「哪些测试可以重排」——后者由别处管。)

跨 PR 相同签名(24h,按失败测试文件聚合):

历史信号:

  • ⚠️ 本 PR 过去 24h 已在队列失败 1 次(不含本次)。 内容未变而反复失败 ⇒ 高度怀疑 flaky 测试或与同组 PR 的语义冲突,重排不解决。
  • 过去 24h 队列共有 9 个失败构建(不含本次)。

分诊清单:

  1. 失败测试在本 PR 改动的包里 → 真回归,修 PR。
  2. 失败测试与本 PR 无关 → 看上面的「跨 PR 相同签名」;已有汇总 issue ⇒ flaky/环境问题实锤,去那张 issue 上谈,修好前重排只会再烧一轮全队列。
  3. 两者都不是 → 可能与同组 PR 语义冲突;等前面的 PR 落地或失败出队后再重排一次即可,不要连续重排。

Generated by Claude Code · merge-queue-triage workflow (#4859)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/xl skip-changeset PR has no user-facing published change; bypasses the changeset gate tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Scenario library: mid-tier card — 请假与假期余额 leave & balances (8–10 问数 + 4–6 write operations, row-scoped personas)

2 participants