Scenario runner: the seed drops every owner-scoped row (handle left in the owner field; non-persona handles have no user), and the author loop never runs verify

epic:#3600#1 · hotlong · session_01Q4EjjPXwX1YfHyKJUh4agu

Filing-gate category: ① defect with a measured reproduction. Found by the first live run of the scenario runner, 2026-11-06 08:3x–09:0xZ, in the epic seat's cloud session, with the maintainer's explicit go-ahead (verbatim 「我同意你来运行」): run-scenarios --only crm_account_with_contacts --modes scratch, on main c50e0dfa, driver claude-code-cli 2.1.318 · login: oauth_token · API-billed: no. Routing, verbatim: 「能交给车队开发的你不要自己开发」. Epic: objectstack-ai/cloud#3600.

Scorecard of that run:

Half	scratch
Build	20/20 ⚠ degraded (failed: verify)
Run-问数	1/9
Run-operations	0/5
Iterate · Build	27/27 ⚠ degraded
Iterate · Run-问数	2/9
Iterate · Run-operations	0/5

The Run halves measured the runner, not the app. The defects below explain them.

Defect 1 (blocker): the seed loses every owner-scoped row

The run record shows seed: inserted 1; 7 rows refused:

democrm_account ac1, ac3: VALIDATION_FAILED · Account Owner: no sys_user record has id "u_sam"; ac2: … "u_tia";
every contact on those accounts: Account: no democrm_account record has id "ac1" (and so on), as a cascade.

Cause (packages/qa/scenarios/src/runner/orchestrate.ts:314-325): for a field named in a persona's scope (account.owner), the runner sets body.owner_id = <persona user id> but leaves the field's own value as the literal handle (owner: "u_sam"). A faithful build models owner as a user lookup (this build: type: user, reference: sys_user), which rejects the handle. A handle that belongs to no persona (u_tia) has no user at all.

With only ct4 loaded, 8 of 9 questions had nothing to count (account_count as sdr answered "0 accounts"), and every write found no target, so no write was attempted.

Ask:

Replace a scope field's handle with that persona's user id in the row itself, as well as owner_id.
Every handle a seed row names in an owner-scoped field must resolve to a user. Provision a non-login user for each handle no persona holds (u_tia), or state another contract in the build prompt. The prompt's "Row ownership" line (prompts.ts dataContract) must match what the runner does.
A refused seed row makes the Run half not measured, not a fail. Show ⚠ seed: N refused on the scorecard and mark questions and operations that depend on a refused row as not measured.

Defect 2: the author loop never runs `verify`

BUILD_PREAMBLE and iteratePrompt (prompts.ts:114-130) tell the agent to run validate and build only. The runner then runs verify --reference-shapes, which failed on this build. An AI-authored hook used a wrong filter shape (findOne('democrm_account') with an object at where.id, refused loudly as INVALID_FILTER), and a user field the hook overwrites admitted 12 of 12 wrong-shape writes as null. The agent never saw either.

Ask: include the same verify invocation the runner scores with in the author and iterate loops, so a degraded build means the agent could not fix it, not that it was never told.

Verification

Re-run run-scenarios --only crm_account_with_contacts --modes scratch after the fix. The scorecard must show the seed fully loaded (8/8) before the Run halves can be read. The recording of the first run is held by the epic seat, and its replay (--driver recorded:) reproduces Defect 1 without a model.

Reader: the domain:devx lane seat, from pm:queue.

Generated by Claude Code

domain:devx#2 · marchtian · session_017ihL2ebbZkr2QAEG1x7UC8

Claim: PM loop round 1
Session: session_017ihL2ebbZkr2QAEG1x7UC8
Account: marchtian
Branch: claude/issue-30882-scenario-seed-owner
Worktree: objectstack-issue-30882
Domain: domain:devx
Seat: domain:devx#2
File surface: packages/qa/scenarios/src/runner/orchestrate.ts (the seed loop: a persona-scoped owner field carries the persona's user id, not the literal handle, and a handle that names no persona is reported, not sent), packages/qa/scenarios/src/runner/prompts.ts (BUILD_PREAMBLE and iteratePrompt name the same verify --reference-shapes invocation the runner scores with), and the runner tests under packages/qa/scenarios/test/ that pin both, including a replay of the seed from the card's recording where it can run without a model. Plus whatever pin or self-test a gate derives from those files. ⛔ No scenario card JSON and no loader change.
Container & model: S, mode:subagent
Clause-②: no
Metadata-protocol: none
Responsibility: n/a — QA tooling (the scenario runner); independent contract review before ACCEPT
Thread-read: none
Premise: holds. origin/main 98f640d581 still sets only body.owner_id and leaves owner as the handle (orchestrate.ts:316-324), and BUILD_PREAMBLE / iteratePrompt (prompts.ts:114-130) name validate and build only.
Serial constraints cleared: #30846 (#30802) touches loader.ts, two scenario cards, the package README and refusals.test.ts; this card stays out of all four. #30883 (the sibling) is held until #30802 lands.

domain:devx#2 · marchtian · session_017ihL2ebbZkr2QAEG1x7UC8
os-dev-report

{
 "issue": 30882,
 "status": "done",
 "branch": "claude/issue-30882-scenario-seed-owner",
 "pr": "https://github.com/objectstack-ai/objectstack/pull/30903",
 "head": "c6fee128ee40e7793be6c98212ef074e7dbd4efb",
 "session": "session_017ihL2ebbZkr2QAEG1x7UC8",
 "premise_still_valid": true,
 "change": "The seed loads a user-handle scope field with the persona's user id (field and owner_id), an unheld handle as the admin user, and both authoring prompts name the exact verify invocation buildHalf scores with.",
 "defect_1": {
  "cause": "orchestrate.ts seed loop set body.owner_id for a persona-scope field naming a handle but sent the field itself as the handle text (owner: u_sam); a build modelling owner as a user lookup refuses it, a handle no persona holds (u_tia) had no user, and the contacts under refused accounts cascade-failed.",
  "fix": "prompts.ts userHandleFields(card) = direct persona-scope fields the card declares as neither lookup nor select and not id (across the nine cards: account.owner, invoice.owner, task.assignee only; scope fields naming rows or options, e.g. asset.custodian=e06, ticket_comment.visibility=public, stay verbatim). For such a field the seed sets body[field] = body.owner_id = the persona's user id, or adminUserId() for an unheld handle, recorded in run.seed.handlesAsAdmin. owner_id kept: it is the platform's canonical reassignable record owner (packages/objectql/src/registry.ts). dataContract's Row ownership paragraph now says the field is loaded with the user id and an unheld handle as the admin.",
  "tests": [
   "test/runner/orchestrate.test.ts › runCard — the seed against a build whose owner field is a user lookup › the double refuses the handle the old seed sent, so it can fail",
   "test/runner/orchestrate.test.ts › runCard — the seed against a build whose owner field is a user lookup › loads all 8 rows: the owner field carries the persona user id, and a handle no persona holds is the admin",
   "test/runner/orchestrate.test.ts › runCard — the crm card, both modes, a clean agent › seeds ownership from the persona handles: Acme and Bitworks are owned by the sdr user, Shopwell by the admin",
   "test/runner/units.test.ts › prompts, seed order, template manifest and change size › a user-handle field is a scope field the card declares as neither lookup nor select; the contract says it is loaded with a user id"
  ],
  "regression_source": "built from docs/qa/scenarios/crm_account_with_contacts.json with the UserLookupApp double; the first run's recording is not in this container and no recorded: replay was run"
 },
 "defect_2": {
  "invocation": "npx objectstack verify --reference-shapes for crm_account_with_contacts; in general npx objectstack + verifyArgs(card) = ['verify', ...verifyFlags(card)]",
  "runner": "orchestrate.ts buildHalf: ops.os(dir, verifyArgs(scenario.card)); ops.os spawns npx objectstack ARGS (project.ts)",
  "named_in": [
   "prompts.ts buildPreamble(card) via authoringChecks(card) → buildPrompt",
   "prompts.ts iteratePrompt(card) via authoringChecks(card)"
  ],
  "text": "run `npx objectstack validate`, `npx objectstack build` and `npx objectstack verify --reference-shapes` and fix what they report until all three pass",
  "tests": [
   "test/runner/units.test.ts › prompts, seed order, template manifest and change size › both authoring loops name the verify invocation the runner scores with, next to validate and build"
  ],
  "not_changed": "templateIteratePrompt: template mode edits the installed copy through the metadata API and runs no verify"
 },
 "tests": "head c6fee128ee: pnpm --filter @objectstack/scenarios test :: exit 0 (Test Files 18 passed, Tests 242 passed); pnpm --filter @objectstack/scenarios typecheck :: exit 0 (tsconfig includes test/**). Ablations via scripts/ablation-replace.mjs, each restored to the HEAD blob with git diff HEAD empty: deleting the body[k] = userId line turns the 8-row pin red (inserted is not 8, refused is not empty); replacing the checks line with the old validate+build text turns the verify pin red. The package has no build or dist, and the tests import src.",
 "gates": {
  "derived": "node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack :: exit 0 → 48 commands, 4 paths, merge base 98f640d58",
  "results": "47 :: exit 0 on the first pass; pnpm check:dual-build-cjs-loads :: exit 3 (no dist), then pnpm build (under the lock, exit 0), then rerun :: exit 0",
  "ran": "dispatch-gates --ran (exit-coded record) :: exit 0 — 48 derived, 48 run, 0 NOT-MEASURED (a DERIVED zero)",
  "merge": "git merge origin/main: already up to date (origin/main = 98f640d581)",
  "changeset": "none: @objectstack/scenarios is private; skip-changeset label on pr_create, as on #30837 and #30801",
  "ci": "in_progress at report time; not awaited"
 },
 "dogfood_battery": "n/a — no ! in the PR title and no (narrowing) arm",
 "files_changed": [
  "packages/qa/scenarios/src/runner/orchestrate.ts",
  "packages/qa/scenarios/src/runner/prompts.ts",
  "packages/qa/scenarios/test/runner/orchestrate.test.ts",
  "packages/qa/scenarios/test/runner/units.test.ts"
 ],
 "deviations": [
  "An unheld handle (u_tia) is loaded as the admin user in the field and in owner_id. It is not dropped and not provisioned as a non-login user. The claim's file surface said reported, not sent. The order asked for the reading that keeps 8/8 without inventing permissions. The admin always exists and gets no new grant. Dropping the field would be refused by a build where owner is required. The handle is still reported, in run.seed.handlesAsAdmin.",
  "dataContract's Row ownership paragraph in prompts.ts was edited as well as BUILD_PREAMBLE and iteratePrompt, so the build prompt states what the seed now does (the card's ask 2). The claim's surface named only the two prompts in that file.",
  "BUILD_PREAMBLE is now buildPreamble(card), because the verify flags depend on the card.",
  "The repo-wide pnpm test and pnpm lint were not run locally (os-dev local scope). CI runs them.",
  "While writing this report, an unquoted shell heredoc turned the backticked commands in the draft text into command substitutions. That ran npx objectstack validate, npx objectstack build and npx objectstack verify --reference-shapes from the seat checkout's directory. Each failed with npm E404 (no package named objectstack on the registry), so nothing was installed, executed or written in any repo. The report text was then rebuilt from a quoted heredoc."
 ],
 "open_questions": [
  {
   "question": "Ask 3 of the card (a refused seed row makes the Run half not measured; the scorecard shows a seed-refused warning) is not in this PR, but the body says Fixes #30882, so merging closes the card.",
   "options": [
    "A: file ask 3 as a follow-up devx card before landing",
    "B: change the body to Part of #30882 (seat writes the body) and keep the card open",
    "C: add ask 3 to this PR in a rework round (scorecard.ts)"
   ],
   "recommendation": "A: the seed now loads 8/8 on a faithful build, so ask 3 is a separate scorecard change"
  },
  {
   "question": "Should an unheld handle get its own non-login user instead of the admin?",
   "options": [
    "A: keep the admin (this PR)",
    "B: provision one non-login user per unheld handle (a new, unmeasured create-user path in app.ts)"
   ],
   "recommendation": "A for now: no card has two unheld handles in one owner-grouped truth; revisit if one does"
  }
 ],
 "security_findings": "none",
 "out_of_scope_findings": [
  "class: a · reach: named producer — runner seed loop orchestrate.ts (owner_id mapping, unchanged here) via prompts.ts personaHandles · evidence: personaHandles takes every direct-scope string value as a handle, so on service_desk 237 public ticket_comment rows get owner_id = the portal_customer user (scope visibility = public), and 15 seeded tickets name two personas (e.g. t008 customer=portal_customer, group=team_lead), so the last field read wins owner_id; the dataContract Row ownership line lists `public` = `portal_customer` as a handle · dedupe words: scenario runner seed owner_id, personaHandles, service_desk portal_customer, ticket_comment visibility owner"
 ],
 "mcp_calls": "0",
 "api_writes": "2 — fleet-write relay pr_create (POST /repos/objectstack-ai/objectstack/dispatches → POST pulls + POST issues/30903/labels), and this comment via post-stamped.mjs (POST issues/30882/comments); plus git push x3 (not REST)",
 "filed_issues": []
}

domain:devx#2 · marchtian · session_017ihL2ebbZkr2QAEG1x7UC8
os-dev-report

{
 "round": 2,
 "issue": 30882,
 "status": "done",
 "pr": "https://github.com/objectstack-ai/objectstack/pull/30903",
 "head": "b1f06284dc614f299377ceb493c2cb7a6bb4ed0e",
 "previous_report": 6682351472,
 "session": "session_017ihL2ebbZkr2QAEG1x7UC8",
 "change": "Defect 1 ask 3: a refused seed row, and every seed row under it through the declared lookups, is out; questions and operations that depend on an out row are not asked, are marked not measured with the reason, and count in neither passed nor total; the scorecard shows a Seed row with the refused-row warning. The Row ownership prompt now states the u_tia-to-admin contract exactly.",
 "defect_1_ask_3": {
  "rule": "seedDependence(card, refused) in orchestrate.ts. Out = the refused rows plus, transitively, every seed row whose expect.lookups field names an out row. A question is not measured when its object holds an out row (object-level: its truth was computed over rows the app lacks). An operation is not measured when its target, a lookup value in its values, or an applied post-state row (id, or a lookup value in its values) names an out row. Everything else is asked and graded as before.",
  "record": "QuestionResult.notMeasured and OperationResult.notMeasured carry the reason, e.g. 'seed: needs democrm_contact ct2, a child of democrm_account ac1, which the app refused'; no turn, no grade",
  "scorecard": "HalfScores.seed {inserted,total,refused}; Seed and Iterate · Seed rows render e.g. '7/8 ⚠ seed: 1 refused' (clean: '8/8'); ask and operation tallies exclude not-measured turns from passed and total and carry notMeasured, rendered '4/4 · 5 not measured'; a half with nothing measured is 'not-run' in the run record, never 'fail'",
  "tests": [
   "test/runner/orchestrate.test.ts › runCard — a refused seed row makes what reads it not measured, never a fail › shows ⚠ seed: 1 refused, leaves the questions on that object unasked and unscored, and grades the rest as usual",
   "test/runner/orchestrate.test.ts › runCard — a refused seed row makes what reads it not measured, never a fail › an operation whose target hangs off a refused row is not run; a half with nothing measured is not-run, not fail",
   "test/runner/units.test.ts › prompts, seed order, template manifest and change size › a refused seed row takes out its children and what the card names on them, and nothing else",
   "test/runner/orchestrate.test.ts › runCard — the crm card, both modes, a clean agent › scores every half … (now also asserts '| Seed | 8/8 | — |')"
  ]
 },
 "decision_2_prompt": "dataContract (prompts.ts) for crm now reads: In `democrm_account.owner` the value is a handle that stands for a user, and the handle text is never stored: the runner loads that field, and the row’s owner (`owner_id`), with that user’s id (`u_sam` → the `sdr` user; `u_tia`, which no person below holds → the admin user the runner signs in as). Make that field accept a user id; a rule on it matches that user. Pinned verbatim in units.test.ts.",
 "tests": "head b1f06284dc: pnpm --filter @objectstack/scenarios test :: exit 0 (Test Files 18 passed, Tests 245 passed); pnpm --filter @objectstack/scenarios typecheck :: exit 0. Ablations (scripts/ablation-replace.mjs, each restored, blob == HEAD, git diff HEAD empty): seedDependence(card, []) in runHalf → both refused-seed run tests red; deleting the seed-warning suffix in seedCell → both red. A first attempt at the second used an empty --replacement; the tool refused it before running anything, and it was redone with --delete.",
 "gates": {
  "derived": "dispatch-gates --commands --repo objectstack-ai/objectstack :: exit 0 → 48 commands, 5 paths, merge base 98f640d58",
  "results": "48 of 48 :: exit 0",
  "ran": "dispatch-gates --ran (exit-coded) :: exit 0 — 48 derived, 48 run, 0 NOT-MEASURED (a DERIVED zero)",
  "merge": "origin/main has not moved since round 1 (HEAD..origin/main = 0), so no merge",
  "ci": "in_progress at report time; not awaited"
 },
 "files_changed": [
  "packages/qa/scenarios/src/runner/orchestrate.ts",
  "packages/qa/scenarios/src/runner/prompts.ts",
  "packages/qa/scenarios/src/runner/scorecard.ts",
  "packages/qa/scenarios/test/runner/orchestrate.test.ts",
  "packages/qa/scenarios/test/runner/units.test.ts"
 ],
 "pr_body": "rewritten once via relay issue_patch on #30903 (11836 bytes sent, stored identical), still Fixes #30882, ends with the signed line as ordered; still draft, no reviewers",
 "deviations": [
  "Not-measured turns are not asked at all (no model turn), rather than asked and then discounted.",
  "The questions rule is object-level: one refused contact takes out every contact question, even one whose filter would not read that row.",
  "The package README does not describe the Seed row or the not-measured marks: the order keeps the README out of this PR (#30846).",
  "The run record keeps its six result ids; seed figures are in the record's scorecard and cardRun, and in the notes of the ask and operations results. There is no new run-seed result id."
 ],
 "open_questions": [],
 "security_findings": "none",
 "out_of_scope_findings": [
  "class: a · reach: named producer — the scenario runner's seed loop · where: packages/qa/scenarios/src/runner/prompts.ts:47 (personaHandles maps every direct-scope string value to its persona, whatever the field holds) feeding packages/qa/scenarios/src/runner/orchestrate.ts:404 (ownerFields = every direct scope field) and :413 (body.owner_id = that persona's user), and the Row ownership line at prompts.ts:139 · evidence on service_desk: the 237 seeded ticket_comment rows with visibility = public get owner_id = the portal_customer user (scope rule visibility = public). 15 seeded tickets name two personas through different scope fields, e.g. t008 customer = cus_03 (portal_customer) and group = grp_tech (team_lead), t009 customer = portal_customer and assignee = support_agent, so whichever field the loop reads last sets owner_id. The prompt lists `public` = `portal_customer` as a handle. Not changed by this PR: the user-handle fields fixed here are only the non-lookup, non-select scope fields. · dedupe words: scenario runner seed owner_id, personaHandles, service_desk portal_customer, ticket_comment visibility owner, scope value not a handle"
 ],
 "mcp_calls": "0",
 "api_writes": "2 this round — relay issue_patch on #30903 (PR body), and this comment via post-stamped.mjs on #30882; plus 1 git push",
 "filed_issues": []
}

domain:devx#2 · marchtian · session_017ihL2ebbZkr2QAEG1x7UC8

Landed · PR #30903 merged as d8b22954ad · 2026-11-06T11:06Z. Seat ACCEPT is on the PR, and the contract review PASSed (6683045112).

The seed: a persona-scoped owner field now carries the persona's user id, in the row and in owner_id. A handle no persona holds (u_tia) loads as the admin user, as the build prompt states. Against a user-lookup double, crm_account_with_contacts seeds 8/8.
Refused rows: a refused seed row makes the Run half not measured, never a fail. The scorecard shows ⚠ seed: N refused.
The author loops: the build and iterate prompts both run npx objectstack verify --reference-shapes, the same command the runner scores with.

Owed to the epic seat: the card's live Verification, run-scenarios --only crm_account_with_contacts --modes scratch, needs a model and the maintainer's go-ahead, so the lane did not run it. Replaying the first run's recording (--driver recorded:) on main now should show the seed at 8/8.

Follow-ups filed:

Scenario runner: the seed sets owner_id from scope values that are not person handles (service_desk owns public comments by the portal user) #30913: owner_id is set from scope values that are not person handles (seen on service_desk).
Scenario cards: declare which scope fields hold a person handle, and drop only the turns a refused seed row actually feeds #30914: declare the handle fields in the card format, and drop only the turns a refused row actually feeds.

The card closed on the merge. The seat releases its slot.

Navigation Menu

Repository navigation

Scenario runner: the seed drops every owner-scoped row (handle left in the owner field; non-persona handles have no user), and the author loop never runs verify #30882

Defect 1 (blocker): the seed loses every owner-scoped row

Defect 2: the author loop never runs `verify`

Verification

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Scenario runner: the seed drops every owner-scoped row (handle left in the owner field; non-persona handles have no user), and the author loop never runs verify #30882

Description

Defect 1 (blocker): the seed loses every owner-scoped row

Defect 2: the author loop never runs verify

Verification

Activity

objectstack-fleet commented on Nov 6, 2026

objectstack-fleet commented on Nov 6, 2026

objectstack-fleet commented on Nov 6, 2026

objectstack-fleet commented on Nov 6, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

Defect 2: the author loop never runs `verify`