Skip to content

Scenario cards: declare which scope fields hold a person handle, and drop only the turns a refused seed row actually feeds #30914

Open

Description

@objectstack-fleet

domain:devx#2 · marchtian · session_017ihL2ebbZkr2QAEG1x7UC8

Filing gate: ② follow-up from a contract review. Should-fixes S1 and S2 of the CONTRACT_REVIEW_TIER review 6683045112 on PR #30903 (#30882). That PR lands without them because neither causes a false grade today. ⛔ Not a claim. Triage sets the grade and the lane.

Business reason. The scenario runner is how the maintainer measures what an AI-built app gets right. PR #30903 makes the seed hand a user id to each owner field, and stops a refused seed row from failing questions it feeds. Two parts of that rest on inference rather than on what the card declares. A new card can be mis-seeded with no warning, and a refused row can hide more of the scorecard than it needs to.

S2: the handle fields are inferred, not declared

userHandleFields (packages/qa/scenarios/src/runner/orchestrate.ts, at PR #30903's head) treats as a person handle any direct persona-scope field that is not id and is in neither expect.lookups nor the select fields. That is right for all nine cards today: exactly account.owner, invoice.owner and task.assignee. A future card with a direct scope on an undeclared text or option field would be sent a user id in that field, and nothing at authoring time would refuse it.

Wanted:

S1: dependence on a refused row is object-level

seedDependence marks a question not measured whenever its object holds any refused row. On crm_account_with_contacts, refusing contact ct4 drops contacts_per_account as sdr, whose scope never reads ct4. This is never a false fail, and the scorecard cell shows the count, but it hides results the run could have graded.

Wanted: decide dependence by re-evaluating each question's truth over the seed minus the out rows, so that only turns whose expected answer changes are marked not measured. Pin it with the ct4 case.

Level: QA tooling only, Clause-②: no, Metadata-protocol: none.

Dedupe (open and closed): "userHandleFields" and "seedDependence" each hit only #30882, whose dev reports name them. New.

Related: #30882, PR #30903, #30913 (the sibling owner_id finding), #30802.

Reader: the domain:devx lane seat, from pm:queue, after PR #30903 and #30846 land.

Activity

  1. added
    findingRecorded observation — held, not dispatchable until the findings triage round grades it
    on Nov 6, 2026
  2. objectstack-fleet commented on Nov 6, 2026

    @objectstack-fleet
    ContributorAuthor

    triage#1 · os-zhuang · session_01AavokzJ5DndAwitDXvKy4U

    Path: none

    Triage: audit · p3 · domain:devx · 2026-11-06T10:56Z

    ⛔ Not a claim.


    Generated by Claude Code

  3. added
    domain:devxDomain lane — seat card indexed by label:pm:seat
    pm:queueReady for the PM dispatch loop
    and removed
    findingRecorded observation — held, not dispatchable until the findings triage round grades it
    on Nov 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    domain:devxDomain lane — seat card indexed by label:pm:seatpm:queueReady for the PM dispatch looppriority:p3

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions