Idea: parity runs — replay one flow in two sessions and compare at named checkpoints

[tothvoj-gl]

Hi! I'm a senior React Native engineer. After Shopify's RN → native posts, I've been mapping what tooling exists for proving that a rewritten screen behaves like the original. agent-device already covers most of the driving and evidence side, so I wanted to ask here before building anything.

The gap

During a migration (or any cross-platform or refactor work) you want to answer: "in state X, do app A and app B show the same thing and emit the same events?" Shopify built this internally for the Shop app: screenshots and event windows captured from the RN and native apps at named checkpoints, with agents comparing event names, counts and payloads while ignoring values like timestamps (post).

With agent-device today I can approximate it:

• two named sessions on two simulators (one simulator can't run two copies of the same bundle id)
• the same .ad script replayed in each, which works when accessibility ids match
• screenshot, snapshot -i, logs and network dump per step
• then diffing everything by hand, because diff snapshot only compares against the previous baseline in the same session

What I have in mind

1. Named checkpoints in .ad scripts (something like checkpoint "payment-confirm"). Each one captures a bundle: screenshot, snapshot tree, and the log/network window since the previous checkpoint.
2. A paired run. Something like agent-device parity ./flow.ad --reference rn-ios --candidate native-ios, or the equivalent via the Node client. It replays the flow in both sessions and stores the bundles side by side.
3. A compact report with three parts:
   • a structural tree diff (labels, ids, roles; platform noise ignored)
   • a side-by-side screenshot composite for a vision reviewer
   • an event diff with ignore rules (timestamps, UUIDs)

Use cases beyond RN → native: iOS vs Android parity of the same RN app, before/after a refactor or RN upgrade, and old vs new design system.

Questions

• Is anything like this on the roadmap, or already possible in a way I've missed?
• If it fits, would you prefer it in core (a checkpoint command plus diff snapshot --against <session>), or as a separate package built on the Node.js client / a skill?
• I'm happy to build it. Any constraints I should know before starting (artifact layout, replay semantics, Maestro export)?

Thanks!