fix: rotate landscape screenshots in the pixels, not with an exif tag - #93
Conversation
In landscape, XCTest returns the panel's portrait pixels plus an orientation, which the png and jpeg encoders wrote as an EXIF tag. Decoders that ignore the tag saw a portrait image while the ui tree and taps use the rotated screen, so coordinates read off a screenshot landed somewhere else. The image is now redrawn upright before it is encoded.
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Advanced Run ID: 📒 Files selected for processing (3)
Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review. WalkthroughThe screenshot handler now rotates image pixels upright before PNG or JPEG encoding, with fallback behavior when rotation is unavailable. New test helpers parse PNG and JPEG dimensions and EXIF orientation. RPC tests capture landscape screenshots and check their dimensions and upright EXIF orientation. Priority: ⬇️ Low Merge Risk: ⚪ Minimal · up to The screenshot rotation change is mergeable subject to normal checks. The supplied evidence establishes no actionable defect in screenshot encoding or the landscape tests. 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
Summary
In landscape,
device.screenshotreturned an image that disagreed with the UI tree and with tap coordinates.XCTest hands a rotated screen back as the panel's portrait pixels plus an orientation, and both encoders wrote that orientation as an EXIF tag (8, "rotate 270"). A decoder that ignores the tag (Go's
imagepackages in mobilecli, among others) sees a 1206x2622 portrait image, whiledevice.dump.uianddevice.io.tapwork in the rotated 874x402 space. Coordinates read off the screenshot land somewhere else.Screenshot.swift: when the captured image carries an orientation, it is redrawn upright before encoding, so the rotation is in the pixels. No EXIF rotation is left for the viewer to apply.pngRepresentation/jpegDataon the original image).Result on iPhone 17 Pro simulator (iOS 26.0), Safari in landscape
device.dump.uiThrough mobilecli with this runner,
mobilecli screenshotin landscape is 2622x1206, and cropping it at the rectangledump uireports for a button (x3) lands exactly on that button.Testing
tests/rpc.test.ts("device.screenshot in landscape"), for png and jpeg:tests/image.ts: a small png/jpeg reader for the size and the orientation tag, so the tests need no image library.device.io.swipe › swipes over an explicit duration, which passes on its own and fails when it runs after another swipe (about 1390ms against a 1400ms floor); it does not involve the screenshot handler.swiftlint lint --strictreports nothing on the added lines.Not covered
The MJPEG stream has no orientation handling either; it is not changed or tested here.