Repository navigation
feat(replay): capture screenshots at one pixel per dp by default with a filtered downscale - #906
Conversation
… a filtered downscale
dustinbyrne
left a comment
There was a problem hiding this comment.
nice!
some thoughts:
- considering this is additional work to apply bilinear filtering i think it could be worth adding a configuration option to toggle it on/off.
RenderThreadmedian time moving from 2.73 / 1.60 ms to 4.74 / 3.31 ms (if i'm reading this correctly) seems significant enough to warrant it. - with all these controls, it's getting to a point where we might want to expose a single, simple, smart control. something like
screenshotQuality: BEAUTIFUL | BALANCED | PERFORMANTthat dictates the scale, filtering, etc. - i've added some golden snapshots in #899. it'd be good to include this (it's not merged yet, so likely this is a follow up)
There was a problem hiding this comment.
LGTM, thanks for the benchmarks.
- This covers #766 but flips it from opt-in to the default. Worth a
Closes #766with a line on why? - This brings Android in line with iOS, which moved to 1 px/pt in PostHog/posthog-ios#967 after PostHog/posthog-ios#958 showed the cost of native-res frames.
- Flutter always sends
screenshotScale = 1.0, so Flutter Android stays at native res after this while Flutter iOS gets 1 px/pt. Can we open a follow-up to make it nullable there? - Need a RN plugin update too?
- On a filtering toggle, I'd keep it internal for now since
screenshotScalealready lets apps skip the extra pass.
| targetWidth: Int, | ||
| targetHeight: Int, | ||
| ): Bitmap { | ||
| val target = Bitmap.createBitmap(targetWidth, targetHeight, config ?: Bitmap.Config.ARGB_8888) |
There was a problem hiding this comment.
This allocates a new target bitmap every frame (about 1.4 MiB at 402x874 ARGB). #756 pooled the PixelCopy buffer to avoid per-frame allocations, so I think it might be worth pooling the 1x target too. Fine as a follow-up.
There was a problem hiding this comment.
🤖 An AI agent wrote this on behalf of @ioannisj.
Yeah, left it unpooled on purpose. Pooling it means a second slot in PixelCopyBitmapBuffer's lease logic, and I'd rather not widen that in this PR. It only allocates on the two-stage path (copy ratio above 2:1), at most once per throttle tick, so I'll pick it up as a follow-up.
ioannisj
left a comment
There was a problem hiding this comment.
🤖 An AI agent wrote this on behalf of @ioannisj.
@turnipdabeets added Closes #766 to the description with the why: it asked for density-aware sizing as an opt-in mode, and this ships it as the default since the storage saving is the point.
On Flutter, it's a bit narrower than that. Flutter screens are captured in Dart at one pixel per logical pixel already, so it's only native screens under captureNativeScreens that stay at native res, since the Dart option defaults to 1.0 and is always sent.
RN builds fine as is, the plugin pins posthog-android 3.72.9 and the bridge still compiles against Float?. RN apps that don't set screenshotScale pick up the 1x default once it bumps, but two plugin tests expect a 1f default (PosthogReactNativePluginModuleTest.kt:73 and :113) and need updating.
Once this is released I'll open follow-up PRs for both: make the Flutter option nullable and only send it when set, and bump RN with the test fix.
@dustinbyrne yeah you're reading it right, it's about 2 ms more on the RenderThread copy on that emulator. I'd keep filtering internal for now though, like @turnipdabeets suggested. The extra pass only runs above 2:1, so screenshotScale = 0.5f already skips it, and it's still cheaper than main's full-res copy today.
Agree on a single preset, I think that's the better home for a filtering choice too. It'd need to land on iOS, Flutter and RN as well, so I'd take it to sdk-specs as a follow-up rather than add it here.
And yeah, #899 as a follow-up. Whichever of the two lands second needs to re-record the default-size goldens (native-defaults.png at least).
| targetWidth: Int, | ||
| targetHeight: Int, | ||
| ): Bitmap { | ||
| val target = Bitmap.createBitmap(targetWidth, targetHeight, config ?: Bitmap.Config.ARGB_8888) |
There was a problem hiding this comment.
🤖 An AI agent wrote this on behalf of @ioannisj.
Yeah, left it unpooled on purpose. Pooling it means a second slot in PixelCopyBitmapBuffer's lease logic, and I'd rather not widen that in this PR. It only allocates on the two-stage path (copy ratio above 2:1), at most once per throttle tick, so I'll pick it up as a follow-up.
Sizes the secure-window frame with the same dp-scaled target as #906's normal path and masks before the downscale. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
💡 Motivation and Context
Session replay screenshots upload at native resolution today, so a 3x phone sends 9 pixels for every dp. We agreed to default both iOS and Android to one pixel per dp to cut storage. On a 1206x2622 screen a 3x frame is about 62 KB and a 1x frame about 16 KB at WebP quality 30.
Closes #766. It asked for density-aware sizing as an opt-in mode, and this ships it as the default instead, since the storage saving is the point and we agreed on it as a team. An explicit
screenshotScalestill opts out.This deliberately reverses #761's "keep current capture fidelity" default. iOS already uploads at one pixel per point since PostHog/posthog-ios#967, and PostHog/posthog-ios#994 adds the same
screenshotScaleoption there with the same meaning and clamping.Public API change:
sessionReplayConfig.screenshotScalechanges fromFloattoFloat?.nullis the new default and means one pixel per dp: a multiplier of1/density, clamped to 0.1–1.0, so a screen with density ≤ 1 still captures at full resolution.screenshotScale = 1f.nullinstead of1.0.screenshotScaleshipped in 3.63.0 as@PostHogExperimental. The JVM signatures change, so this breaks binary compatibility for code compiled against the old getter or setter:screenshotScale = 0.5fand JavasetScreenshotScale(0.5f)still compile, so callers that only assign it just need a rebuild.Floatno longer compiles and needs a null check.floatcompiles, but throwsNullPointerExceptionwhile it's unset, which is now the default.screenshotScaleand are unchanged.I think a minor with a breaking-change note in the changeset is the right call here, since the option is experimental and only shipped in 3.63.0. Happy to hold it for a major if anyone feels otherwise.
Downsampling
At 1x, copying straight into the smaller bitmap (as #756 does) loses thin strokes. HWUI's
Readback::copySurfaceIntoresamples with one bilinear tap and no mipmaps (libs/hwui/Readback.cpp,SkFilterMode::kLinear). At exactly 3:1 each output pixel centre lands on a source pixel centre, so in practice it's nearest neighbour and two of every three source columns get dropped. A 1 px line test shows it: lines at x = 3k+1 survive at full black, lines at 3k and 3k+2 disappear. At 9sp, "Terms" reads as "ierms" and "VAT" as "VAI".So when the screen is more than 2x the target size, the capture now copies into a 2x intermediate and downscales 2:1 with a filtered
Canvas.drawBitmap. At 2:1 a bilinear tap averages exactly 2x2 pixels. Masks are still painted on the copied bitmap before the downscale, with outward-rounded rects as in #756. At ratios of 2 or less (an explicit0.5, or density ≤ 2) the copy goes straight into the target as before.Emulator results (1206x2622 at density 3, target 402x874, WebP quality 30, interleaved runs). These are relative numbers from one emulator, not a device matrix:
copySurfaceIntomedian (static / animated)main's current full-resolution default.There's no sdk-specs requirement on screenshot scale or resolution, and posthog-js has no screenshot capture.
Heads up on the hybrid SDKs:
screenshotScale(Dart default1.0), so native screens recorded withcaptureNativeScreensstay at native resolution on Android. Worth a follow-up there.posthog-android3.72.9, so nothing changes until it bumps. When it does, two plugin tests that expect a1fdefault need updating, and React Native apps that don't setscreenshotScaleget the new default.💚 How did you test it?
./gradlew :posthog-android:testDebugUnitTest: 789 tests, all pass.ceil(px / density)frames, including no extra pixel from float error at density 3.50.5and density 2 copying in one step, and density 1 capturing at full sizenull1f, 12 of these fail. With outward mask rounding or the float tolerance reverted, their tests fail too.make checkFormat,make api,./gradlew apiCheck.wm size 1206x2622,wm density 480) compared the four options above with Perfetto. That used a bench activity, not the SDK capture path. I haven't run the SDK path on a device after the change.📝 Checklist
If releasing new changes
pnpm changesetto generate a changeset file🤖 Agent context
DRI: @ioannisj
Autonomy: Human-driven (agent-assisted)