Skip to content

feat(replay): capture screenshots at one pixel per dp by default with a filtered downscale - #906

Merged
ioannisj merged 1 commit into
mainfrom
feat/replay-screenshot-1x-default
Oct 30, 2026
Merged

ioannisj merged 1 commit into
mainfrom
feat/replay-screenshot-1x-default

Conversation

@ioannisj

@ioannisj ioannisj commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

💡 Motivation and Context

Session replay screenshots upload at native resolution today, so a 3x phone sends 9 pixels for every dp. We agreed to default both iOS and Android to one pixel per dp to cut storage. On a 1206x2622 screen a 3x frame is about 62 KB and a 1x frame about 16 KB at WebP quality 30.

Closes #766. It asked for density-aware sizing as an opt-in mode, and this ships it as the default instead, since the storage saving is the point and we agreed on it as a team. An explicit screenshotScale still opts out.

This deliberately reverses #761's "keep current capture fidelity" default. iOS already uploads at one pixel per point since PostHog/posthog-ios#967, and PostHog/posthog-ios#994 adds the same screenshotScale option there with the same meaning and clamping.

Public API change: sessionReplayConfig.screenshotScale changes from Float to Float?.

  • null is the new default and means one pixel per dp: a multiplier of 1/density, clamped to 0.1–1.0, so a screen with density ≤ 1 still captures at full resolution.
  • An explicit value keeps its meaning, a multiplier of the physical width and height, clamped to 0.1–1.0. Apps that want native resolution set screenshotScale = 1f.
  • NaN and infinite values now reset to null instead of 1.0.

screenshotScale shipped in 3.63.0 as @PostHogExperimental. The JVM signatures change, so this breaks binary compatibility for code compiled against the old getter or setter:

-	public final fun getScreenshotScale ()F
+	public final fun getScreenshotScale ()Ljava/lang/Float;
-	public final fun setScreenshotScale (F)V
+	public final fun setScreenshotScale (Ljava/lang/Float;)V
  • Kotlin screenshotScale = 0.5f and Java setScreenshotScale(0.5f) still compile, so callers that only assign it just need a rebuild.
  • Kotlin code that reads it into a Float no longer compiles and needs a null check.
  • Java code that reads it into a float compiles, but throws NullPointerException while it's unset, which is now the default.
  • Constructors don't take screenshotScale and are unchanged.

I think a minor with a breaking-change note in the changeset is the right call here, since the option is experimental and only shipped in 3.63.0. Happy to hold it for a major if anyone feels otherwise.

Downsampling

At 1x, copying straight into the smaller bitmap (as #756 does) loses thin strokes. HWUI's Readback::copySurfaceInto resamples with one bilinear tap and no mipmaps (libs/hwui/Readback.cpp, SkFilterMode::kLinear). At exactly 3:1 each output pixel centre lands on a source pixel centre, so in practice it's nearest neighbour and two of every three source columns get dropped. A 1 px line test shows it: lines at x = 3k+1 survive at full black, lines at 3k and 3k+2 disappear. At 9sp, "Terms" reads as "ierms" and "VAT" as "VAI".

So when the screen is more than 2x the target size, the capture now copies into a 2x intermediate and downscales 2:1 with a filtered Canvas.drawBitmap. At 2:1 a bilinear tap averages exactly 2x2 pixels. Masks are still painted on the copied bitmap before the downscale, with outward-rounded rects as in #756. At ratios of 2 or less (an explicit 0.5, or density ≤ 2) the copy goes straight into the target as before.

Emulator results (1206x2622 at density 3, target 402x874, WebP quality 30, interleaved runs). These are relative numbers from one emulator, not a device matrix:

Option 9sp / 11sp text WebP RenderThread copySurfaceInto median (static / animated) Extra CPU Peak bitmap memory
PixelCopy straight into 1x strokes dropped 17.4 KB 2.73 / 1.60 ms 0 1.34 MiB
PixelCopy 3x + bilinear to 1x identical to the above 17.4 KB 7.15 / 5.34 ms 1.1 ms 13.4 MiB
PixelCopy 2x + 2:1 bilinear (this PR) matches the native screen 15.9 KB 4.74 / 3.31 ms 0.8 ms 6.7 MiB
PixelCopy 3x + halve + bilinear matches the native screen 15.3 KB 8.49 / 5.87 ms 2.6 ms 16.4 MiB
  • The filtered frame is about 8% smaller, since the smoother image compresses better.
  • It costs about 2 ms more RenderThread copy time than copying straight into 1x, and about 0.8 ms on the replay thread.
  • The pooled buffer holds a 2x intermediate, about 5.4 MiB more on this screen. That's still about half of main's current full-resolution default.
  • Frame p90 with an animation running showed no measurable difference.

There's no sdk-specs requirement on screenshot scale or resolution, and posthog-js has no screenshot capture.

Heads up on the hybrid SDKs:

  • posthog-flutter captures Flutter screens in Dart at one pixel per logical pixel already, but it always sends screenshotScale (Dart default 1.0), so native screens recorded with captureNativeScreens stay at native resolution on Android. Worth a follow-up there.
  • The posthog-react-native plugin pins posthog-android 3.72.9, so nothing changes until it bumps. When it does, two plugin tests that expect a 1f default need updating, and React Native apps that don't set screenshotScale get the new default.

💚 How did you test it?

  • ./gradlew :posthog-android:testDebugUnitTest: 789 tests, all pass.
  • New tests cover:
    • the default giving ceil(px / density) frames, including no extra pixel from float error at density 3.5
    • a 2x PixelCopy destination at density 3 while the uploaded frame is the target size
    • an explicit 0.5 and density 2 copying in one step, and density 1 capturing at full size
    • masks covering every pixel of the masked view after the downscale, with mask edges off the 3 px grid
    • config defaults, clamping, and NaN/infinity resetting to null
    • a 1 px line pattern keeping every phase through the 2:1 downscale
  • With the default set back to 1f, 12 of these fail. With outward mask rounding or the float tolerance reverted, their tests fail too.
  • make checkFormat, make api, ./gradlew apiCheck.
  • An emulator bench (API 36, wm size 1206x2622, wm density 480) compared the four options above with Perfetto. That used a bench activity, not the SDK capture path. I haven't run the SDK path on a device after the change.

📝 Checklist

  • All commits are signed (see Commit signing)
  • I reviewed the submitted code.
  • I added tests to verify the changes.
  • I updated the docs if needed.
  • No breaking change or entry added to the changelog.

If releasing new changes

  • Ran pnpm changeset to generate a changeset file

🤖 Agent context

DRI: @ioannisj
Autonomy: Human-driven (agent-assisted)

@ioannisj ioannisj self-assigned this Oct 29, 2026
@ioannisj
ioannisj marked this pull request as ready for review October 29, 2026 14:32
@ioannisj
ioannisj requested a review from a team as a code owner October 29, 2026 14:32

@dustinbyrne dustinbyrne left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice!

some thoughts:

  • considering this is additional work to apply bilinear filtering i think it could be worth adding a configuration option to toggle it on/off. RenderThread median time moving from 2.73 / 1.60 ms to 4.74 / 3.31 ms (if i'm reading this correctly) seems significant enough to warrant it.
  • with all these controls, it's getting to a point where we might want to expose a single, simple, smart control. something like screenshotQuality: BEAUTIFUL | BALANCED | PERFORMANT that dictates the scale, filtering, etc.
  • i've added some golden snapshots in #899. it'd be good to include this (it's not merged yet, so likely this is a follow up)

@turnipdabeets turnipdabeets left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks for the benchmarks.

  • This covers #766 but flips it from opt-in to the default. Worth a Closes #766 with a line on why?
  • This brings Android in line with iOS, which moved to 1 px/pt in PostHog/posthog-ios#967 after PostHog/posthog-ios#958 showed the cost of native-res frames.
  • Flutter always sends screenshotScale = 1.0, so Flutter Android stays at native res after this while Flutter iOS gets 1 px/pt. Can we open a follow-up to make it nullable there?
  • Need a RN plugin update too?
  • On a filtering toggle, I'd keep it internal for now since screenshotScale already lets apps skip the extra pass.

targetWidth: Int,
targetHeight: Int,
): Bitmap {
val target = Bitmap.createBitmap(targetWidth, targetHeight, config ?: Bitmap.Config.ARGB_8888)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This allocates a new target bitmap every frame (about 1.4 MiB at 402x874 ARGB). #756 pooled the PixelCopy buffer to avoid per-frame allocations, so I think it might be worth pooling the 1x target too. Fine as a follow-up.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 An AI agent wrote this on behalf of @ioannisj.

Yeah, left it unpooled on purpose. Pooling it means a second slot in PixelCopyBitmapBuffer's lease logic, and I'd rather not widen that in this PR. It only allocates on the two-stage path (copy ratio above 2:1), at most once per throttle tick, so I'll pick it up as a follow-up.

@ioannisj ioannisj left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 An AI agent wrote this on behalf of @ioannisj.

@turnipdabeets added Closes #766 to the description with the why: it asked for density-aware sizing as an opt-in mode, and this ships it as the default since the storage saving is the point.

On Flutter, it's a bit narrower than that. Flutter screens are captured in Dart at one pixel per logical pixel already, so it's only native screens under captureNativeScreens that stay at native res, since the Dart option defaults to 1.0 and is always sent.

RN builds fine as is, the plugin pins posthog-android 3.72.9 and the bridge still compiles against Float?. RN apps that don't set screenshotScale pick up the 1x default once it bumps, but two plugin tests expect a 1f default (PosthogReactNativePluginModuleTest.kt:73 and :113) and need updating.

Once this is released I'll open follow-up PRs for both: make the Flutter option nullable and only send it when set, and bump RN with the test fix.

@dustinbyrne yeah you're reading it right, it's about 2 ms more on the RenderThread copy on that emulator. I'd keep filtering internal for now though, like @turnipdabeets suggested. The extra pass only runs above 2:1, so screenshotScale = 0.5f already skips it, and it's still cheaper than main's full-res copy today.

Agree on a single preset, I think that's the better home for a filtering choice too. It'd need to land on iOS, Flutter and RN as well, so I'd take it to sdk-specs as a follow-up rather than add it here.

And yeah, #899 as a follow-up. Whichever of the two lands second needs to re-record the default-size goldens (native-defaults.png at least).

targetWidth: Int,
targetHeight: Int,
): Bitmap {
val target = Bitmap.createBitmap(targetWidth, targetHeight, config ?: Bitmap.Config.ARGB_8888)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 An AI agent wrote this on behalf of @ioannisj.

Yeah, left it unpooled on purpose. Pooling it means a second slot in PixelCopyBitmapBuffer's lease logic, and I'd rather not widen that in this PR. It only allocates on the two-stage path (copy ratio above 2:1), at most once per throttle tick, so I'll pick it up as a follow-up.

@ioannisj
ioannisj merged commit 5f5cee2 into main Oct 30, 2026
17 checks passed
@ioannisj
ioannisj deleted the feat/replay-screenshot-1x-default branch October 30, 2026 10:19
turnipdabeets added a commit that referenced this pull request Oct 30, 2026
Sizes the secure-window frame with the same dp-scaled target as #906's normal path and masks before the downscale.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Session replay: support opt-in density-aware screenshot sizing

3 participants