Skip to content

feat: add experimental screenshotScale option for default replay screenshots - #994

Merged
ioannisj merged 5 commits into
mainfrom
feat/replay-screenshot-scale-option
Oct 30, 2026
Merged

ioannisj merged 5 commits into
mainfrom
feat/replay-screenshot-scale-option

Conversation

@ioannisj

@ioannisj ioannisj commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

💡 Motivation and Context

Follow-up to #967, which uploads every default-path screenshot at one pixel per point. Adds an experimental sessionReplayConfig.screenshotScale so an app can pick a different resolution, and makes the resize smooth instead of nearest-neighbour.

Public API change: a new @objc public var screenshotScale: NSNumber? on PostHogSessionReplayConfig.

It means the same thing as posthog-android's screenshotScale: a multiplier of the screen's native resolution, clamped to 0.1 to 1.0. On a 3x iPhone, 1.0 uploads native 3x and 0.5 uploads 1.5 pixels per point. One pixel per point isn't a fixed multiplier (it's 1 / screen.scale), so the option is nullable, and nil (the default) keeps the #967 behaviour. NaN and infinite values reset it to nil. It's an NSNumber? because Objective-C can't represent CGFloat?, same as sampleRate.

How it works:

  • Capture doesn't change. drawHierarchy still renders at native scale on main, and the resize stays in the encode-time redraw on the replay queue, off main. It now targets ceil(points × native × screenshotScale) pixels instead of always one pixel per point.
  • Masks are painted on a native-resolution copy before the resize, so resampling can only blend their black into neighbouring pixels, never what they cover. Each mask is rounded out to whole native pixels, so masks can grow by up to one native pixel at their edges. This costs a native-size buffer (about 12.6 MB at 1206×2622) for masked frames only.
  • At 1.0 with nothing to mask, the native frame is uploaded as is, with no redraw.
  • Wireframe mode is unchanged.

The second commit switches the redraw from .none to .medium interpolation. Nearest-neighbour drops thin strokes when downsampling 3:1 ("Settings" comes out as "Sellings" at 9pt), and any 1 px line that falls on a skipped source column disappears.

Redraw time on an iPhone 17 Pro (iOS 26.7.1), a 402×874 pt screen of 9 to 20pt labels at 3x, 200 redraws on the replay queue per row, median / p95 over two runs:

output filter median ms p95 ms WebP q30 bytes
1x none 1.52 / 1.46 1.62 / 2.08 29,726
1x low 1.69 / 1.67 2.25 / 1.72 29,726
1x medium 4.58 / 4.45 4.86 / 4.76 27,314
1x high 8.04 / 8.03 8.43 / 8.37 27,140
1.5x none 3.38 / 3.25 3.53 / 3.40 49,608
1.5x low 9.84 / 9.76 10.96 / 10.13 46,406
1.5x medium 7.06 / 7.06 7.49 / 7.43 46,372
1.5x high 10.92 / 10.91 11.44 / 11.17 45,408

low is the same as none at 3:1. medium reads about as well as high for a bit over half the cost, and uploads 7 to 8% smaller than none. At the default 1x it costs about 3 ms more per frame, on a background queue, at most once a second.

Cross-SDK:

  • posthog-android has the same name, meaning and clamping. Its parallel change defaults to one pixel per dp and resamples with a filtered 2:1 downscale. On the same screens at 402×874, the two platforms' uploads are within 3% in bytes and look alike.
  • posthog-react-native has screenshotScale, documented as Android only. iOS could be forwarded later.
  • posthog-js records the DOM through rrweb, so there's no screenshot path.
  • sdk-specs has nothing on screenshot size or scale. session-replay-privacy asks for masks to be painted into the bitmap before encoding, and they still are.

Heads up for #911: it adds its own screenshotScale (in pixels per point) and a maskScale in the same places. When it's rebased it should read this option instead, and set wireframe.screenshotPixelScale to the scale its GPU frame was rendered at.

💚 How did you test it?

New tests in PostHogReplayScreenshotSizeTest (Swift Testing, iOS simulator):

  • an unset screenshotScale uploads one pixel per point, masked or not
  • 0.5 and 1.0 upload that fraction of native resolution, masked or not
  • clamping to 0.1 to 1.0, and NaN/±infinity resetting to nil
  • a fractional output size rounds up and leaves no pixel undrawn
  • a mask lands on the secret's pixels at 1.5x
  • a mask edge that falls mid-pixel covers that whole native pixel
  • an unmasked frame at native resolution is returned without a redraw
  • downsampling keeps a one-pixel line whichever source pixel it falls on
  • no pixel of a resized masked frame shows any of the secret, at 1x and 1.5x, with the secret offset by 0, ⅓ and ⅔ pt

I checked that each one fails with its piece of the change reverted (no outward rounding, truncating size, no native skip, scale ignored, .none filter, masks painted after the resize with a wider filter). PostHogReplayScreenshotSizeTest, PostHogMaskFailClosedTest and RRWireframeTest pass (41 tests), including #969's corner tests.

PostHogMaskFailClosedTest forced a failed render with an image 1/8 pt wide, which only failed because the renderer truncated it to zero pixels. With rounding up it renders one pixel, so the fixture is now 1/10,000 pt wide. The fail-closed behaviour itself didn't change.

Masking goldens: I re-recorded 24 of them. In each one, the only changed pixels are in the masked panel and every one got darker: mask edges going from a half-blended row to solid black. The original panels didn't change, and 4 cases were identical. Painting masks before the resize leaves all 28 pixel-identical, since the snapshot suite masks 1x images and never resizes. The pinned Xcode 26.3 isn't installed locally, so I recorded under Xcode 27.1 on the pinned iOS 26.2 runtime. Main's own goldens verify unchanged under that setup, so CI's make maskSnapshots should agree.

Before rebasing onto main, I ran the full iOS simulator suite. It was green apart from the fixture above (since fixed) and one existing known-issue marker in PostHogMaskFlattenedContentTest. Also before the rebase, the SDK built for iOS, macOS, Mac Catalyst, tvOS, watchOS and visionOS. make lint passes on the rebased branch.

📝 Checklist

  • All commits are signed (see Commit signing)
  • I reviewed the submitted code.
  • I added tests to verify the changes.
  • I updated the docs if needed.
  • No breaking change or entry added to the changelog.

If releasing new changes

  • Ran pnpm changeset to generate a changeset file

🤖 Agent context

DRI: @ioannisj
Autonomy: Human-driven (agent-assisted)

@ioannisj ioannisj self-assigned this Oct 29, 2026

@posthog posthog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PostHog Review · Standard · reviewed c554f80: 1 Must fix.

This is the only automatic review while this pull request is a draft. Comment @posthog review to review new changes sooner. Otherwise, a review of the new changes runs when you mark it ready for review.

See this review and its findings in PostHog Review. Comment @posthog review --deep for a Deep review. A Deep review takes longer and looks deeper than a Standard review.

Comment thread PostHog/Replay/RRWireframe.swift Resolved
@github-actions

github-actions Bot commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

posthog-ios Compliance Report

Date: 2026-10-30T06:33:45.910143+00:00
Duration: 221827ms

⚠️ Some Tests Failed

46/47 tests passed, 1 failed


Capture Tests

⚠️ 29/30 tests passed, 1 failed

View Details
Test Status Duration
Format Validation.Event Has Required Fields ✅ 2816ms
Format Validation.Event Has Uuid ✅ 2760ms
Format Validation.Event Has Lib Properties ✅ 2680ms
Format Validation.Distinct Id Is String ✅ 2621ms
Format Validation.Token Is Present ✅ 417ms
Format Validation.Custom Properties Preserved ✅ 2629ms
Format Validation.Event Has Timestamp ✅ 2702ms
Format Validation.Non Utc Event Timestamp Is Converted To Utc ✅ 2785ms
Retry Behavior.Retries On 503 ✅ 11747ms
Retry Behavior.Does Not Retry On 400 ✅ 4739ms
Retry Behavior.Does Not Retry On 401 ✅ 2350ms
Retry Behavior.Respects Retry After Header ✅ 7680ms
Retry Behavior.Implements Backoff ✅ 15404ms
Retry Behavior.Retries On 500 ✅ 9543ms
Retry Behavior.Retries On 502 ✅ 9280ms
Retry Behavior.Retries On 504 ✅ 5406ms
Retry Behavior.Max Retries Respected ❌ 21785ms
Deduplication.Generates Unique Uuids ✅ 2991ms
Deduplication.Preserves Uuid On Retry ✅ 9259ms
Deduplication.Preserves Uuid And Timestamp On Retry ✅ 16344ms
Deduplication.Preserves Uuid And Timestamp On Batch Retry ✅ 7830ms
Deduplication.No Duplicate Events In Batch ✅ 2985ms
Deduplication.Different Events Have Different Uuids ✅ 2888ms
Compression.Sends Gzip When Enabled ✅ 2655ms
Batch Format.Uses Proper Batch Structure ✅ 2832ms
Batch Format.Flush With No Events Sends Nothing ✅ 253ms
Batch Format.Multiple Events Batched Together ✅ 2901ms
Error Handling.Does Not Retry On 403 ✅ 4729ms
Error Handling.Does Not Retry On 413 ✅ 4784ms
Error Handling.Retries On 408 ✅ 5355ms

Failures

retry_behavior.max_retries_respected

Expected 4 requests, got 6

Feature_Flags Tests

✅ 17/17 tests passed

View Details
Test Status Duration
Request Payload.Request With Person Properties Device Id ✅ 2799ms
Request Payload.Flags Request Uses V2 Query Param ✅ 2815ms
Request Payload.Flags Request Hits Flags Path Not Decide ✅ 2740ms
Request Payload.Flags Request Omits Authorization Header ✅ 2751ms
Request Payload.Token In Flags Body Matches Init ✅ 2765ms
Request Payload.Groups Round Trip ✅ 2795ms
Request Payload.Groups Default To Empty Object ✅ 2665ms
Request Payload.Disable Geoip False Propagates As Geoip Disable False ✅ 2701ms
Request Payload.Disable Geoip Omitted Defaults To False ✅ 2792ms
Request Payload.Flag Keys To Evaluate Contains Only Requested Key ✅ 2837ms
Request Lifecycle.No Flags Request On Init Alone ✅ 75ms
Request Lifecycle.No Flags Request On Normal Capture ✅ 2829ms
Request Lifecycle.Two Flag Calls Produce Two Remote Requests ✅ 5373ms
Request Lifecycle.Mock Response Value Is Returned To Caller ✅ 2792ms
Retry Behavior.Retries Flags On 502 ✅ 2823ms
Retry Behavior.Retries Flags On 504 ✅ 2801ms
Side Effect Events.Get Feature Flag Captures Feature Flag Called Event ✅ 2961ms

@ioannisj

Copy link
Copy Markdown
Contributor Author

@PostHog review

@posthog

posthog Bot commented Oct 29, 2026

Copy link
Copy Markdown
Contributor

@ioannisj PostHog Review is reviewing this pull request.

@posthog posthog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PostHog Review · Standard · reviewed f7e4a62: no findings.

@dustinbyrne dustinbyrne left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is awesome, love to see how dramatic the performance characteristics are depending on the size of the output buffer and depth of filtering. i feel like we have a really good handle on snapshot performance at this point.

some comments similar to PostHog/posthog-android#906 (review)

looking at the reported performance characteristics, 1x/no filtering is roughly 3x faster than the default decided in this PR. i'd personally consider that as the default given it's less likely to affect downstream applications, though, i suppose the iOS case is different because we're not consuming render thread time in this case. just food for thought.

@ioannisj

ioannisj commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor Author

this is awesome, love to see how dramatic the performance characteristics are depending on the size of the output buffer and depth of filtering. i feel like we have a really good handle on snapshot performance at this point.

some comments similar to PostHog/posthog-android#906 (review)

looking at the reported performance characteristics, 1x/no filtering is roughly 3x faster than the default decided in this PR. i'd personally consider that as the default given it's less likely to affect downstream applications, though, i suppose the iOS case is different because we're not consuming render thread time in this case. just food for thought.

This is my thinking around keeping this:

  • Turning it off makes small text harder to read and 1 px separators vanish.
  • The uploads get 7–8% bigger with nearest-neighbour
  • The cost is about 3 ms per frame (second) on a utility-priority background queue, so almost negligible
  • Will anyone reach for it?

On Android side this also hits UI thread, which is a different story, but for iOS I would trade a couple of ms for smaller snapshot size and better quality at 1x.

If we go with the presets idea from PostHog/posthog-android#906 (review) that could be a good place to add this?

@ioannisj
ioannisj enabled auto-merge (squash) October 30, 2026 06:24
@ioannisj
ioannisj merged commit 02e113e into main Oct 30, 2026
53 checks passed
@ioannisj
ioannisj deleted the feat/replay-screenshot-scale-option branch October 30, 2026 06:47
turnipdabeets added a commit that referenced this pull request Oct 30, 2026
…cated

Keeps screenshotScale from #994 without the removed screenshotMode options and stays on SPM only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants