Skip to content

perf(replay): encode screenshots and batches off the main thread - #148

Merged
marandaneto merged 3 commits into
PostHog:mainfrom
rbzn-play-perfect:perf/replay-off-main-thread
Oct 22, 2026
Merged

marandaneto merged 3 commits into
PostHog:mainfrom
rbzn-play-perfect:perf/replay-off-main-thread

Conversation

@rbzn-play-perfect

Copy link
Copy Markdown
Contributor

💡 Motivation and Context

With session replay on, every 1 s capture does about 20 ms of main-thread work: Texture2D upload, EncodeToJPG and base64. Each upload batch is serialized and gzipped on the main thread too, which caused 118–157 ms stalls on an iPhone 16e. That shows up as visible hitches in games.

This PR moves that work to worker threads:

  • Screenshots: the readback is copied into a reused byte[] and encoded with ImageConversion.EncodeArrayToJPG (thread-safe) on Task.Run. The data URL is built in one string.Create call. A new ScreenshotCapture.Poll(), called by the capture loop every frame, hands the result back on the main thread, so Enqueue still runs there. CancelPending() drops a readback or encode still in flight when replay stops or the session rotates, so a frame from the previous session is never enqueued under the new session id.
  • Batches: JSON serialization, UTF-8 and gzip run on Task.Run while the SendBatch coroutine yields. A batch that fails to serialize is dropped (reported as 400) instead of retried forever, the same way analytics events that fail serialization are dropped.

This builds on #126: the vertical flip already happens in the blit, so the readback needs no CPU pass. Replay stays disabled on WebGL (PostHogSDK), so no threadless platform reaches the new code. No public API changes; bin/check-public-api is clean.

💚 How did you test it?

Measured on an iPhone 16e with a 6,000-cube scene (one draw call per cube) in a Unity 6000.0.84 development build, 30 s runs with the default replay settings (1 s, quality 80, scale 0.75):

Metric v1.1.1 This PR
Average fps 57.8 60.0 (matches the no-SDK baseline)
Worst frame 190–217 ms 17 ms
Frames over 20 ms 2.6% 0%
Managed allocations 31 MB/s 5.4 MB/s
Main-thread cost per capture ~20 ms ~0.9 ms

The "This PR" column comes from this branch (rebased on main), built and run on the device. The per-capture main-thread cost was measured with temporary step timing on the same change applied to v1.1.1; that timing code is not in this PR.

  • An editor comparison showed the uploaded screenshots are identical to v1.1.1's.
  • Recorded replays from this branch play correctly on PostHog.
  • Unit tests: bin/test passes (517). New tests cover PrepareBody (gzip round-trip equals the JSON) and ToJpegDataUrl (matches Convert.ToBase64String for lengths 0–5 and 100,001 bytes). bin/fmt --check is clean.

📝 Checklist

  • I reviewed the submitted code.
  • I added tests to verify the changes.
  • I updated the docs if needed.
  • No breaking change or entry added to the changelog.

If releasing new changes

  • Ran pnpm changeset to generate a changeset file

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

Move the screenshot JPEG and base64 encoding, and the replay batch JSON
serialization and gzip, from the main thread to worker tasks. A new
ScreenshotCapture.Poll() hands the encoded screenshot back on the main
thread, and CancelPending() drops an in-flight capture when replay stops
or the session rotates.
@rbzn-play-perfect
rbzn-play-perfect requested a review from a team as a code owner October 15, 2026 11:25

@marandaneto marandaneto left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated advisory code review.

Comment thread com.posthog.unity/Runtime/SessionReplay/SessionReplayIntegration.cs
…oding

The capture loop delivers a finished encode before it checks for a session
rotation, so after Reset() the previous user's frame could be enqueued under
the new session and distinct id. Each capture now records its session and the
result is dropped if the session is no longer current.
@marandaneto
marandaneto requested a review from a team October 21, 2026 12:53
@marandaneto

Copy link
Copy Markdown
Member

@ioannisj @turnipdabeets probably something for you to validate very mobile specific

@turnipdabeets

Copy link
Copy Markdown
Contributor

Thanks @rbzn-play-perfect, great change, and the numbers hold up on an iPhone 14 Pro Max (Unity 6000.0.85f1, IL2CPP, 6,000-cube scene):

without PostHog main this PR
avg fps 60.0 58.5–59.5 60.0
worst frame 16.8 ms 80–128 ms 16.8–18.2 ms
frames > 20 ms 0% 1.9–2.7% 0%

"without PostHog" is the same app with the SDK never initialized, so this PR is as smooth as not running the SDK at all. Screenshots are pixel-identical to main, and 37fae99 looks good for the paused-reset case.

One request: the replay batch sent when the app goes to the background now waits until the app comes back. On main, SendBatch called SendWebRequest() before its first yield, so iOS sent it before suspending the app (it arrived 0.6 s after backgrounding). Here the coroutine yields while PrepareBody runs, Unity's loop stops, and the batch only arrived when I foregrounded the app 20 s later. The queue is in memory only, so if the user kills the app from the app switcher, those screenshots are lost.

Could Pause() prepare the batch inline, e.g. Flush(prepareInline: true) → SendBatch calls PrepareBody directly? Timed and threshold-triggered flushes can stay on the worker. Edge case: a flush already waiting on the worker when the app backgrounds still causes the new one to be skipped (_isFlushing). Finishing it synchronously or accepting that small window are both fine; just leave a comment saying which.

Happy to re-run the device test once it's in!

…uspended

Pause() flushes when the app goes to the background, but the batch was
prepared on a worker while the coroutine yielded. iOS stops Unity's loop
right after, so the request only went out when the app came back. The pause
flush now prepares its first batch on the main thread, so the request starts
before the first yield, as it did on main. Timed and threshold flushes stay
on the worker.
@rbzn-play-perfect

Copy link
Copy Markdown
Contributor Author

@turnipdabeets thanks for the device run! Done in 36c804e. Pause() now calls Flush(prepareInline: true), and SendBatch prepares the body directly in that case, so the request starts before the first yield, as on main. Timed and threshold flushes stay on the worker. Only the first batch of that flush is prepared inline, since on main the next batch also waited for the first response.

For the _isFlushing edge case I went with accepting the window, and the comment above the check says so: if a flush is still preparing on the worker when the app backgrounds, its batch waits until the app resumes. A flush waiting on its response skips the background one the same way main did.

One note: Pause() also runs when the app only loses focus (Notification Center, system sheets), so that now costs one main-thread prepare. Main paid that on every flush.

Would be great if you could re-run the background test.

@turnipdabeets

Copy link
Copy Markdown
Contributor

@rbzn-play-perfect thanks! this looks good to me. Do you mind signing your commits so that we can merge this? They need to be verified to merge.

@rbzn-play-perfect
rbzn-play-perfect force-pushed the perf/replay-off-main-thread branch from 36c804e to ed75602 Compare October 22, 2026 05:37
@marandaneto
marandaneto merged commit 9c335e7 into PostHog:main Oct 22, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants