Skip to content

Ascending timestamps - #558

Open
eoghanmurray wants to merge 7 commits into
rrweb-io:mainfrom
statcounter:ascending-timestamps
Open

eoghanmurray wants to merge 7 commits into
rrweb-io:mainfrom
statcounter:ascending-timestamps

Conversation

@eoghanmurray

@eoghanmurray eoghanmurray commented May 14, 2021 •

Copy link
Copy Markdown
Collaborator

This started with the idea that we don't want 'overlapping' events, i.e. a mouse click event followed by a mousemove event whose negative timestamps put the start of the mouse movement before the click.

This is important in the logical processing of a series of events so that we don't have to consider the next event when processing the current event e.g. in building up a graphical timeline.

It could also be important in streaming 'live' events to a player, as although a mousemove can still introduce up to 500ms lag, it now can't result in an out-of-order situation where that click is played back before a mousemove that should precede it.

2026 Edit: tests added in this PR in packages/rrweb/test/record.test.ts There seem to be no tests that recreate the mouse movement due to the mentioned difficulty with 'random' mouse movements in the headless browser.

I've added a test (via util.py) to check that timestamps are weakly ascending before we strip them out, but that didn't catch anything prior to these changes.

@eoghanmurray

Copy link
Copy Markdown
Collaborator Author

The downside of this pull request is that now long mouse movements can potentially be split up into many shorter events.

A danger would be on a page which has many ongoing background mutations ... let me know if that is a concern.

@eoghanmurray
eoghanmurray force-pushed the ascending-timestamps branch 3 times, most recently from d01b44f to 17c9efd Compare July 8, 2021 15:13
@eoghanmurray
eoghanmurray force-pushed the ascending-timestamps branch 2 times, most recently from 146cb14 to 0f1780b Compare August 5, 2021 09:45
Comment thread packages/rrweb/src/record/index.ts Outdated
Comment thread packages/rrweb/src/record/index.ts Outdated
const wrappedMutationEmit = (m: mutationCallbackParam) => {
const wrappedMutationEmit = (m: mutationCallbackParam, timestamp?: number) => {
if (typeof timestamp === 'undefined') {
timestamp = Date.now()

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why mutation emit need to create timestamp by itself while other callback still using the wrappedEmit method?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be written without a timestamp = Date.now(); in this function; I will rewrite it now and force push this and prior change.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've made that change now, but maybe you meant something else?

@eoghanmurray

eoghanmurray commented Aug 10, 2021 •

Copy link
Copy Markdown
Collaborator Author

2021 Eoghan: ~ I definitely need to create some proper tests for this PR as it involves intricate interactions between ongoing mouse movement and interrupting mutations.~

Done in updated PR

2021 Eoghan: ~Actually I haven't fully thought about whether splitting a single mouse movement event in two will have any differences in replay rendering? ~

In replay, every position is queued independently regardless of whether positions [P1,P2,P3] arrive as one event or three, each P lands in the timer at the same instant with the same (x,y)"

2021 Eoghan: Even with this PR, I still think it's a good idea to emit an event at the start of mousemovent for the use of live replayers and for cleaner processing (even though that starting timestamp info is also contained in the eventual throttled event).

This is actually how it always(?) worked; the first mouse movement is emitted immediately with one position

Comment thread packages/rrweb/src/record/index.ts Outdated
@eoghanmurray

Copy link
Copy Markdown
Collaborator Author

Can retry (rebase on master) after #1475 is merged as I incorporated some of this PR into that PR

eoghanmurray added a commit to eoghanmurray/rrweb that referenced this pull request Apr 14, 2025
…sumed had been merged - it allows the timestamps to be set outside of the emission event
eoghanmurray added a commit to eoghanmurray/rrweb that referenced this pull request Aug 26, 2025
…crementalSnapshot (mutation) that generated them.

The logic is that this is the time they were created, even if capture was delayed at record time.
If sorting by timestamp is performed before replay, then so long as the sort algorithm is stable, this will bring the asset events closer to the snapshot that they are associated with, where they are needed for replay

Note: This is a rework of a subset of the code from unmerged rrweb-io#558 allowing timestamps to be set outside of the emission event
eoghanmurray added a commit to rrweb-cloud/rrweb that referenced this pull request Nov 7, 2025
…crementalSnapshot (mutation) that generated them.

The logic is that this is the time they were created, even if capture was delayed at record time.
If sorting by timestamp is performed before replay, then so long as the sort algorithm is stable, this will bring the asset events closer to the snapshot that they are associated with, where they are needed for replay

Note: This is a rework of a subset of the code from unmerged rrweb-io#558 allowing timestamps to be set outside of the emission event
eoghanmurray added a commit to eoghanmurray/rrweb that referenced this pull request Jun 18, 2026
…crementalSnapshot (mutation) that generated them.

The logic is that this is the time they were created, even if capture was delayed at record time.
If sorting by timestamp is performed before replay, then so long as the sort algorithm is stable, this will bring the asset events closer to the snapshot that they are associated with, where they are needed for replay

Note: This is a rework of a subset of the code from unmerged rrweb-io#558 allowing timestamps to be set outside of the emission event
@changeset-bot

changeset-bot Bot commented Sep 21, 2026 •

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 614c50f

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 22 packages
Name Type
rrweb Minor
@rrweb/record Minor
@rrweb/all Minor
@rrweb/browser-client Minor
@rrweb/packer Minor
@rrweb/replay Minor
rrdom-nodejs Minor
rrdom Minor
rrvideo Minor
rrweb-player Minor
rrweb-snapshot Minor
@rrweb/types Minor
@rrweb/utils Minor
@rrweb/web-extension Minor
@rrweb/rrweb-plugin-canvas-webrtc-record Minor
@rrweb/rrweb-plugin-canvas-webrtc-replay Minor
@rrweb/rrweb-plugin-console-record Minor
@rrweb/rrweb-plugin-console-replay Minor
@rrweb/rrweb-plugin-network-record Minor
@rrweb/rrweb-plugin-network-replay Minor
@rrweb/rrweb-plugin-sequential-id-record Minor
@rrweb/rrweb-plugin-sequential-id-replay Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@pkg-pr-new

pkg-pr-new Bot commented Sep 21, 2026 •

Copy link
Copy Markdown
@rrweb/all

npm i https://pkg.pr.new/rrweb-io/rrweb/@rrweb/all@614c50f

@rrweb/browser-client

npm i https://pkg.pr.new/rrweb-io/rrweb/@rrweb/browser-client@614c50f

@rrweb/packer

npm i https://pkg.pr.new/rrweb-io/rrweb/@rrweb/packer@614c50f

@rrweb/record

npm i https://pkg.pr.new/rrweb-io/rrweb/@rrweb/record@614c50f

@rrweb/replay

npm i https://pkg.pr.new/rrweb-io/rrweb/@rrweb/replay@614c50f

rrdom

npm i https://pkg.pr.new/rrweb-io/rrweb/rrdom@614c50f

rrdom-nodejs

npm i https://pkg.pr.new/rrweb-io/rrweb/rrdom-nodejs@614c50f

rrvideo

npm i https://pkg.pr.new/rrweb-io/rrweb/rrvideo@614c50f

rrweb

npm i https://pkg.pr.new/rrweb-io/rrweb@614c50f

rrweb-player

npm i https://pkg.pr.new/rrweb-io/rrweb/rrweb-player@614c50f

rrweb-snapshot

npm i https://pkg.pr.new/rrweb-io/rrweb/rrweb-snapshot@614c50f

@rrweb/types

npm i https://pkg.pr.new/rrweb-io/rrweb/@rrweb/types@614c50f

@rrweb/utils

npm i https://pkg.pr.new/rrweb-io/rrweb/@rrweb/utils@614c50f

@rrweb/rrweb-plugin-canvas-webrtc-record

npm i https://pkg.pr.new/rrweb-io/rrweb/@rrweb/rrweb-plugin-canvas-webrtc-record@614c50f

@rrweb/rrweb-plugin-canvas-webrtc-replay

npm i https://pkg.pr.new/rrweb-io/rrweb/@rrweb/rrweb-plugin-canvas-webrtc-replay@614c50f

@rrweb/rrweb-plugin-console-record

npm i https://pkg.pr.new/rrweb-io/rrweb/@rrweb/rrweb-plugin-console-record@614c50f

@rrweb/rrweb-plugin-console-replay

npm i https://pkg.pr.new/rrweb-io/rrweb/@rrweb/rrweb-plugin-console-replay@614c50f

@rrweb/rrweb-plugin-network-record

npm i https://pkg.pr.new/rrweb-io/rrweb/@rrweb/rrweb-plugin-network-record@614c50f

@rrweb/rrweb-plugin-network-replay

npm i https://pkg.pr.new/rrweb-io/rrweb/@rrweb/rrweb-plugin-network-replay@614c50f

@rrweb/rrweb-plugin-sequential-id-record

npm i https://pkg.pr.new/rrweb-io/rrweb/@rrweb/rrweb-plugin-sequential-id-record@614c50f

@rrweb/rrweb-plugin-sequential-id-replay

npm i https://pkg.pr.new/rrweb-io/rrweb/@rrweb/rrweb-plugin-sequential-id-replay@614c50f

commit: 614c50f

@eoghanmurray

eoghanmurray commented Sep 24, 2026 •

Copy link
Copy Markdown
Collaborator Author

The downside of this pull request is that now long mouse movements can potentially be split up into many shorter events. A danger would be on a page which has many ongoing background mutations

Answering my own 2021 comment above:

I did some quantitative analysis on this on a corpus of rrweb events with this PR deployed, and got the AI to reconstruct how many batched events had been split up that would not have been before (based on timestamp adjacency).

Bottom line: this PR generated an estimated 54% more mouse move events (as anticipated) by sometimes preventing the full 500ms batching that has been there since initMousemoveObserver was introduced. We get the same positions data output, so no effect on replay, just that these events are more often emitted separately with smaller numbers of positions. The per-event overhead is 70 bytes and looking at this extra bytes vs. other event types, the effect is negligible.

eoghanmurray and others added 6 commits September 24, 2026 19:02
…he mouse move event immediately so that there is negative timeOffsets never result in an out of sequence event stream
…timestamp sequence is (weakly) ascending.

 - Overruling the timestamp of `wrapEvent` here also ensures that the `Date.now()` used to calculate the negative `totalOffset` matches the event timestamp, so is presumably slightly more accurate
…akly ascending - mutationBuffer emission can be triggered by another event, so a new `Date.time()` could otherwise be larger than the timestamp of the event that triggered it
…tions unfreeze on a clean boundary

Instead of back-dating the unfrozen mutation's timestamp to the start of an
in-progress mouse move, make freezePage() interrupt the move like any other
event: flush the pending (throttled) move immediately and reset the move
emission throttle so the next movement starts on a fresh leading edge (a
single position at timeOffset 0). Any later unfreeze is then triggered either
by that leading-edge move or by a non-move event, neither of which needs the
timestamp adjustment.

Adds a cancel() to throttle() (clears the pending trailing timeout and resets
its window) and threads a reset flag through ongoingMove/moveEmission.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ascending

Two dedicated tests exercise the guarantee that a throttled mouse move,
when flushed by an interrupting event, is emitted before that event with
its (negative) position timeOffsets, so the effective timestamp sequence
stays weakly ascending:
 - a move interrupted by a keystroke
 - buffered (frozen) mutations plus a move interrupted by a click

These replace the assertion previously embedded in the shared
stringifySnapshots helper, which no longer exercised the mouse-move path
(that helper filters MouseMove events out) and imposed the check on every
snapshot test. Each test fails without the interrupting-flush fix.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
eoghanmurray added a commit that referenced this pull request Oct 21, 2026
… a delayed mutation emission is short circuit triggered by another event (another type of forcing)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants