Skip View::update() on drag/release when mouse mode is mmUSER

[aidanmorales]

Hi!

I'm the maintainer of the rTwig package and use rgl extensively to visualize point clouds and 3d models of trees!

Recently, I noticed that both your and my custom implementation of pan3d, which you can find here, have become very laggy and slow. I don't know why this changed, but I think the current reason is rgl's mouse handlers always force a redraw after every drag, even when a custom mmUSER callback (like pan_plot()) already redraws on its own. This creates multiple redraws instead of one. The reprex below should demonstrate the problem before and after the fix.

library(rgl)

if (!requireNamespace("rTwig", quietly = TRUE)) {
  install.packages("rTwig")
}

library(rTwig)

set.seed(1)
n <- 2000000
x <- rnorm(n)
y <- rnorm(n)
z <- rnorm(n)

rgl::open3d()
rgl::points3d(x, y, z)

rTwig:::pan_plot(button = 2)

# Right mouse button pans the figure
# Unpatched rgl: noticeable stutter
# Patched rgl: should feel smooth

[dmurdoch]

Thanks for the PR. It's going to take me a few days to get time to look at this; I'll give some comments then.

[dmurdoch]

I'm reluctant to accept your PR, because someone might have a mouse handler that depends on the automatic redraw. (Though I can't think of an example, except maybe for someone who follows the advice below.)

I'm not seeing much lag, but maybe my hardware is just faster. So if lag is a problem for you, then wouldn't it be best to suppress the drawing that happens in your handler, instead of suppressing the auto draw? Here's a rewrite of your handler that does that.

pan_plot <- function (button = 2) 
{
  start <- vector(mode = "list", length = 5)
  begin <- function(x, y) {
    par <- rgl::par3d(no.readonly = TRUE)
    start[[1]] <<- par$userMatrix
    start[[2]] <<- par$viewport
    start[[3]] <<- par$scale
    start[[4]] <<- rgl::rgl.projection()
    start[[5]] <<- rTwig:::rtwig_window2user(x = x/start[[2]][3], 
                                     y = 1 - y/start[[2]][4], z = 0.5, projection = start[[4]])
  }
  update <- function(x, y) {
    save <- rgl::par3d(skipRedraw = TRUE)
    on.exit(par3d(save))
    rgl::par3d(userMatrix = rTwig:::user_matrix(x, y, 0.5, start))
  }
  rgl::rgl.setMouseCallbacks(button, begin, update)
  cat("Panning plot on rgl device:", rgl.cur())
}

[dmurdoch]

BTW, a more general patch that skipped the redraw if the scene was known to be unchanged would be more acceptable. Then the auto redraw just wouldn't happen. But that would be harder to write, because we'd need to set some sort of flag on every change.

[aidanmorales]

@dmurdoch,

Thanks for looking into this! I tried your skipRedraw change to pan_plot(), and it does make it better, and reduces the stutter. However, it still has stutter that seems to happen at random. Skipping View::update() eliminates the problem entirely. Below is a visual demonstration testing all 3 configurations on the same scene, as it is easier for me to show it live.

https://youtu.be/Q0z0oueTTwQ?si=ZvRGHtM0cdenSH8O

[dmurdoch]

This is mysterious. I added Rprintf("View::update\n") into the View::update method, and added calls to message() at the start and end of the R update() function.

Without your patch, I see View::update being called 3 times per call to the R update(): twice within the call, and once between calls.

With your patch, it is still called twice within the call, but not between calls. That makes sense, the auto update is gone.

With just my patch to the R function, it is being called 3 times, but the two inner ones don't actually ask for the window to be painted.

So it seems like the difference in observed behaviour with your patch is too big. It removes one paint call, whereas mine removes 2. Something else is going on, but I don't know what.

By the way, what platform are you using? I'm on MacOS Sequoia 15.7.7.

[aidanmorales]

@dmurdoch: I am running Windows 11 25H2 build 26200 on all of my test systems.

I do remember specifically that my old workstation, which had a i7-8700k and an Nvidia Quadro P2000 did not have this issue on Windows 10. When I was forced to upgrade it to Windows 11 around October of last year for security reasons, the stutter appeared. All of my new devices that shipped with the latest build of Windows 11 also have the stutter issue.

[dmurdoch]

I've thought of a possible explanation for the strange timing.

rgl normally uses double buffering for updates. It displays the "front" buffer, and future writes go to the "back" buffer. When those writes are done, it swaps buffers. This is done because on slow hardware writing to the buffer that is being displayed causes weird artifacts.

On modern hardware, I wouldn't be surprised if the writes can be completed entirely before the next screen update, so your code (with an even number of writes) will never actually change the buffer being displayed. Both versions of my code have an odd number of writes, so the system will actually have to switch what is being displayed after each update. Perhaps these swaps cause some resource to be thrown away, and a garbage collection is eventually necessary.

I will have to think about ways to test this.