Visual metrics pick up too early First Visual change sometimes on Samsung A51

[soulgalore]

Looking at the filmstrip:

And the actual video.

en.m.wikipedia.org_wiki_Barack_Obama-4-chrome-2024-12-18142810-video.mp4

My guess is that it picks up the change in the URL bar? I'm using Chrome 131 in the tests.

[soulgalore]

Let me collect the original video and try to run it locally.

[soulgalore]

1-original.mp4

[soulgalore]

Hi @gmierz I need your help here if you have time. I have an issue when running test on an Samsung A51. The problem is that depending on what thumb size I set, the first visual change differs.

I run it like this:
./visualmetrics-portable.py -i 1-original.mp4 --orange --force --renderignore 5 --viewport --viewportretries 60 --viewportminheight 100 --viewportminwidth 100 -q 75 --thumbsize 400 gives me a first visual change of 735 that is correct.

If I increase the thumb size to for example 600 (to have better quality), I get a first visual change that is 216.
./visualmetrics-portable.py -i 1-original.mp4 --orange --force --renderignore 5 --viewport --viewportretries 60 --viewportminheight 100 --viewportminwidth 100 -q 75 --thumbsize 200

I wonder the best way we should handle it. My guess is that when we set the thumbsize higher, the URL bar come into play and affect the metric? Do you have any input?

[gmierz]

I hit a similar issue in a regression bug here: https://bugzilla.mozilla.org/show_bug.cgi?id=1929579#c5

It doesn't seem to be exactly the same thing as what is happening here though. To figure out where this issue is coming from, you could change the crop fields for the orange frame here: https://github.com/sitespeedio/browsertime/blob/main/visualmetrics/visualmetrics-portable.py#L440

You could add +50 to the viewport["y"] value, to see if the load bar is interfering here. Also, to see if there are some issues related to the scroll bar, you could subtract 50/100 from the width.

I'm seeing a small grey bar at the bottom of the screen though which might be triggering this. You can check that by subtracting 100 from the viewport height to check that. Not sure where the grey bar is coming from though.

[soulgalore]

Thank you @gmierz you got eagle eyes, I missed the grey in the bottom. If I removed 4 pixel from the height it works fine.

Do you think it's ok if I change https://github.com/sitespeedio/browsertime/blob/main/visualmetrics/visualmetrics-portable.py#L625 that to remove 14 lines instead of 10?

[gmierz]

That sounds good to me - do you see this grey bar on other test pages as well? I don't see the grey bar in the pageload videos that we have for chrome 131. Here's an archive for one of those tests: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/Dwd3NMsZStyfYGrhvjyPAg/runs/0/artifacts/public/test_info/browsertime-videos-original.tgz

Here's a link which includes a few other tests: https://treeherder.mozilla.org/jobs?repo=mozilla-central&selectedTaskRun=Dwd3NMsZStyfYGrhvjyPAg.0&tier=1%2C2%2C3&searchStr=browsertime%2Candroid%2Ctp6%2Cchrome&revision=d01b815e2e1a7baed43f9b0a1e04e109109648bf

Is it possible the dark mode vs light mode is causing the difference here?

[soulgalore]

I just setting up and I don't other things to compare with but I think the grey bar is Chromes loading bar that at least on desktop was common. I'll check what it is before I merge the change.

dark mode vs light mode

Hm, I'll check.

[soulgalore]

It's a grey line under the orange background and I think you are right that dark mode creates it. I'll push the change.