I'm questioning the speed calculation for a couple generators

[scottchiefbaker]

I use the speed battery to compare PRNGs against each other. Checking the speeds for a handful I get the following:

Generator	Speed GiB/s
pcg32	4.32227
pcg64_64	8.60865
wyrand	8.60972
xorshift++	3.15084
xoshiro256+	2.14274
xoshiro256++	2.14189
xoshiro256**	1.94117
mwc128	3.15034
splitmix	7.10461

SmokeRand is saying that pcg64_64 is more than three times as fast as xoshiro256++? These numbers seem pretty different than what is listed on the xorshiro site. Perhaps I'm reading the output wrong? I'm running the speed battery and and reading the "Corrected result (GiB/sec)" part of the first section.

[alvoskov]

SmokeRand is saying that pcg64_64 is more than three times as fast as xoshiro256++?

1. I don't see measurements for pcg64_64 at this site.
2. For direct comparison with their numbers we should dig into their measurement method, especially if we work with something faster than 1 cpb. It seems that it is closer to my "sum before correction".
3. pcg64_64 is the 64-bit generators, xoshiro256++ is the 32-bit one. Usage of 64-bit arithmetics on modern CPUs makes generators faster. Wyrand, SplitMix and MWC128 with multiplications are also extremely fast.

About the comparison at their page: their numbers are likely reasonable. But I see an unusual thing: they don't compare performance of "bithacks" with state-of-art generators like AES, ChaCha, Speck and ThreeFish.

[scottchiefbaker]

1. That site doesn't measure pcg64_64. Let's use splitmix64 as a comparison point instead.
2. According to the xoshiro site xoshiro256++ and splitmix64 are within 15% of each other (0.29 vs 0.34). According to SmokeRand splitmix64 is 331% faster. That's a MASSIVE discrepancy
3. xoshiro256++ is 100% 64bit, I'm not sure where you get 32bit

I just want to "sanity check" SmokeRand. When SmokeRand says something is 331% faster, and another trusted site on the internet says 15% that makes me question things.

[alvoskov]

Yes, I agree xoshiro256++ is the 64-bit one, probably mismatched with xoshiro128++.

I agree that discrepancy is strange, but comparison of two benchmarks should be made on the same computer using the same compiler with the same optimization settings. Especially for such extremely fast generators. But there are good news: Vigna has so called "harness" at the "PRNG shootout" website, so you can try to reproduce the results. I will try too.

[tansy]

How do these hashes compare in smhasher?

[alvoskov]

@tansy most of them are not hashes, only wyrand and splitmix resemble simplified hash functions. It seems that I've found a possible reason of problems with xoshiro speed, it seems that GCC 15 became "too clever" in some optimization techniques if -O3 key is used.

[alvoskov]

@scottchiefbaker

I just want to "sanity check" SmokeRand. When SmokeRand says something is 331% faster, and another trusted site on the internet says 15% that makes me question things.

It seems that I've found a reason: newer versions of GCC (e.g. GCC 15.2) at -O3 optimization settings caused performance degradation in some cases during too excessive code vectorization. I've detected it during replacement of TDM-GCC 10.3 to w64devkit. The solution was addition of the -fno-tree-slp-vectorize key to Makefile.gnu. The fix is included into the v0.46 prerelease.

[scottchiefbaker]

I'm assuming you're talking about the 58a58cd commit in the main branch? I pulled down the latest from main and recompliled and I'm still seeing the same results as above.

[alvoskov]

Such recompilation will require make -f Makefile.gnu clean because changing that place of makefike won't automatically invalidate object files. Did you make such reset? I think if the generators were really recompiled - then probably analysis with disassembler will be required.

[scottchiefbaker]

That fix definitely sped some generators up. This is closer to what I would expect.

Generator	Speed
pcg32	4.29 GiB/s
pcg64_64	8.62 GiB/s
wyrand	8.63 GiB/s
xorshift128++	8.58 GiB/s
xoshiro256+	8.61 GiB/s
xoshiro256++	8.59 GiB/s
xoshiro256**	8.52 GiB/s
mwc128	2.67 GiB/s
splitmix	6.87 GiB/s

I do wonder if there is some arbitrary limitation around 8.61GiB/s as several of the generators top out around the same value.

[alvoskov]

It depends on the exact metric that you use. In some cases the function call itself or for loop for the "sum" mode will be a limiting factor. If the generator is really fast then the function call overhead may be a limiting factor.