tools: reduce source tarball compression in CI

[inoway46]

Make source-tarball gzip compression configurable, keeping level 9 as
the default. Use level 6 in build-tarball and test-shared to reduce
tarball creation time with a small increase in archive size.

Benchmark results:

• tools: reduce source tarball compression in CI #66228 (comment) (build-tarball)
• tools: reduce source tarball compression in CI #66228 (comment) (test-shared)

Refs: #64972 (comment)

[nodejs-github-bot]

Review requested:

• @nodejs/actions

[aduh95]

Testing with only 10 seems a bit low, I've started a test with more: https://github.com/aduh95/node/actions/runs/35836595280

[aduh95]

From my test, the results is that on 256 runs with ubuntu-slim, 253 succeeded, 2 timed out. Not sure it's worth doing anything about tbh.

Runner	Success rate	Avg	Min	P90	P99	Max
ubuntu-24.04-arm	256/256 (100%)	2m 23s ($0.715)	1m 55s ($0.575)	2m 30s ($0.75)	2m 44s ($0.765)	2m 57s
ubuntu-slim	253/256 (98.8%)	4m 24s ($0.528)	2m 50s	5m 46s ($0.692)	13m 02s ($1.564)	18m 13s

Worth noting that even the minimal time on the ARM runner costs more than the average slim runner usage (which includes the ones that timed out at 15 minutes).

A more concerning result is that the resulting tarball does not seem to be deterministic at all, oscillating between 119 MiB and 131 MiB 🤔

[inoway46]

@aduh95 Thanks for running the larger test! This suggests the timeout rate on ubuntu-slim is lower than I initially expected.

On cost, do you know roughly how often build-tarball runs in the private fork used for security releases? If those runs are relatively infrequent, switching runners may still be worthwhile for contributor time and developer experience despite the higher per-run cost there.

For the tarball size difference, one possibility is the runner memory difference: tools/doc/web.doc-kit.config.mjs only generates HTML section-pages when os.totalmem() is above 5 GiB (and Lightning CSS is available), so the runners may be producing different documentation files.

[inoway46]

Another option could be to keep ubuntu-slim and lower gzip compression.

https://github.com/inoway46/node/actions/runs/35888707875

gzip level	Success rate	Avg	P95	Max	Size increase vs. -9
-9	64/64	252.3s	374.4s	585s	—
-6	64/64	179.3s	253.2s	307s	+1.74%
-1	64/64	169.1s	297.1s	353s	+24.09%

-6 cut mean job time by 29% for a 1.7% size increase; -1 saved another 10s but produced much larger archives.

Would keeping -9 as the default and using -6 only in this workflow be preferable to switching runners?

[inoway46]

Follow-up testing: gzip -6 on ubuntu-slim completed 256 jobs with no timeouts. (254 succeeded; the two failures were a runner shutdown during Python setup and a DNS error during artifact upload)
Mean job duration was 173s, P95 281s, and max 518s.

[aduh95]

I find this approach much more interesting, thanks for sharing your findings. It doesn’t seem that GH makes us pay for the size of artefacts (if we reach the limit faster, I guess they’ll simply keep fewer items from older runs, which is definitely not a big deal), so not only we win by getting fewer timeouts, but we also have to pay less. So I’m definitely +1 on any size increase that ends up with an even greater percentage of time gain.
We should make the same change on test-shared workflow, for the same reason.

[inoway46]

If the larger artifact size is acceptable, -1 seems like a good choice. I’ll apply it to both this workflow and test-shared.

[aduh95]

Run a few more tests (https://github.com/aduh95/node/actions/runs/36052019083):

Compression level	Count	Avg	Min	P90	P99	Max	Median artifact size
1	32 /32	2m 35s ($0.31)	1m 55s ($0.23)	3m 25s ($0.41)	4m 29s ($0.54)	4m 40s ($0.56)	147.19 MB
2	32 /32	3m 04s ($0.37)	1m 56s ($0.23)	4m 16s ($0.51)	6m 39s ($0.80)	7m 25s ($0.89)	140.96 MB
3	32 /32	2m 39s ($0.32)	1m 54s ($0.23)	3m 22s ($0.41)	3m 52s ($0.46)	3m 55s ($0.47)	136.03 MB
4	32 /32	2m 52s ($0.34)	2m 01s ($0.24)	4m 10s ($0.50)	5m 15s ($0.63)	5m 24s ($0.65)	128.19 MB
5	32 /32	2m 50s ($0.34)	1m 58s ($0.24)	3m 28s ($0.42)	4m 20s ($0.52)	4m 32s ($0.54)	123.14 MB
6	32 /32	2m 48s ($0.34)	2m 01s ($0.24)	4m 01s ($0.48)	5m 24s ($0.65)	5m 40s ($0.68)	120.70 MB
7	31 /32	3m 04s ($0.37)	2m 08s ($0.26)	4m 00s ($0.48)	6m 31s ($0.78)	7m 17s ($0.87)	119.46 MB
8	32 /32	3m 50s ($0.46)	2m 32s ($0.30)	5m 23s ($0.65)	9m 26s ($1.13)	9m 53s ($1.19)	118.77 MB
9	253/256 (98.8%)	4m 24s ($0.53)	2m 50s	5m 46s ($0.69)	13m 02s ($1.56)	18m 13s	118.60 MB

The sweet spot seems to be between compression levels 5 and 6 (probably because a larger artifact takes longer to upload), I suggest we go with 5.

[aduh95]

Also tested for the "slim" tarballs used by test-shared.yml in https://github.com/aduh95/node/actions/runs/36060338939:

Compression level	Avg	Min	P90	P99	Max	Median Size
1	1m 55s ($0.23)	1m 09s ($0.14)	2m 37s ($0.32)	5m 09s ($0.62)	6m 05s ($0.73)	53.01 MB
2	1m 53s ($0.23)	1m 08s ($0.14)	2m 48s ($0.34)	3m 38s ($0.44)	3m 47s ($0.45)	50.83 MB
3	1m 44s ($0.21)	1m 09s ($0.14)	2m 40s ($0.32)	3m 33s ($0.43)	3m 39s ($0.44)	49.13 MB
4	1m 31s ($0.18)	1m 07s ($0.13)	1m 56s ($0.23)	2m 48s ($0.34)	2m 57s ($0.35)	45.97 MB
5	1m 41s ($0.20)	1m 13s ($0.15)	2m 06s ($0.25)	5m 06s ($0.61)	6m 07s ($0.73)	44.35 MB
6	1m 38s ($0.20)	1m 07s ($0.13)	2m 05s ($0.25)	2m 33s ($0.31)	2m 35s ($0.31)	43.62 MB
7	1m 44s ($0.21)	1m 08s ($0.14)	2m 15s ($0.27)	3m 16s ($0.39)	3m 18s ($0.40)	43.06 MB
8	1m 47s ($0.21)	1m 17s ($0.15)	2m 08s ($0.26)	3m 15s ($0.39)	3m 36s ($0.43)	42.91 MB

Here compression level 5 looks worse than both 6 and 4, so 🤷 maybe pick 6 after all (or I suppose we could pick different values for each workflow)

[inoway46]

Thanks for running these comparisons! Given the test-shared results, -6 seems like a reasonable balance for both workflows. I’ll use it in both.

[aduh95]

Can you split this in two commits? E.g. one build: add `GZIP_COMPRESSION` Make variable and one tools: reduce source tarball compression in CI

[inoway46]

Updated the remaining gzip invocations and split the changes into two commits.

[nodejs-github-bot]

CI: https://ci.nodejs.org/job/node-test-pull-request/77874/

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 90.33%. Comparing base (3d85c94) to head (40bf6b7).
⚠️ Report is 96 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #66228      +/-   ##
==========================================
+ Coverage   90.29%   90.33%   +0.04%     
==========================================
  Files         790      789       -1     
  Lines      272880   273493     +613     
  Branches    52103    52280     +177     
==========================================
+ Hits       246385   247062     +677     
+ Misses      16936    16900      -36     
+ Partials     9559     9531      -28     

see 56 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[github-actions]

Failed to resume CI

   ✖  Makefile
✖  Refusing to resume CI: failures reference files changed by this PR

Full Auto Start CI output

�[36m⠋�[39m Validating Jenkins credentials
�[36m⠋�[39m Validating Jenkins credentials
✔  Jenkins credentials valid
�[36m⠙�[39m Looking for CI runs for pull request 66228
�[36m⠙�[39m Looking for CI runs for pull request 66228
�[36m⠙�[39m Getting PR from nodejs/node/pull/66228
�[36m⠙�[39m Getting reviews from nodejs/node/pull/66228
�[36m⠙�[39m Getting comments from nodejs/node/pull/66228
✔  Found PR CI job 77874
�[36m⠹�[39m Querying data for job/node-test-pull-request/77874/
�[36m⠹�[39m Querying data for job/node-test-pull-request/77874/
�[36m⠹�[39m Querying API for job/node-test-pull-request/77874/
✔  Build data downloaded
�[36m⠹�[39m Checking failures against changed PR files
�[36m⠹�[39m Checking failures against changed PR files
�[36m⠸�[39m Querying data for job/node-test-pull-request/77874/
�[36m⠸�[39m Querying API for job/node-test-pull-request/77874/
✔  Build data downloaded
�[36m⠸�[39m Querying failures of job/node-test-commit/92671/
�[36m⠸�[39m Querying failures of job/node-test-commit/92671/
�[36m⠸�[39m Querying API for job/node-test-commit-linux/73461/
�[36m⠼�[39m Querying API for job/node-test-commit-linux/nodes=fedora-latest-x64/73461/
�[36m⠼�[39m Querying console text for job/node-test-commit-linux/nodes=fedora-latest-x64/73461/
✔  Data downloaded
   ✖  Makefile
✖  Refusing to resume CI: failures reference files changed by this PR

View workflow run

[nodejs-github-bot]

CI: https://ci.nodejs.org/job/node-test-pull-request/77899/

[nodejs-github-bot]

CI: https://ci.nodejs.org/job/node-test-pull-request/77918/

[nodejs-github-bot]

CI: https://ci.nodejs.org/job/node-test-pull-request/77922/