Mysql shell build matrix - #4358
Open
EvgeniyPatlan wants to merge 7 commits into
Open
Conversation
Builds mysql-shell packages on every supported OS and architecture in parallel
and reports a pass/fail matrix. Intended for validating changes to the
packaging builder before they are merged, so it neither signs nor publishes
anything.
Differs from hetzner-mysql-shell-8.0-RELEASE in four ways:
- the builder is used from a git checkout rather than a single downloaded
file, because it now needs the patches/ directory alongside it
- each OS runs the whole pipeline standalone (source tarball, source package,
binary package) instead of stashing artifacts between stages, so a failure
is attributable to one OS
- a failing cell marks the build UNSTABLE and the rest of the matrix still
runs, with a summary listing every cell and its duration
- each built package is installed into a clean container of the same
distribution and smoke tested: both language modes, no unresolved shared
libraries, and the bundled libexec binaries execute
Verification uses a second container that has only the package mounted, not
the build container. A package installed where the build tree still exists
resolves its build-time RPATHs and passes every check even when it cannot
start on a user's machine.
Matrix is EL8, EL9, EL10, Amazon Linux 2023, Debian bookworm and trixie,
Ubuntu jammy, noble and resolute, on x86_64 and aarch64 by default. ARCHES can
narrow it to one architecture while iterating.
APPLY_PATCHES selects whether the Percona patch series is applied, so a failing
cell can be re-run against vanilla upstream to establish whether the patches or
upstream are at fault. The package is renamed to percona-mysql-shell either
way. REFRESH_PATCHES selects whether the series is regenerated from the fork
first. Both are recorded in the summary so a result is self-describing.
The job cannot load its pipeline until the groovy exists on the branch the
job's SCM points at, so builds fail with:
ERROR: Unable to find ps/jenkins/mysql-shell-build-matrix.groovy
from git https://github.com/Percona-Lab/jenkins-pipelines.git
Point it at mysql-shell-build-matrix so the pipeline can be iterated on
without pushing every fix to hetzner. Switch back to hetzner before merging.
Do not depend on the executable bit surviving the clone; the release job invokes the script the same way.
The install line rendered to "if ! apt-get update && apt-get install", which the shell parses as "(! apt-get update) && apt-get install". The update succeeded, so the negation was false, the && short circuited and the install never ran, while the failure branch was skipped too. Every later check then ran against a container with no mysqlsh on it. Group the command so the negation and the redirection cover all of it. Only deb cells were affected; the rpm install is a single command. No cell passed falsely, they failed for the wrong reason. Also skip debug symbol packages, which nobody installs and which added 225MB to the deb check, and report a missing binary directly instead of letting an argument-less ldd print "0 unresolved" in the middle of a failing run.
The builder is no longer a single file. It needs the patches directory beside it, so clone the packaging repository instead of downloading the script, and run it by absolute path with the working directory left at the workspace root. That keeps SCRIPT_DIR pointing at the patches and CURDIR pointing at source_tarball, srpm, rpm, deb and source_deb, so every artifact still lands where the upload steps look for it. The properties file is written next to the script's working directory rather than inside the build directory, so read it from there. Drop Ubuntu Focal and Debian Bullseye. The builder supports el8, el9, el10, amzn2023, bookworm, trixie, jammy, noble and resolute, and dies on anything else. Focal only ran for 8.0, which is out of scope for the refactor; bullseye also ran for 8.4, so this removes Debian 11 from what 8.4 publishes. Pass --refresh_patches=0 so a release uses the series checked into the packaging repository instead of refetching it from the fork at build time, and move the stale 8.0.33 defaults to 9.7.1 and release-9.7.1-1. Drop PS_MAJOR_RELEASE, which nothing selects on any more.
EvgeniyPatlan
requested review from
a team,
eleo007,
kaushikpuneet07,
mchawla16,
mohitj1988 and
panchal-yash
as code owners
August 19, 2026 18:18
Contributor
|
Hi @EvgeniyPatlan Could you check why the job was never succesful? |
The post block died on "StringBuilder leftShift", which the Jenkins script security sandbox does not permit, so a run that had finished every cell reported no summary at all. Build the report by string concatenation instead, and drop String.format, Map.sort and take, none of which were reached and any of which could have been rejected in turn. Two defects found while rewriting it. Groovy has no Integer.plus(String), so the minutes column would have thrown once the sandbox let it run. Cell keys were GStrings, which do not hash like Strings, so a lookup could miss silently; store and look them up as Strings. Report the cells in matrix order rather than sorting the map, which is deterministic and avoids another restricted call.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.