Adds tarballs to PPG 17 - #2
Open
rjd15372 wants to merge 79 commits into
Open
Conversation
rjd15372
added
RockyLinux_9
no-dep-cascade
|
|
OBS Build Check — ❌ Sync failedThe OBS sync step failed. See the workflow run for details.
|
rjd15372
force-pushed
the
tarballs-simpleimage
branch
from
July 28, 2026 12:11
587f5ba to
e08a04f
Compare
rjd15372
added
the
keep-pr-build
rjd15372
force-pushed
the
tarballs-simpleimage
branch
2 times, most recently
from
August 26, 2026 07:42
695fdf9 to
e14de92
Compare
Signed-off-by: Ricardo Dias <ricardo.dias@percona.com>
…ball.sh Patch postgres.real (the wrapper shell script at bin/postgres cannot be patchelf'd, and postmaster is gone since PG 16) and drop the error swallowing so a failed patchelf aborts the build. RPATH the bundled libperl.so to $ORIGIN so it finds libcrypt in CORE/ (RUNPATH is not transitive). Replace the ldd-based library audit with a DT_NEEDED soname audit: inside the fully-populated buildroot ldd resolves everything from /usr/lib64 via ld.so.cache, so it could never catch a library that copy_deps failed to bundle. Each NEEDED soname must now be either in the system exclude list or present under /opt. Signed-off-by: Ricardo Dias <ricardo.dias@percona.com>
…iner Ran build-tarball.sh to completion inside a rockylinux:9 container provisioned with the staging:17 RockyLinux_9 RPMs (mirroring the OBS simpleimage buildroot: EPEL + CRB + staging repo) and structure-diffed the result against the official percona-postgresql-17.10-ssl3-linux-x86_64.tar.gz. Fixes found: - simpleimage: drop python3.12-idle BuildRequires — the package does not exist on EL9 (RHEL strips IDLE from python3.12), so the OBS build would be unresolvable. The official tarball ships no _tkinter either. - simpleimage: add percona-pgpool-II-pgNN-devel and -extensions — the official tarball ships the pgpool2 headers (include/pgpool2/) and the pgpool_adm/pgpool-recovery PostgreSQL extensions. - bundle_deps: accept extra directory trees walked recursively so python lib-dynload/ + site-packages C extensions (libsqlite3, libncursesw, libuuid, libgdbm, libpq via psycopg2) and perl XS module deps (libdb for DB_File, libsombok for Unicode::LineBreak) are bundled; perl XS .so get RPATH /opt/percona-perl/lib. These were the NEEDED-audit true positives; nothing was added to SYSTEM_LIBS_EXCLUDE. - site-packages copy list: add jmespath, s3transfer, psycopg2 (all shipped by the official tarball; psycopg2 is required by patroni). - license/doc staging: RPM %license files live under /usr/share/licenses, pgbouncer docs under unprefixed /usr/share/doc/pgbouncer; pgpool headers are flat in /usr/include and get nested under include/pgpool2/ per the reference layout; bundle the python man page under share/man. - section 15: precompute the bundled-soname list once instead of a full 'find /opt' per NEEDED entry (audit re-verified to fail on a synthetic missing soname and pass once bundled). Verification: NEEDED audit 0 unresolved; smoke commands pass (initdb, postgres.real, python+ssl+yaml, patroni import, perl, tcl); tarball top-level is exactly the percona-* component dirs. Accepted structure divergences vs the official tarball: - percona-haproxy/ absent (out of scope for this build, per spec). - percona-pgbadger/auto/ absent (perl .packlist install artifact of the official source build; RPM installs do not create it). - percona-tcl/share/ absent (official bundles man pages for tcl extensions itcl/tdbc/thread/sqlite built from source; EL9 does not package those extensions — our tcl man pages live under percona-tcl/man, which the official tarball also has). - percona-python3/lib64 extra (symlink to lib/pythonX.Y so the RPM python, compiled with platlibdir=lib64, resolves PYTHONHOME). - percona-postgresql17/lib/pgpool-regclass.so absent (obsolete extension for PG < 9.4; pgpool 4.7 RPMs no longer build it). - site-packages: psycopg (v3) absent (official pip-installs it; patroni works via the packaged psycopg2). Known blocker for the OBS build, out of scope here: the published python3.12-boto3 requires python3.12dist(jmespath) and python3.12dist(s3transfer), which no project in the RockyLinux_9 repo path provides — percona-patroni-aws is currently uninstallable in the buildroot. jmespath/s3transfer packages need to be added to ppg:common:deps (validated locally with stand-in RPMs built from the PyPI sdists). Signed-off-by: Ricardo Dias <ricardo.dias@percona.com>
Review fixes on top of the container validation, re-validated in the
Rocky 9 harness (audit clean, all smoke commands pass, structure diff
unchanged):
- RPATH python C extensions (lib-dynload/ + site-packages) at
/opt/percona-python3/lib: the python3 wrapper sets LD_LIBRARY_PATH,
but embedded interpreters (plpython3 inside postgres.real) don't run
through the wrapper, and the executable's RUNPATH does not apply to
dlopened extensions' own dependencies.
- Section 15: build the bundled-soname list with -xtype f so dangling
symlinks never count as bundled. This immediately caught a real bug:
copy_deps only copied files matching the ${base}.so* glob, so when
the SONAME symlink resolves to a differently-named real file
(libldap.so.2 -> libldap_r.so.2.0.200, libopenblaso.so.0 ->
libopenblaso-r0.3.29.so on EL9) only a dangling symlink was bundled
and the library silently resolved from the buildroot. copy_deps now
copies the resolved real file explicitly.
- Perl XS RPATH loop: drop the 2>/dev/null || true swallow — every file
fed in is a real XS .so, so a patchelf failure is meaningful and must
fail the build. The new python loop keeps a file(1) ELF guard (the
tree contains non-ELF *.so* test fixtures) but also fails loudly on
patchelf errors.
Gate spot-check re-run: audit fails on a missing soname and on a
dangling symlink, passes once the real file (or a valid symlink to it)
is bundled.
Signed-off-by: Ricardo Dias <ricardo.dias@percona.com>
Signed-off-by: Ricardo Dias <ricardo.dias@percona.com>
Signed-off-by: Ricardo Dias <ricardo.dias@percona.com>
Signed-off-by: Ricardo Dias <ricardo.dias@percona.com>
python3.12-idle does not exist on EL9 (dropped from BuildRequires); the ldd-based verification audit was replaced by a DT_NEEDED soname audit, which detects unbundled libraries even inside the populated buildroot. Signed-off-by: Ricardo Dias <ricardo.dias@percona.com>
Signed-off-by: Ricardo Dias <ricardo.dias@percona.com>
The published python3.12-boto3 RPM carries runtime requirements python3.12dist(jmespath) >= 0.7.1 < 2 and python3.12dist(s3transfer) >= 0.12 < 0.13, but no project in the OBS repository path chains provides them, leaving percona-patroni-aws (and the tarball packages that install it) unresolvable in build chroots. Package both, mirroring the python3-boto3/python3-botocore layout: obs_scm service pinned to the upstream tag (jmespath 1.0.1, s3transfer 0.12.0 - the only 0.12.x release), python3.12 stack on EL, noarch, Epoch 1. s3transfer requires botocore, matching upstream metadata; jmespath has no python dependencies. Add ppg:staging:17 _aggregate packages so the staging repositories stay self-contained, same as python3-boto3. Signed-off-by: Ricardo Dias <ricardo.dias@percona.com>
Signed-off-by: Ricardo Dias <ricardo.dias@percona.com>
…nt repos User decision 2026-07-21: SSL variants become repositories (ssl1.1, ssl3, ssl3.5) of one ppg:staging:17:tarballs subproject instead of three subprojects. Per-repo differences move into %if "%_repository" conditionals (OBS defines _repository in every build config). The artifact is now written by the %build script directly to /usr/src/packages/OTHER with the exact official name, since the simpleimage recipe's raw-sed Name/Version extraction cannot vary per repository. Signed-off-by: Ricardo Dias <ricardo.dias@percona.com>
…t repos Replace the three per-SSL-variant subprojects (tarballs/ssl1.1, ssl3, ssl3.5) with a single ppg:staging:17:tarballs subproject that carries three repositories, one per variant, each stacked on its EL base (ssl1.1 -> RockyLinux 8, ssl3 -> RockyLinux 9, ssl3.5 -> RockyLinux 10). User decision: variants are repositories, not subprojects. Per-repository differences move to %if "%_repository" conditionals: the Python 3.12 vs default-python3 BuildRequires split in the simpleimage recipe, and the per-base resolution hints in the merged project config. The TARBALL_SSL_VARIANT / TARBALL_PYTHON_PKG macros and the per-variant macros.yaml files are gone. Artifact naming changes with it: the simpleimage recipe extracts Name:/Version: with a raw (unexpanded) sed, so the output name cannot vary per repository. The %build script now derives the full official name itself and writes the tarball directly into /usr/src/packages/OTHER (the OBS result dir), skipping /.simpleimage.tar.gz entirely (#!NoTarBall keeps the recipe's own tar step off). The SSL variant is derived from the buildroot's EL major version (os-release, with a glibc %dist-tag fallback) rather than the openssl-libs version, which can no longer tell EL9 from EL10 (EL 9.8+ and EL10 both ship OpenSSL 3.5.x). Unmapped EL majors fail the build loudly. The name now uses the official dash-arch form (percona-postgresql-<ver>-<variant>-linux-x86_64.tar.gz). Delete tests/test_tarball_variants.py: with a single package copy there is no byte-identity to guard. Signed-off-by: Ricardo Dias <ricardo.dias@percona.com>
The tarballs section still said the SSL variant is detected from the buildroot's openssl-libs version, which is the abandoned approach: EL 9.8+ and EL10 both ship OpenSSL 3.5.x, so the build script derives the variant from the buildroot's EL major version instead (/etc/os-release PLATFORM_ID, glibc %dist tag as fallback). Align the README with the committed script. Signed-off-by: Ricardo Dias <ricardo.dias@percona.com>
Signed-off-by: Ricardo Dias <ricardo.dias@percona.com>
…et::SSLeay The ssl variant labels promise host compatibility (ssl1.1 = OpenSSL 1.1 hosts, ssl3 = any OpenSSL 3.0+ host, ssl3.5 = 3.5+ hosts), and what enforces that at the ELF level is each binary's versioned symbol references (version NEEDS) against the host-provided libssl/libcrypto. That property was unguarded: the buildroot may ship a newer OpenSSL than the promise (the EL9 buildroot has 3.5.x), so a staging rebuild could silently start referencing newer version nodes. Add an OpenSSL host-ABI audit to the section-15 verification gate: readelf -V version NEEDS (needs only, never version definitions) are parsed per needed soname, filtered to libssl.so.*/libcrypto.so.*, and checked against a per-variant allowed-node pattern (ssl1.1: OPENSSL_1_1_*, ssl3: OPENSSL_3.0.x only, ssl3.5: up to 3.5.x). The /opt/percona-python3 tree is excluded: it bundles its own OpenSSL copy, so its OpenSSL symbol needs are satisfied internally and are not part of the host promise. The SSL_VARIANT detection (EL-major mapping) moves from section 16 to the top of section 15 since the audit's policy selection depends on it; section 16 reuses the variable. Prune Net::SSLeay from the bundled perl (auto/Net/SSLeay XS dirs plus Net/SSLeay.pm/.pod), for parity with the official tarball, which does not ship it; its XS module links the host libssl/libcrypto with OPENSSL_3.2.0 needs that would violate the variant promises. NOTE: the gate now intentionally FAILS on the current EL9/Rocky 9.8 base — pgcrypto.so inherits an OPENSSL_3.4.0 reference from the staging build (verified: the official tarball's pgcrypto needs only OPENSSL_3.0.0). The ssl3 variant is being rebased onto Ubuntu 22.04 debs in a follow-up commit (its pgcrypto needs only OPENSSL_3.0.0); the strict OPENSSL_3.0.x policy is the documented reason for that rebase and stays as-is. Signed-off-by: Ricardo Dias <ricardo.dias@percona.com>
Signed-off-by: Ricardo Dias <ricardo.dias@percona.com>
…-deb.sh) Section-by-section port of build-tarball.sh to Debian/Ubuntu paths and tools, sharing the same layout, helpers, SYSTEM_LIBS_EXCLUDE list and verification gate (DT_NEEDED soname audit, per-variant OpenSSL host-ABI audit, smoke commands). Key mappings: PostgreSQL under /usr/lib/postgresql/<major> (real binaries — /usr/bin/psql is a postgresql-common pg_wrapper symlink), ecpg + libecpg/libpgtypes from libecpg-dev, headers from /usr/include/postgresql with the <major>/server nesting flattened to include/server, pgbouncer/pgpool tools in /usr/sbin, config in /etc/pgpool2, python 3.10 with platlibdir=lib (no lib64 symlink hack) and third-party modules merged into the version-agnostic lib/python3/dist-packages that Debian's site.py resolves under a foreign PYTHONHOME, perl flattened from archlib/privlib/vendorarch/vendorlib, tcl stdlib from /usr/share/tcltk, multiarch libdir throughout, and version discovery via dpkg-query (epoch and revision stripped). Variant mapping is deliberate per deb base: ubuntu-22.04 -> ssl3, anything else is fatal. Validated end-to-end in an ubuntu:22.04 container against the published staging Ubuntu_22.04 debs: both audits clean under the strict ssl3 policy (only OPENSSL_3.0.x version nodes), all six smoke commands pass, artifact percona-postgresql-17.10-ssl3-linux-x86_64.tar.gz with the ten official percona-* component dirs. Signed-off-by: Ricardo Dias <ricardo.dias@percona.com>
The ssl3 variant promises to run on any OpenSSL 3.0 host, but its EL9 base can no longer deliver that: Rocky 9.8+ ships OpenSSL 3.5.x, so staging-built binaries (pgcrypto.so) reference OPENSSL_3.4.0 symbol version nodes and the gate added in commit cd4f043 correctly fails the build. Ubuntu 22.04 ships OpenSSL 3.0.2 (glibc 2.35), so its binaries can only reference OPENSSL_3.0.x nodes — verified against the published staging Ubuntu_22.04 debs. - simpleimage: wrap BuildRequires in %if "%_repository" == "ssl3" (deb binary package names, verified against the published Packages index) with the existing RPM list — including the nested ssl3.5 python3 split — in the %else branch; %build becomes a small os-release dispatcher running build-tarball-deb.sh on deb bases and build-tarball.sh elsewhere. - project.yaml: ssl3 repo paths now mirror staging's Ubuntu_22.04 chain (staging + common:deps + Ubuntu:22.04/universe); the EL9 hdf-libs Prefer block is dropped (no deb-side hints needed yet). - build-tarball.sh: EL9 is now an unmapped (fatal) tarball base; prune IO::Socket::SSL alongside Net::SSLeay (pure perl, but it can never load without the pruned XS module, and the official percona-perl tree ships neither); clarify the python-tree OpenSSL audit exclusion comment. - root/README.md: document the new ssl3 base, the two parallel builder scripts + dispatcher, and the deb-side python divergences (3.10, dist-packages). Signed-off-by: Ricardo Dias <ricardo.dias@percona.com>
RHEL 9.8 rebased to OpenSSL 3.5 and the EL9 staging pgcrypto.so references OPENSSL_3.4.0, breaking the ssl3 (OpenSSL 3.0 hosts) promise. ssl3 now consumes staging Ubuntu 22.04 debs via a second builder (build-tarball-deb.sh) selected by an os-release dispatcher; the verification gate enforces each variant's OpenSSL promise via a versioned-symbol-needs audit. Signed-off-by: Ricardo Dias <ricardo.dias@percona.com>
Signed-off-by: Ricardo Dias <ricardo.dias@percona.com>
…r decision) Signed-off-by: Ricardo Dias <ricardo.dias@percona.com>
…3.0 symbols
Completes WIP a772758 (task 20, user-approved plan addition).
Built on Rocky 9 (OpenSSL 3.5 headers), two extension modules picked up
versioned symbol needs newer than 3.0.0:
- _ssl: X509_STORE_get1_objects@OPENSSL_3.3.0
- _hashlib: EVP_MD_CTX_get_size_ex@OPENSSL_3.4.0 (via OpenSSL 3.4's
EVP_MD_CTX_size macro remap)
On an OpenSSL 3.0 host (Ubuntu 22.04/24.04, Debian 12) the postgres
backend already has the HOST libcrypto/libssl mapped, soname dedup wins
over the tarball's bundled 3.5 copies, and plpython3u `import ssl` dies
with "version OPENSSL_3.3.0 not found". The official tarball carries the
identical defect; this fixes it.
python-3.12-openssl30-symbol-compat.patch forces CPython's own
3.0-compatible paths (same philosophy as the pgcrypto OpenSSL-3.4
patch): the _ssl polyfill of X509_STORE_get1_objects (upstream code,
compiled unconditionally under a renamed symbol) and the pre-3.4
EVP_MD_CTX_size expansion. Inert on EL8 (OpenSSL 1.1.1 headers):
before/after version needs are byte-identical there.
A %check gate pins the promise at the source: rpmbuild fails loudly if
_ssl/_hashlib need OPENSSL_3.[1-9]* (EL9) or RHEL-only OPENSSL_1_1_1x
nodes (EL8), or if any other lib-dynload module links OpenSSL at all.
Validated: rebuilt in rockylinux:8 and rockylinux:9; EL9 needs now floor
at OPENSSL_3.0.0; standalone `import ssl, hashlib` green on both bases;
decisive e2e on ubuntu:22.04 (OpenSSL 3.0.2) with the tarball-linkage
simulation and bare `postgres -D` from `env -i`:
`DO $$ import ssl $$ LANGUAGE plpython3u` failed before
("version OPENSSL_3.3.0 not found") and passes after.
Signed-off-by: Ricardo Dias <ricardo.dias@percona.com>
…ssing-module guard build-tarball.sh (task-17 review riders): - PY_BIN: append || true inside the command substitution — under set -e, both lookups failing aborted the assignment itself, making the crafted FATAL on the next line dead code. - Section 0: assert bin/pip3 for the python tree — section 7's ln -sf pip3 bin/pip would silently create a dangling symlink if ensurepip ever stops running in the runtime RPM. percona-tarball-python3.spec (task-20 review rider): - %check module loop: guard the unmatched-glob case with an explicit missing-module FATAL. Previously a module that silently failed to build left $so as the literal glob, readelf errored and the || true swallowed it into a false PASS. No Release bump: the package has never been synced or built outside local validation. Signed-off-by: Ricardo Dias <ricardo.dias@percona.com>
…Ls, haproxy, perl constraint)
root/README.md, tarballs section:
- compiled /tmp socket defaults for server, initdb and every libpq
client via the same-length ELF string patch — no PGHOST, no
/run/postgresql host directory;
- zero-env PLs via the /opt-prefixed percona-tarball-{perl,tcl,python3}
runtimes in ppg:common:deps; only the psql readline shim remains;
- perl version constraint documented (5.26.3/5.32.1 distro-ABI-tied via
plperl.so; official's 5.38 comes from building PG against its own perl);
- eleven components incl. percona-haproxy; bytecode-free python;
- ssl3 promise now also covered by the CPython _ssl/_hashlib
OPENSSL_3.0.0 pin (import ssl works on all 3.x hosts);
- host prerequisites updated (postgres-style user, host readline for
psql, tzdata, /opt copy step); pruned the deleted machinery (initdb
wrapper, PGHOST, PERL5LIB/TCL_LIBRARY/PYTHONHOME exports and the
archive_command env -u PYTHONHOME leak note).
Spec doc: dated 2026-07-28 revision note summarizing the six QA-round
changes with commit refs; pipeline/wrappers/RPATH steps updated to the
implemented mechanism; caveats refreshed (PYTHONHOME caveat obsolete,
perl constraint and host prerequisites added); superseded env/initdb
wrappers moved to Rejected alternatives.
Signed-off-by: Ricardo Dias <ricardo.dias@percona.com>
Signed-off-by: Ricardo Dias <ricardo.dias@percona.com>
Signed-off-by: Ricardo Dias <ricardo.dias@percona.com>
…eline exclusions, dlopen gate) Signed-off-by: Ricardo Dias <ricardo.dias@percona.com>
…rl,tcl,python3} Package names now match their /opt/percona-* install prefixes and the tarball component directories (same scheme as the upcoming percona-psql, percona-gdal and percona-proj). No functional change: install paths, sources and patches are untouched. Signed-off-by: Ricardo Dias <ricardo.dias@percona.com>
The tarball's psql was built against the host's libreadline, which minimal Debian/Ubuntu hosts do not ship — hence the LD_PRELOAD/libreadline.so.7 symlink wrapper around psql.bin. Build psql from the same PG 17.10 source with --with-libedit-preferred instead, so it links BSD libedit (bundled with the tarball) and needs nothing from the host. The package is PG-version-bound and must be compiled against the exact EL base of the tarball that bundles it, so it lives in the tarballs project. That project's prjconf forced "Type: simpleimage" globally, so: - add plain RPM build repos RockyLinux_8 / RockyLinux_9 (same path chains as ssl1.1 / ssl3, publish disabled), - make "Type: simpleimage" (and Binarytype) repo-scoped to ssl1.1 / ssl3, - give ssl1.1 / ssl3 the sibling build repo as their first path so the simpleimage chroots can install percona-psql, - disable the tarball recipe on the RPM repos and percona-psql on the ssl repos via package.yaml build flags. The spec's %check gates the whole point of the package: the shipped binary must NEED libedit.so.0 and libpq.so.5 and must not mention libreadline. Verified with local rpmbuild on rockylinux:8 and rockylinux:9 (both green, %check passing), and by running the resulting psql against a PostgreSQL 17 server from a debian:12-slim container with no libreadline installed — Ctrl-A/Ctrl-D line editing and history recall both work. Signed-off-by: Ricardo Dias <ricardo.dias@percona.com>
Signed-off-by: Ricardo Dias <ricardo.dias@percona.com>
The tarball bundles the RPM-built postgis-*.so, so it has to carry a
libproj. EPEL's has its PROJ_DATA default compiled to /usr/share/proj, a
path that does not exist on the tarball's target hosts, so proj.db is
never found and ST_Transform() only works behind an env-exporting
wrapper — which QA proved is bypassable.
Build PROJ from source with the prefix /opt/percona-proj so the compiled
-in data path is /opt/percona-proj/share/proj and proj.db sits right
there: zero environment variables, the official tarball's mechanism (the
tarball is extracted at /opt, like percona-{perl,tcl,python3} already
are).
The upstream version is per base so the SONAME matches the libproj the
shipped PostGIS RPMs linked against, making ours a drop-in for the
bundled file:
RHEL 8 (ssl1.1): PROJ 6.3.2 -> libproj.so.15.3.2 (autotools)
RHEL 9 (ssl3): PROJ 9.6.0 -> libproj.so.25.9.6.0 (cmake; 9.x dropped
autotools)
Both real filenames match the ones in the pr-2 tarball artifacts. The
libproj.so.NN()(64bit) Provides is deliberately NOT filtered: the ssl
chroots will use a prjconf Prefer: to let this package satisfy PostGIS's
SONAME Requires instead of EPEL's proj-libs. RockyLinux_8/9 only.
%install asserts share/proj/proj.db exists and that the prefix string is
compiled into the library — the two properties the package exists for.
Distinct from ppg:common:deps `proj`, which is a Debian-only distro
shadow built to /usr; the two never meet in a chroot.
Signed-off-by: Ricardo Dias <ricardo.dias@percona.com>
The tarball bundles the RPM-built postgis_raster.so, so it has to carry a
libgdal. EPEL's was wrong on two counts:
1. EPEL enables every optional driver, so libgdal drags ~70 surplus
shared objects into the artifact (armadillo, poppler, hdf4/hdf5,
netcdf, OPeNDAP, xerces, libkml, unixODBC, mariadb, cfitsio, ogdi,
jasper/openjpeg, ...). Among them is a BLAS/LAPACK chain via
armadillo whose FlexiBLAS ELF constructor abort()s on Rocky hosts
(its dlopen'ed backend plugin is not bundled), and it needs
libtirpc/libexpat/libpcre2-posix that minimal hosts lack.
2. Its GDAL data dir is compiled to /usr/share/gdal, absent on the
tarball's target hosts.
Build GDAL from source with the prefix /opt/percona-gdal and exactly the
driver set the OFFICIAL Percona tarball's libgdal links — derived from
readelf -d on the official artifact's libgdal.so.36 — with an explicit
--without- for every other external-library driver. The data dir is
compiled to /opt/percona-gdal/share/gdal and PROJ comes from
percona-proj, so both resolve their resources with zero environment
variables.
Version is per base so the SONAME is a drop-in for the bundled file:
RHEL 8 (ssl1.1): GDAL 3.0.4 -> libgdal.so.26.0.4
RHEL 9 (ssl3): GDAL 3.4.3 -> libgdal.so.30.0.3
Both real filenames match the ones in the pr-2 tarball artifacts.
Two deliberate reductions below official: --with-gif=internal (official
links no libgif at all) and --with-qhull=internal — same driver
functionality, two fewer shared objects. EL8 uses --with-pcre because
GDAL 3.0.4 has no --with-pcre2 (and no --with-lz4); libpcre.so.1 is
already bundled via libspatialite, so it costs nothing.
The libgdal.so.NN()(64bit) Provides is deliberately NOT filtered, same
Prefer: mechanism as percona-proj. RockyLinux_8/9 only.
%install gates both invariants: the resource files must be in the
directory libgdal was told about (GDAL also carries a
<prefix>/share/gdal fallback string, so grepping the library alone would
pass a wrong --datadir), and libgdal must link none of the libraries the
official one does without.
Signed-off-by: Ricardo Dias <ricardo.dias@percona.com>
Three things the first green local rpmbuild on each base exposed:
1. GDAL 3.4.3 APPENDS "/gdal" to --datadir; GDAL 3.0.4 uses it verbatim.
A single --datadir=<prefix>/share/gdal therefore installed the EL9
resource files at <prefix>/share/gdal/gdal and compiled INST_DATA to
match. Self-consistent, so it worked, but it is not the path the
tarball builder (task 23) or anyone reading the spec expects. Made
--datadir per-version via %{gdal_datadir} so both bases land on
<prefix>/share/gdal.
2. That mistake slipped past the data-path gate, because GDAL also
carries an independent <prefix>/share/gdal fallback string, so
grepping the library alone passes even on a wrong --datadir. The gate
now also asserts the resource files are really there
(share/gdal/gdalicon.png, present in both 3.0.4 and 3.4.3) and that
share/gdal/gdal does not exist.
3. RHEL 9's %optflags carry -flto=auto, and libgdal is big enough that
the final link degenerates into "lto-wrapper: using serial compilation
of 128 LTRANS jobs" — minutes of single-threaded work that risks an
OBS build timeout for no benefit (upstream's own --enable-lto is off
by default and the autotools build is not LTO-tested). Disabled with
%global _lto_cflags %{nil}; a no-op on RHEL 8, whose optflags have no
LTO.
Also moved --without-sde into the RHEL 8 branch: GDAL 3.4.3 does not
register it and warned "unrecognized options". Both configures are now
warning-free.
Signed-off-by: Ricardo Dias <ricardo.dias@percona.com>
…SONAME gates
Review round 1 on task 22, three findings.
1. percona-gdal gets an explicit `Requires: percona-proj`. The automatic
libproj.so.NN()(64bit) ELF requires is ambiguous: EPEL's proj-libs
provides the very same symbol on both bases (6.3.2 -> libproj.so.15,
9.6.0 -> libproj.so.25), so `dnf install percona-gdal` could satisfy it
with proj-libs, leaving /opt/percona-proj/lib absent, libgdal's rpath
dead and the compiled-in data path mechanism broken. The point of the
package is *our* libproj, not any libproj with a matching SONAME.
2. libgdal gets a second rpath entry, `$ORIGIN`, next to
/opt/percona-proj/lib. postgis_raster-3.so's own RUNPATH $ORIGIN does
not help resolve libgdal's NEEDED libproj.so.NN — DT_RUNPATH is not
inherited down the load chain, it applies only to the object that
carries it. So without this, libproj resolution inside the tarball
depends entirely on task 23 materialising /opt/percona-proj/lib; with
it, a flattened tarball lib/ works too. Escaped as \$ORIGIN so the
literal reaches the linker rather than an empty shell expansion.
3. Both specs now gate SONAME parity, the one property they exist for:
test -e <prefix>/lib/libproj.so.{15,25} and libgdal.so.{26,30}, chosen
per base by %if. A source-URL bump, or the %if picking the wrong branch
on an unexpected %{?rhel}, would otherwise silently produce a library
that is no longer a drop-in for what PostGIS linked. The existing
data-path grep and the lean-driver readelf loop now address the same
explicit SONAME path instead of a libgdal.so.*.*.* / libproj.so.*.*
glob, so those gates fail loudly rather than on a glob mismatch.
Signed-off-by: Ricardo Dias <ricardo.dias@percona.com>
Signed-off-by: Ricardo Dias <ricardo.dias@percona.com>
…en gates
Wires the task-21/22 helper packages into the tarball build and turns the
QA findings into build-time gates.
simpleimage / project.yaml:
- BuildRequires percona-psql, percona-gdal, percona-proj.
- prjconf `Prefer: percona-gdal` / `Prefer: percona-proj` in BOTH ssl repo
blocks: PostGIS's libgdal.so.NN()(64bit)/libproj.so.NN()(64bit) Requires
are a "have choice" against EPEL's gdal-libs/proj and must resolve to
ours (EPEL's dragged ~70 surplus shared objects — including the
armadillo/FlexiBLAS chain whose ELF constructor abort()s on Rocky hosts
— and compiled its data paths to /usr/share/{gdal,proj}).
build-tarball.sh:
- Section 0a: assert the percona-gdal/percona-proj trees and their
compiled-in resource dirs, assert percona-psql, and FATAL if EPEL
gdal/gdal-libs/proj/proj-libs reached the chroot anyway.
- Section 2b: bin/psql is now the real libedit-linked percona-psql binary;
the whole psql.bin + LD_PRELOAD/libreadline.so.7-symlink wrapper is
deleted.
- Section 12b: prune /opt/percona-{gdal,proj} to the lean tarball layout
(lib/ + share/{gdal,proj}); they ship as two more top-level components,
so their compiled data paths resolve with zero environment variables.
- Section 13: run the bundling passes with LD_LIBRARY_PATH pointing at the
two /opt lib dirs (subshell-scoped, never leaks) so postgis_raster's
NEEDED libgdal/libproj resolve to OURS and get bundled next to it.
- SYSTEM_LIBS_EXCLUDE is now a documented universal host baseline;
libtirpc/libnsl/libeconf/libpcre2-8/libpcre2-posix/libexpat/libreadline
are removed from it and therefore bundled (QA finding 1).
- New gates: surplus dependency-chain audit (flexiblas/armadillo/hdf/
netcdf/dap/poppler/mariadb/odbc/kml/xerces/arpack/superlu), host-baseline
self-consistency gate, psql link audit (libedit in, libreadline out),
extension dlopen smoke from an empty environment (RTLD_LAZY, every
Pg_magic_func module plus the bundled libgdal/libproj), psql --version
smoke, component inventory 11 -> 13.
Signed-off-by: Ricardo Dias <ricardo.dias@percona.com>
…/proj layout Review round 1 on the task-23 gates. - CRITICAL: the dlopen smoke probe used ctypes.RTLD_LAZY, which does not exist (ctypes exports only RTLD_GLOBAL/RTLD_LOCAL; RTLD_LAZY lives in os) — every module would have failed with an AttributeError and killed §15 on both ssl repos. Payload is now 'import ctypes, os, sys; ctypes.CDLL(sys.argv[1], mode=os.RTLD_LAZY)'. - "undefined symbol" is no longer an unconditional pass: the symbol name is extracted from the loader message and must be defined by bin/postgres (nm -D --defined-only, computed once, @vers stripped) — the precise "postgres-internal" test. A bundled-dependency ABI mismatch (undefined symbol: GEOSDensify) is now FATAL instead of being logged as OK. Degraded mode (no nm) says so explicitly. - Host-baseline gate resolves PER COMPONENT: a formerly-excluded soname must sit in the needing ELF's own component lib/ dir or next to the ELF ($ORIGIN), not merely somewhere under /opt — a libexpat in percona-python3/lib no longer satisfies haproxy's NEED. - percona-gdal/percona-proj ship DATA ONLY: lib/ is pruned too, so the artifact carries no dead second copy of libgdal/libproj (unloadable, several MB, extra SSL-ABI audit surface). The prune moved to a new section 13a, AFTER the bundling pass that reads those very lib/ dirs through LD_LIBRARY_PATH; new assert that lib/ is gone. Inventory stays at 13 components. Signed-off-by: Ricardo Dias <ricardo.dias@percona.com>
Signed-off-by: Ricardo Dias <ricardo.dias@percona.com>
Reusable acceptance battery for the OBS-built PostgreSQL binary tarballs. It never builds anything: given an artifact it derives the SSL variant from the file name and runs it on that variant's minimal container images (debian:11-slim/ubuntu:20.04 for ssl1.1; debian:12-slim, ubuntu 22.04/24.04 and rockylinux 9/10-minimal for ssl3), podman or docker, autodetected. Per host: copy every top-level percona-* component to /opt (the documented install step), create a non-root user, initdb and start a bare `postgres -D` under env -i, assert the compiled /tmp socket default and that /run/postgresql is never created, then CREATE EXTENSION every shipped share/extension/*.control (one restart with shared_preload_libraries for the extensions that need it, from a small allowlist), the PostGIS/GDAL/PROJ deep checks (postgis_full_version DATABASE_PATH, ST_Transform to 3857, ST_AddBand raster, GDAL_DATA), the three PLs, psql in pipe and pty mode and every bundled client with zero environment. Failures are reported by name; the run exits non-zero if any check fails on any host, and prints a host x check matrix. The only packages it may install are the artifact's two documented host prerequisites: tzdata, and the distro OpenSSL runtime of the variant's generation (some minimal images ship none at all; --no-install turns even that off and reports the image as skipped). Never readline, libtirpc, expat, pcre2, gdal, proj, python, perl or tcl — the checks that would need a tool the image lacks (readelf) run host-side against the extracted tree: component inventory (13), the psql link audit (libedit in, libreadline out), the per-component NEEDED closure against the universal host baseline (read from build-tarball.sh when reachable, embedded copy otherwise), and the surplus-library scan (libflexiblas & co.). Signed-off-by: Ricardo Dias <ricardo.dias@percona.com>
…ROJ) root/README.md's tarballs section had drifted: it described one package, two repositories, eleven components and a host-readline psql shim. Updated to the tree as it is: two packages (percona-postgresql-tarball, percona-psql), four repositories (ssl1.1/ssl3 simpleimage, published; RockyLinux_8/RockyLinux_9 RPM build repos for percona-psql, unpublished), thirteen components including the data-only percona-gdal/percona-proj resource trees, the libedit psql with no wrapper anywhere, the universal host-baseline bundling contract and its two gates, and host prerequisites that are now just a user, tzdata and copying ALL percona-* dirs to /opt (with the caveat that a few minimal images ship no OpenSSL runtime at all). Also mentions tools/tarball-acceptance.sh. The design doc gets a "Revision note — 2026-07-29 QA round 2" in the style of the 2026-07-28 one: the three findings (postgis_raster on minimal hosts, the FlexiBLAS constructor abort on Rocky 9/10, psql needing host readline), their root causes (an exclusion list copied from pg_tarballs_builder.sh, EPEL's kitchen-sink GDAL with its dlopen'ed BLAS plugin, the readline psql) and the fixes; the body sections that described the superseded mechanisms (exclusion list, psql wrapper, eleven components, one-package project, project-wide Type: simpleimage, the acceptance step) are updated in place. Signed-off-by: Ricardo Dias <ricardo.dias@percona.com>
Four review findings. 1. (critical) The NEEDED-closure host check applied the per-component reachability rule to every non-baseline soname, so it flagged the three deliberate cross-component PL RUNPATHs (plperl.so -> /opt/percona-perl/lib/<ver>/CORE, plpython3.so -> /opt/percona-python3/lib, pltcl.so -> /opt/percona-tcl/lib, where libperl/libpython3.12/libtcl8.6 live) as MISPLACED and would have failed a correct artifact. It now mirrors build-tarball.sh's two gates: artifact-wide bundling for sonames in general, the strict per-component rule only for FORMERLY_EXCLUDED_LIBS — read live out of the builder alongside the baseline, with an embedded fallback copy. 2. A run in which every image was skipped printed ACCEPTANCE PASSED. It is now ACCEPTANCE INCOMPLETE with exit status 3, and a partially skipped run says so in the PASSED line. 3. A failed OpenSSL install was indistinguishable from an image that ships none, so a no-network apt-get silently dropped coverage. Setup now reports present / missing-install-disabled / missing-after-install-attempt, and only the middle one is a skip — a failed install attempt fails that host. 4. Nothing but the server-ready wait was bounded. The per-image battery now runs under timeout(1) (--host-timeout, default 1200s, env TARBALL_ACCEPTANCE_HOST_TIMEOUT), and initdb, CREATE EXTENSION, the deep queries, both psql modes and the client checks have their own shorter timeouts inside the guest, which probes for a usable timeout(1) first and says so if it has none. Also fixes the design doc's Rocky image names to the fully-qualified Docker Hub form (the short rockylinux: repo has no 10-minimal tag). Signed-off-by: Ricardo Dias <ricardo.dias@percona.com>
Signed-off-by: Ricardo Dias <ricardo.dias@percona.com>
…ew fixes
PostGIS requires the distro GDAL/PROJ BY NAME (rpmspec -P: EL8 gdal-libs +
proj, EL9 gdal3.4-libs + proj, all in the main percona-postgis35_NN package),
and `Prefer:` only breaks ties between providers of the same symbol. So the
fat EPEL packages landed in both ssl chroots regardless of the Prefer lines
and build-tarball.sh section 0a would have aborted on the first sync. Add
scoped `Ignore: percona-postgis35_%!{PG_MAJOR_VERSION}:<deps>` rules to both
ssl blocks; keep the Prefer lines, which remain load-bearing for the
automatic libgdal.so.NN()(64bit)/libproj.so.NN()(64bit) soname Requires.
Also from the whole-round review:
* build-tarball.sh section 0a: replace the name-blind EPEL-absence assert
(which missed EL9's gdal3.4-libs) with a property check on
/usr/lib64/lib{gdal,proj}.so.* — ours live under /opt — and print
rpm -qf plus rpm -q --whatrequires so the log names the requirer.
* project.yaml: `Prefer: hdf-libs` is live, not inert — PostGIS requires
hdf5 by name and hdf5's libdf.so.0 has two providers. Fix the comment.
* percona-psql.spec: %global __requires_exclude ^libpq\\.so — libpq.so.5 has
two providers in the ssl chroots (percona-postgresql17-libs and Rocky's
libpq); the tarball bundles its own libpq and %check still asserts NEEDED.
* percona-gdal/percona-proj: ExcludeArch: aarch64 — the tarball is x86_64
only, but ppg:common:deps inherits aarch64 from root/project.yaml.
* tarball-acceptance.sh: postgis_full_version() only prints DATABASE_PATH=
above a PROJ version floor and ssl1.1 links PROJ 6.3.2, so an absent
suffix is a WARN and only a foreign path is a FAIL (ST_Transform stays the
proj.db proof); when script(1) is missing (rockylinux:*-minimal) drive the
pty with the artifact's own bundled python3 instead of skipping, degrade to
a "PASS*" matrix cell plus a legend line only if that also fails, and
assert the computed result (41+1) rather than the pty's own input echo.
Signed-off-by: Ricardo Dias <ricardo.dias@percona.com>
b90ac24 made the check property-based but left `stray_gis=$(ls … 2>/dev/null)` unguarded. build-tarball.sh runs under `set -e`, and a bare `var=$(cmd)` takes the substitution's exit status: with no match the globs stay literal, ls exits 2, and the script died BEFORE the `if` — so every CLEAN build (the expected state) and every partial hit aborted silently at section 0a, and the FATAL diagnostics could only ever print when BOTH globs matched. Same trap the file already documents for PY_BIN and for copy_deps' find. Append `|| true` inside the substitution and leave `if [ -n "$stray_gis" ]` as the only gate. Verified with the block extracted verbatim: empty dir continues (rc 0), a dir holding only libproj.so.25 takes the FATAL path with the rpm -qf owner and the --whatrequires requirer (rc 1). Also surface the I5 warning on the console: the host prints only the status field into the matrix, so report the postgis cell as "PASS*" when postgis_full_version() prints no DATABASE_PATH= (PROJ below the floor that reports it), and generalise the existing legend line to cover both the psql and postgis cases. Signed-off-by: Ricardo Dias <ricardo.dias@percona.com>
Signed-off-by: Ricardo Dias <ricardo.dias@percona.com>
The first OBS build of the ssl1.1/ssl3 simpleimage repos proved that OBS does not apply prjconf `Ignore:` rules when expanding image-type recipes: the effective project config carried `Ignore: percona-postgis35_17:gdal-libs,proj` (and the EL9 variant), yet EPEL's gdal-libs/gdal3.4-libs and Rocky's proj were still installed and build-tarball.sh section 0a FATALed as designed. `Prefer:` IS honoured. Replace the inert Ignore: lines with a payload-free shim package, percona-gis-compat, that Provides the distro names percona-postgis35_17 requires by name (proj/gdal-libs on EL8, proj/gdal3.4-libs on EL9, versioned to track percona-proj/percona-gdal so the `>= 3.0.4` Requires stays satisfied) and Requires percona-gdal/percona-proj. Each by-name Requires becomes a real have-choice that `Prefer: percona-gis-compat` resolves towards our lean /opt runtimes. Like percona-psql it builds only in this project's unpublished RockyLinux_8/RockyLinux_9 repos, so the fake Provides stay invisible to every other project, and the simpleimage recipe BuildRequires it explicitly so its presence is deterministic. Section 0a's property-based FATAL is unchanged; its diagnostic now queries rpm --whatrequires with a package NAME and the soname capability instead of the full NEVRA, which had made it print "no package requires ...". Signed-off-by: Ricardo Dias <ricardo.dias@percona.com>
The RockyLinux_8 build of da1e31c failed with "error: line 75: Unknown tag: %elif 08 == 9": %elif landed in rpm 4.15, so EL8's rpm 4.14 cannot parse it (RockyLinux_9 built fine). Rewrite the per-base Provides block as nested %if/%else/%endif and note why. No functional change: rpmspec -P still yields proj = 6.3.2 / gdal-libs = 3.0.4 on EL8, proj = 9.6.0 / gdal3.4-libs = 3.4.3 on EL9, and %{error} on any other base. Signed-off-by: Ricardo Dias <ricardo.dias@percona.com>
Signed-off-by: Ricardo Dias <ricardo.dias@percona.com>
rjd15372
force-pushed
the
tarballs-simpleimage
branch
from
August 28, 2026 10:12
c255628 to
72cffe5
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.