pgAdmin 4 for UBI-9: SP1 node_modules tooling + SP2 local-npm-registry + SP3 Python 3.12 stack

.github/copilot-instructions.md

@@ -497,13 +497,14 @@ Options:
 
 ### Local service execution
 
-If a package's `obs/_service` contains any service with `mode="manual"`, `sync` automatically runs all non-buildtime services locally before uploading. This is required for packages like Go services that use `go_modules` (mode=manual) to vendor dependencies.
+If a package's `obs/_service` contains any service with `mode="manual"`, `sync` automatically runs all non-buildtime services locally before uploading. This is required for packages like Go services that use `go_modules` (mode=manual) or npm frontends that use `npm_lockfile` + `node_modules` (mode=manual) to vendor dependencies.
 
 **Execution order and file handling:**
 1. All services with `mode` not in `{buildtime, serveronly, disabled}` are run in XML declaration order.
 2. Each service binary is invoked from `/usr/lib/obs/service/<name>` with its `<param>` values and `--outdir`.
 3. Service outputs are merged into a shared work directory so later services can consume earlier outputs (e.g. `go_modules` consuming `obs_scm` tarballs).
 4. Only files produced by `mode="manual"` services are committed to OBS. Files produced by no-mode services (e.g. obs_scm source tarballs) are used locally but **not** uploaded — OBS regenerates those on its server.
+5. `.obscpio` files produced by `mode="manual"` services (e.g. `node_modules.obscpio`) are kept and uploaded; only `obs_scm` archives are extracted locally.
 
 If a service binary is missing from `/usr/lib/obs/service/`, a warning is logged and the service is skipped. A non-zero service exit code aborts the entire `sync` run.
 
@@ -541,6 +542,8 @@ Before invoking each `download_url` service binary, `percona-obs` computes `para
 
 **`--no-cache`**: pass to `sync` to bypass all cache levels unconditionally for that run.
 
+**Drift-tolerant artifacts** (`services.drift_tolerant_patterns`): outputs whose bytes depend on an external registry's state at generation time — `cargo_vendor` → `vendor.tar.*`; `node_modules` → `node_modules.obscpio`, `node_modules.spec.inc`, `*package-lock.json` — are compared by presence only in the `--branch-from` content check (`cmd_sync._content_mismatches`). A file missing on either side is still a mismatch.
+
 When targeting a specific package (`sync <project> <package>`), the ancestor project chain is only walked if the target project does not yet exist on OBS (fast path avoids redundant GET calls otherwise).
 
 Project names use colon notation matching the directory hierarchy (e.g. `ppg:staging:17`).
@@ -959,6 +962,8 @@ When branching is involved, always confirm which OBS instance is being queried:
 | Third-party infrastructure service | `ppg/staging/17/etcd/` |
 | OBS aggregate (mirrors another OBS project) | `obs-service-tar_scm/` |
 | Root project config | `root/project.yaml` |
+| Repo-owned OBS service (npm lockfile generation) | `tools/obs-services/npm_lockfile` |
+| Version-independent devel project (pgAdmin) | `root/ppg/devel/pgadmin/project.yaml` |
 | Management script | `percona-obs` (commands: `sync push`, `sync delete`, `sync promote`, `build trigger`, `build status`, `build dependency`, `profile create`, `profile list`, `project verify`) |
 
 ---

.gitignore

@@ -8,3 +8,4 @@ __pycache__/
 tmp/
 docs/plans/
 .percona-obs/
+.superpowers/

CLAUDE.md

@@ -51,6 +51,7 @@ Always use `git commit -s` (signed-off-by). Do not add `Co-Authored-By: Claude`
 | `percona_obs/targets.py` | Target resolution helpers |
 | `percona_obs/services.py` | Local OBS service execution + caching |
 | `percona_obs/git_utils.py` | Git helpers |
+| `tools/obs-services/` | Repo-owned OBS source services run locally by `sync` (e.g. `npm_lockfile`); baked into the obs-tools image |
 | `requirements.txt` | Runtime + dev dependencies (`osc`, `pyyaml`, `black`, `pyright`) |
 | `pyrightconfig.json` | Pyright config pointing at `venv/` and `percona_obs/` |
 | `root/` | All packaging content (project hierarchy mirrors OBS) |

docs/PACKAGING_HOWTO.md

@@ -396,6 +396,123 @@ After fixing packaging files, trigger a rebuild (the `_service` file has not cha
 
 ---
 
+## Python 3.12 packages (pyproject builds)
+
+RPM-only Python libraries for RHEL-family targets are built for `/usr/bin/python3.12`
+and named `python3.12-<name>` (`%{python3_pkgprefix}-<name>`; plain `python3-<name>` on
+openSUSE). The directory is `python3-<PyPI name lower-cased, `.`/`_` → `-`>`. Examples:
+everything under `root/ppg/devel/pgadmin/python3-*` and the build-backend stack in
+`root/ppg/common/deps/` (`python3-flit-core`, `python3-packaging`, `python3-pathspec`,
+`python3-trove-classifiers`, `python3-hatchling`).
+
+**Reuse before you build.** RHEL 9 already ships `python3.12-{cffi,cryptography,idna,
+pycparser,urllib3,setuptools,pip,wheel}` (AppStream) and `python3.12-{packaging 23.2,
+pluggy,pytest,setuptools-rust,flit-core 3.9,Cython 0.29 (RPM name `python3.12-Cython`,
+capital C)}` (CRB); `ppg:common:deps` has
+`six`, `dateutil`, `psutil`, `click`, `dns`. Depend on those by their RPM name — without a
+version floor, since RHEL's versions are often older than upstream's declared minimum but
+work — and only package what is missing. RHEL's *build backends* are too old for current
+PEP 639 metadata; use ours from `ppg:common:deps` (see the conditional below).
+
+**Source:** the PyPI sdist via `download_url`, literal version:
+
+```xml
+<services>
+  <service name="download_url">
+    <param name="url">https://files.pythonhosted.org/packages/source/f/flask/flask-3.1.3.tar.gz</param>
+  </service>
+</services>
+```
+
+**Spec header** (identical in every package; copy from `root/ppg/devel/pgadmin/python3-flask/rpm/python3-flask.spec`):
+
+```rpmspec
+%global debug_package %{nil}          # noarch only; drop for C/Rust extensions
+%if 0%{?rhel} && 0%{?rhel} >= 8
+%global __ospython        %{_bindir}/python3.12
+%global python3_pkgprefix python3.12
+%global python3_buildversion 3.12
+%global __requires_exclude ^python3\\.12dist
+%else
+%global __ospython        %{_bindir}/python3
+%global python3_pkgprefix python3
+%global python3_buildversion 3
+%endif
+%{expand: %%global py3ver %(echo `%{__ospython} -P -c "import sys; print(f'{sys.version_info[0]}.{sys.version_info[1]}')" `)}
+%global python3_sitelib %(%{__ospython} -Esc "import sysconfig; print(sysconfig.get_path('purelib', vars={'platbase': '/usr', 'base': '%{_prefix}'}))")
+# extensions: python3_sitearch with 'platlib' instead
+
+Name:           %{python3_pkgprefix}-flask
+Version:        3.1.3
+Release:        1%{?dist}
+Source0:        https://files.pythonhosted.org/packages/source/f/flask/flask-3.1.3.tar.gz
+BuildArch:      noarch
+Epoch:          1
+BuildRequires:  python%{python3_buildversion}-devel
+BuildRequires:  python%{python3_buildversion}-pip
+BuildRequires:  python%{python3_buildversion}-setuptools
+BuildRequires:  python%{python3_buildversion}-wheel
+```
+
+Then the backend the sdist's `[build-system]` names — `python%{python3_buildversion}-flit-core`,
+`%{python3_pkgprefix}-poetry-core`, `%{python3_pkgprefix}-pdm-backend`, `%{python3_pkgprefix}-setuptools_scm`
+(EPEL) when `setuptools_scm` is listed — and for hatchling the distro/ours switch:
+
+```rpmspec
+%if 0%{?rhel} == 8 || 0%{?rhel} == 9
+BuildRequires:  %{python3_pkgprefix}-hatchling      # ours, ppg:common:deps
+%else
+BuildRequires:  python3-hatchling                    # EL10 CRB / openSUSE
+%endif
+```
+
+Every runtime `Requires:` is also a `BuildRequires:` so `%check` can import the module.
+
+**Build/install — pyproject packages** (setuptools, flit, hatchling, poetry, pdm):
+
+```rpmspec
+%prep
+%autosetup -p1 -n flask-%{version}
+
+%build
+%{__ospython} -m pip wheel --no-deps --no-build-isolation --no-index --wheel-dir dist .
+
+%install
+%{__ospython} -m pip install --no-deps --no-index --root %{buildroot} --prefix %{_prefix} dist/*.whl
+
+%check
+PYTHONPATH=%{buildroot}%{python3_sitelib} %{__ospython} -P -c "import flask"
+
+%files
+%{python3_sitelib}/*
+%{_bindir}/flask
+```
+
+The `-P` flag stops Python from putting the source directory first on `sys.path`;
+without it the smoke test imports the unpacked source tree instead of the installed
+package (and fails for C extensions).
+
+Self-hosting backends (hatchling, poetry-core, pdm-backend, flit-core) add
+`export PYTHONPATH=$PWD/src` (flit-core: `$PWD/.`) before `pip wheel`. Rust extensions
+(bcrypt) add a `cargo_vendor` service (`cargotoml` pointing at the crate's `Cargo.toml`),
+`Source1: vendor.tar.gz`, `%autosetup -a1`, `CARGO_NET_OFFLINE=true`.
+
+**PEP 639 licence metadata with setuptools 68.** Sdists whose `pyproject.toml` has
+`license = "SPDX-ID"` (a string) and/or `license-files = [...]` fail on RHEL's setuptools 68
+(`project.license must be valid exactly by one definition`). Add to `%prep`, after
+`%autosetup`: `sed -i -e 's/^license = "\(.*\)"$/license = {text = "\1"}/' -e '/^license-files = \[$/,/^\]$/d' -e '/^license-files = \[.*\]$/d' pyproject.toml`
+(see `root/ppg/devel/pgadmin/python3-greenlet/rpm/python3-greenlet.spec`).
+
+**Build/install — legacy `setup.py`-only packages** (no `pyproject.toml`): keep the
+`%{__ospython} setup.py build` / `setup.py install --single-version-externally-managed -O1
+--root=%{buildroot} --record=INSTALLED_FILES` recipe and `%files -f INSTALLED_FILES`, as in
+`root/ppg/common/deps/python3-click`.
+
+**Bumping:** edit the sdist URL in `obs/_service`, `Version:` in the spec, add a
+`%changelog` entry. Nothing is generated.
+
+---
+
 ## Adding a Package as an Aggregate in a Subproject
 
 When a package is built in one subproject (e.g. `common:deps:runtime`) and consumed by
@@ -545,6 +662,7 @@ version before assuming this.
 | `debian/patches/` | if needed | Quilt patches applied to upstream source |
 | `debian/*.install` | if needed | File installation lists for binary packages |
 | `rpm/<name>.spec` | yes | `Version: 1.0.0`; `Release: 1%{?dist}`; no `Source0` file in `rpm/` |
+| `rpm/python3-<name>.spec` | Python 3.12 libs | Template in "Python 3.12 packages"; `Version:` literal; runtime deps also `BuildRequires` |
 | `rpm/<extra-sources>` | if needed | Only files referenced as `Source1`, `Source2`, … in the spec |
 | `obs/_service` | yes | Use env-var template for packaging services; customise only the upstream service |
 | `obs/_aggregate` | aggregates only | Replaces `_service`; references pre-built binaries from another project |

docs/PERCONA_OBS_TOOL.md

@@ -17,6 +17,11 @@ dependency vendoring):
 | `obs_scm` | `obs-service-obs-scm` | `obs-service-obs_scm` |
 | `go_modules` | `obs-service-go_modules` | `obs-service-go_modules` |
 | `download_url` | `obs-service-download_url` | `obs-service-download_url` |
+| `node_modules` | `obs-service-node-modules` (openSUSE:Tools repo) | `obs-service-node_modules` (openSUSE:Tools repo) |
+| `npm_lockfile` | this repo — `tools/obs-services/` (see its README) | same |
+
+`node_modules` and `npm_lockfile` additionally need `npm` (Node.js ≥ 18; the CI image
+uses 22) and `cpio` on `PATH`.
 
 Binaries are expected at `/usr/lib/obs/service/<name>`.
 
@@ -121,6 +126,64 @@ see [root/README.md](../root/README.md) — not covered further here.)
 ./percona-obs -P local sync push ppg:staging:17 percona-pg-telemetry
 ```
 
+### Vendoring npm dependencies (`npm_lockfile` + `node_modules`)
+
+Packages with a webpack/npm frontend (pgAdmin 4) vendor their npm dependencies for
+offline OBS builds with two `mode="manual"` services, run locally by `sync` in this order:
+
+```xml
+<service name="npm_lockfile" mode="manual">
+  <param name="archive">*percona-pgadmin4*.obscpio</param>   <!-- the upstream obs_scm output -->
+  <param name="subdir">web</param>                            <!-- where package.json lives -->
+</service>
+<service name="node_modules" mode="manual">
+  <param name="cpio">node_modules.obscpio</param>
+  <param name="output">node_modules.spec.inc</param>
+  <param name="source-offset">10000</param>
+</service>
+```
+
+`npm_lockfile` (repo-owned, `tools/obs-services/`) generates `package-lock.json` with
+`npm install --package-lock-only`; `node_modules` (openSUSE) reads it and downloads every
+tarball into `node_modules.obscpio`, writing `SourceNNNNN:` lines to `node_modules.spec.inc`
+(`%include %{_sourcedir}/node_modules.spec.inc` in the spec; `BuildRequires: local-npm-registry`
+serves the tarballs to `npm` at build time). All three files are uploaded to OBS and cached
+under `.cache/services/<upstream commit>/` — the lockfile is a function of the upstream commit
+plus registry state, so `--no-cache` is the way to deliberately refresh the vendored set.
+Each new upstream commit of an npm-vendored package adds a cache entry of roughly the
+`.obscpio`'s size (~220 MB for pgAdmin); this is bounded locally by `prune_cache`, which
+removes entries unused for 7 days, and in CI additionally by the `actions/cache` size limits.
+`npm_lockfile` also accepts an optional `npm-flags` param (default `--legacy-peer-deps --ignore-scripts`; a given value replaces the default set) for upstreams that need different npm resolution flags.
+
+`root/ppg/devel/pgadmin/percona-pgadmin4/obs/_service` is the reference user of this chain:
+`obs_scm` (tag `REL-9_17`, `versionrewrite-pattern REL-(\d+)_(\d+)` → `9.17`) →
+`npm_lockfile` (`subdir web`) → `node_modules` (`source-offset 10000`) → `tar`/`recompress`/
+`set_version` at build time. A `sync push --dry-run` of that package runs the whole chain
+locally and takes several minutes (it downloads every npm tarball once; later runs hit the
+service cache).
+
+Two rules in `percona-obs` make this work:
+
+- `.obscpio` files produced by **manual** services are uploaded as-is (OBS unpacks them into
+  `SOURCES` at build time); only `obs_scm` archives are extracted locally.
+- **Drift-tolerant artifacts**: outputs whose bytes depend on registry state at generation
+  time are compared by presence only in the `--branch-from` content check, so re-resolution
+  never promotes a package by itself (absence on either side is still a change):
+
+  | Service | Tolerated files |
+  |---|---|
+  | `cargo_vendor` | `vendor.tar.*` |
+  | `node_modules` | `node_modules.obscpio`, `node_modules.spec.inc`, `*package-lock.json` (names follow the `cpio`/`output`/`input` params) |
+
+  Drift tolerance only affects the `--branch-from` content check's changed/unchanged
+  decision — it does not affect what gets uploaded. A sync that is not skipped (no
+  matching `--skip-unchanged` manifest entry, or run with `--force`) still uploads
+  whatever the local service run produced, so a regenerated drift-tolerant artifact
+  (for pgAdmin, a ~220 MB `node_modules.obscpio`) is re-PUT to OBS and commits a new
+  revision, triggering a rebuild even though nothing meaningful changed. `--skip-unchanged`
+  avoids this in the normal CI path because tag-pinned packages are skipped outright before
+  any service reruns; `--no-cache` is the deliberate way to force that refresh.
+
 ### Sync all packages under a subproject
 
 ```sh