diff --git a/.github/workflows/check-updates.yml b/.github/workflows/check-updates.yml index ed920a8d..4b3a2f84 100644 --- a/.github/workflows/check-updates.yml +++ b/.github/workflows/check-updates.yml @@ -156,6 +156,7 @@ jobs: REPOSITORY: ${{ github.repository }} RUN_ID: ${{ github.run_id }} RUN_ATTEMPT: ${{ github.run_attempt }} + RELEASE_TRUST_JSON: ${{ vars.RELEASE_TRUST_JSON }} PROBE_RESULT: ${{ needs.probe.result == 'success' && (needs.historical-probe.result == 'success' || needs.historical-probe.result == 'skipped') && 'success' || 'failure' }} run: | run_url="https://github.com/$REPOSITORY/actions/runs/$RUN_ID" diff --git a/CHANGELOG.md b/CHANGELOG.md index 0ee9f868..99dc41b1 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,16 +6,22 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). -## [1.3.23] - -- Stage native compatibility verification for Plex 1.43.4.10903-e5521bd8c. Certification is pending. - ## [Unreleased] +## [1.3.23] - 2026-10-13 + ### Fixed - Clear old PostgreSQL errors after a cached query recovers, including queries returning no rows. - Keep compatibility status outside GitHub issues. - Run the short compatibility check without starting the full duration test. +- Reuse four native image builds across PostgreSQL versions and limit CI log size. +- Validate matching restart and duration results before publishing the exact tested files. + +### Tested +- Plex 1.43.4.10903-e5521bd8c with PostgreSQL 15 and 18, LinuxServer and PlexInc, on native AMD64 and ARM64. +- All eight configurations passed 100 restarts, 5 hours 50 minutes of playback load and live PostgreSQL recovery. +- Four native ZIP installation and migration tests passed. +- The macOS ARM64 build was checked separately for architecture, exported functions and download checksum. ## [1.3.22] - 2026-10-11 diff --git a/README.md b/README.md index 4ebac256..44bbd6a1 100644 --- a/README.md +++ b/README.md @@ -8,31 +8,15 @@ A small shim library that catches Plex SQLite calls and sends them to PostgreSQL ## Current validation status -**v1.3.22 is a regular release.** Linux AMD64/ARM64 and macOS ARM64 downloads -are available. PRs #31, #32 and #33 are merged. -All four native ZIP checks and all eight 100-restart lanes passed on `5dd3b51`. -The 5h50 load test completed, but the subsequent PostgreSQL restart check -returned HTTP 500. This recovery issue is being investigated. -Native PlexInc/arm64/PG18 passes genuine SQLite import, -normal restart, both search routes, export and restoration into vanilla Plex, -with watched state, server/media identity and source hashes preserved. The two -observed search HTTP 500 causes have regressions and PostgreSQL 15/18 result -parity. Browser playback and server-side transcoding remain unverified. - -The v1.3.22 Linux ZIP includes matching dependencies for both architectures, -fixing the incomplete AMD64 dependency set in v1.3.21. - -The acceptance matrix is **LinuxServer and PlexInc × native amd64 and arm64 × -PostgreSQL 15 and 18** (eight combinations), with 100 restart cycles per lane -and a separate 5-hour-50-minute soak using the same immutable image digest. -See [release gates and known gaps](docs/release-readiness.md) and -[recorded validation evidence](docs/stability-validation-2026-10-09.md). - -## Latest published release: [v1.3.22](https://github.com/cgnl/plex-postgresql/releases/tag/v1.3.22) - -Published on 11 October 2026. Includes runtime, migration, startup ordering and -compatibility automation fixes. Download the Linux/macOS bundles or use the -LinuxServer/PlexInc Docker images. See [the changelog](CHANGELOG.md). +**v1.3.23 is a regular release.** Linux AMD64/ARM64 and macOS ARM64 downloads are available, along with LinuxServer and PlexInc Docker images. + +All eight Linux configurations passed 100 restart cycles, 5 hours 50 minutes of movie and TV playback load with four simultaneous clients, and recovery after a live PostgreSQL restart. All four native ZIP installation and migration checks passed. The macOS ARM64 build was checked separately; the full playback matrix ran on Linux. + +The tested matrix is **LinuxServer and PlexInc × native AMD64 and ARM64 × PostgreSQL 15 and 18**, using Plex **1.43.4.10903-e5521bd8c**. The published Linux ZIP and Docker images are the exact tested files and images. See the [completed test run](https://github.com/cgnl/plex-postgresql/actions/runs/38313571637). + +## Latest published release: [v1.3.23](https://github.com/cgnl/plex-postgresql/releases/tag/v1.3.23) + +Published on 13 October 2026. Fixes stale database errors after PostgreSQL recovery and improves compatibility CI. See [the changelog](CHANGELOG.md). ### Runtime architecture introduced in v1.2.0 @@ -44,13 +28,12 @@ LinuxServer/PlexInc Docker images. See [the changelog](CHANGELOG.md). - 🔧 **Stack safety:** heap-allocated thread-local buffers for Plex's 544K worker thread stacks - ✅ **712 tests passing** across unit, integration, and compatibility suites -[📥 Download v1.3.22](https://github.com/cgnl/plex-postgresql/releases/tag/v1.3.22) | [📋 Full Changelog](CHANGELOG.md) +[📥 Download v1.3.23](https://github.com/cgnl/plex-postgresql/releases/tag/v1.3.23) | [📋 Full Changelog](CHANGELOG.md) New automatic releases require a certificate for the exact tested source and Linux ZIP/image bytes. Tag pushes no longer independently publish assets. -Automatic publication is disabled until genuine bootstrap certification and -reviewed trust configuration exist. New macOS assets require a separate native -acceptance gate. +Automatic Linux publication is configured and requires a valid certificate. +macOS downloads are built and checked separately. Pull requests and `main`/`develop` pushes run `.github/workflows/ci.yml` (script validation + Linux amd64 build check + full test suite + FFI header verification). The compatibility monitor checks public Plex releases every six hours and deduplicates unchanged inputs. Short probes detect regressions; full native diff --git a/scripts/plex_compatibility.py b/scripts/plex_compatibility.py index 8574f7cd..935845c7 100644 --- a/scripts/plex_compatibility.py +++ b/scripts/plex_compatibility.py @@ -526,12 +526,126 @@ def complete(repository, plan, run_id, result, run_url, category="insufficient-e return category +def release_tree(tree): + """Compare executable inputs exactly; only release prose may differ.""" + if tree.get("truncated") is not False: + raise ValueError("Incomplete release source tree") + ignored = {"VERSION", "CHANGELOG.md", "README.md"} + return {item["path"]: (item["mode"], item["type"], item["sha"]) + for item in tree["tree"] + if not (item["type"] == "blob" and item["mode"] == "100644" + and (item["path"] in ignored or (item["path"].startswith("docs/") + and item["path"].endswith(".md")))) + and item["type"] != "tree"} + + +def certificate_from_run(client, run): + from bootstrap_release_trust import read_certificate + run_id = run["id"] + artifacts = client.get(f"actions/runs/{run_id}/artifacts?per_page=100") + certificates = [item for item in artifacts["artifacts"] + if item["name"] == "release-certificate" and not item["expired"]] + if not certificates: + return None + if len(certificates) != 1: + raise ValueError("Published certificate missing or ambiguous") + artifact = certificates[0] + if artifact["size_in_bytes"] > 5 * 1024 * 1024: + raise ValueError("Certificate archive exceeds limit") + data = client.get(f'actions/artifacts/{artifact["id"]}/zip', raw=True) + if (artifact.get("digest") != "sha256:" + hashlib.sha256(data).hexdigest() + or artifact.get("workflow_run", {}).get("id") != run_id + or artifact["workflow_run"].get("head_sha") != run["head_sha"]): + raise ValueError("Published certificate provenance mismatch") + from datetime import datetime + created, started, completed = (datetime.fromisoformat(value.replace("Z", "+00:00")) + for value in (artifact["created_at"], run["run_started_at"], run["updated_at"])) + if not started <= created <= completed: + raise ValueError("Published certificate belongs to another attempt") + return read_certificate(data), artifacts + + +def published_certificate(client, trust, source_sha, tag): + """Inspect at most ten recent successful runs plus genuine bootstrap.""" + runs = client.get(f'actions/workflows/{trust["workflow_id"]}/runs?status=success&event=workflow_dispatch&per_page=10')["workflow_runs"] + seen = set() + for run in runs[:10]: + seen.add(run["id"]) + proof = certificate_from_run(client, run) + if proof is None: + continue + certificate, artifacts = proof + if certificate["candidate_source_sha"] == source_sha and "v" + certificate["project_version"] == tag: + return certificate, run, artifacts + bootstrap_id = trust["bootstrap_run_id"] + if bootstrap_id not in seen: + run = client.get(f"actions/runs/{bootstrap_id}") + proof = certificate_from_run(client, run) + if proof is None: + return None + certificate, artifacts = proof + if certificate["candidate_source_sha"] == source_sha and "v" + certificate["project_version"] == tag: + return certificate, run, artifacts + return None + + +def already_published(repository, plan): + """Suppress duplicate paperwork, without authorizing new release bytes.""" + raw = os.environ.get("RELEASE_TRUST_JSON", "") + if not raw: + return False + from load_release_trust import load + from publish_certified_release import GitHub, expected_coverage + from release_certificate import authenticate + client = GitHub(repository, os.environ["GH_TOKEN"]) + default = github_api(f"repos/{repository}")["default_branch"] + trust, _ = load(".github/release-trust.json", raw, repository, default) + release = client.get("releases/latest") + tag = release["tag_name"] + if (release.get("draft") is not False or release.get("prerelease") is not False + or re.fullmatch(r"v[0-9]+\.[0-9]+\.[0-9]+", tag) is None): + raise ValueError("Latest release is not regular") + ref = client.get("git/ref/tags/" + tag) + if ref["object"].get("type") != "commit": + raise ValueError("Published release tag is not an immutable source commit") + selected = published_certificate(client, trust, ref["object"]["sha"], tag) + if selected is None: + return False + certificate, run, artifacts = selected + source = client.get("commits/" + certificate["candidate_source_sha"]) + current = client.get("commits/" + plan["source_sha"]) + if release_tree(client.get("git/trees/" + source["commit"]["tree"]["sha"] + "?recursive=1")) != release_tree( + client.get("git/trees/" + current["commit"]["tree"]["sha"] + "?recursive=1")): + return False + if certificate["profile_fingerprints"] != [p["fingerprint"] for p in plan["profiles"]]: + return False + policy = client.get("contents/.github/plex-compatibility-policy.json?ref=" + plan["source_sha"]) + policy_hash = hashlib.sha256(base64.b64decode(policy["content"], validate=False)).hexdigest() + reviewed = {path: client.get(f'contents/{path}?ref={run["head_sha"]}')["sha"] + for path in trust["reviewed_file_blobs"]} + authenticate(certificate, trust, run, client.get(f'actions/workflows/{trust["workflow_id"]}'), + artifacts, source, reviewed, expected_coverage(plan["profiles"], policy_hash)) + tag = "v" + certificate["project_version"] + if (release.get("draft") is not False or release.get("prerelease") is not False + or ref["object"].get("type") != "commit" + or ref["object"].get("sha") != certificate["candidate_source_sha"]): + raise ValueError("Published regular release tag mismatch") + assets = {item["name"]: item.get("digest") for item in release["assets"]} + for bundle in certificate["artifacts"]: + for name, digest in bundle["files"].items(): + if assets.get(name) != "sha256:" + digest: + raise ValueError("Published certified asset missing or changed") + return True + + def dispatch_followup(repository, plan, category, run_url): default = github_api(f"repos/{repository}")["default_branch"] current = github_api(f"repos/{repository}/git/ref/heads/{default}")["object"]["sha"] if current != plan["source_sha"]: raise ValueError("Default source changed; restage or reobserve instead of dispatching stale work") if category == "probe-passed": + if already_published(repository, plan): + return "already-published" latest = plan["profiles"][0] inputs = {"merged_source_sha": plan["source_sha"], "plex_version": latest["plex_version"], "upstream_profile_json": json.dumps(latest, separators=(",", ":")), "profile_id": latest["id"]} @@ -547,6 +661,8 @@ def dispatch_followup(repository, plan, category, run_url): def report_pr(repository, plan, category, run_url): + if category == "probe-passed" and already_published(repository, plan): + return "already-published" body = report(plan, category, run_url) marker = "plex-compatibility-report:" + plan["fingerprint"] prs = github_api(f"repos/{repository}/pulls?state=all&per_page=100") diff --git a/scripts/tests/test_release_followup_dedupe.py b/scripts/tests/test_release_followup_dedupe.py new file mode 100644 index 00000000..b0ea6402 --- /dev/null +++ b/scripts/tests/test_release_followup_dedupe.py @@ -0,0 +1,132 @@ +import os +import sys +import unittest +from pathlib import Path +from unittest.mock import patch + +sys.path.insert(0, str(Path(__file__).resolve().parents[1])) +import plex_compatibility as compatibility + + +def tree(entries): + return {"truncated": False, "tree": [{"path": path, "mode": "100644", "type": "blob", "sha": sha} + for path, sha in entries.items()]} + + +class ReleaseDedupeTests(unittest.TestCase): + def test_release_prose_does_not_change_runtime(self): + old = tree({"VERSION": "a", "README.md": "a", "CHANGELOG.md": "a", "docs/status.md": "a", "rust/lib.rs": "a"}) + new = tree({"VERSION": "b", "README.md": "b", "CHANGELOG.md": "b", "docs/status.md": "b", "rust/lib.rs": "a"}) + self.assertEqual(compatibility.release_tree(old), compatibility.release_tree(new)) + + def test_executable_fixture_and_workflow_changes_are_not_ignored(self): + for path in ("rust/lib.rs", "scripts/plex_compatibility.py", ".github/workflows/check-updates.yml", "scripts/media-fixtures.json", "docs/fixture.json"): + with self.subTest(path=path): + self.assertNotEqual(compatibility.release_tree(tree({path: "a"})), + compatibility.release_tree(tree({path: "b"}))) + with self.assertRaises(ValueError): + compatibility.release_tree({"truncated": True, "tree": []}) + + def test_unconfigured_does_not_claim_published(self): + with patch.dict(os.environ, {"RELEASE_TRUST_JSON": ""}): + self.assertFalse(compatibility.already_published("owner/repo", {})) + + def test_published_probe_creates_neither_report_nor_dispatch(self): + plan = {"source_sha": "a" * 40} + with patch.object(compatibility, "already_published", return_value=True), patch.object( + compatibility, "github_api", side_effect=[{"default_branch": "main"}, {"object": {"sha": "a" * 40}}]) as api: + self.assertEqual(compatibility.report_pr("owner/repo", plan, "probe-passed", "url"), "already-published") + self.assertEqual(compatibility.dispatch_followup("owner/repo", plan, "probe-passed", "url"), "already-published") + self.assertEqual(api.call_count, 2) + + def test_unverifiable_proof_stops_instead_of_staging(self): + with patch.object(compatibility, "already_published", side_effect=ValueError("bad proof")): + with self.assertRaises(ValueError): + compatibility.report_pr("owner/repo", {}, "probe-passed", "url") + + +if __name__ == "__main__": + unittest.main() + +class PublishedProofTests(unittest.TestCase): + def setUp(self): + import base64 + import hashlib + import io + import json + import zipfile + import test_release_certificate as fixtures + from release_certificate import trusted_file_paths + from publish_certified_release import expected_coverage + fixture = fixtures.CertificateTests() + fixture.setUp() + self.run, self.certificate, self.trust = fixture.run, fixture.certificate, fixture.trust + self.trust['bootstrap_run_id'] = 1 + self.trust['reviewed_file_blobs'] = {path: '9' * 40 for path in trusted_file_paths} + self.trust['require_current_upstream'] = True + policy = b'{"schema_version":1}' + self.trust['policy_hash'] = self.certificate['policy_hash'] = hashlib.sha256(policy).hexdigest() + profiles = [{'id': 'latest', 'fingerprint': 'e' * 64}] + cells = expected_coverage(profiles, self.trust['policy_hash'])['expected_cells'] + self.certificate['expected_cells'] = self.certificate['observed_cells'] = cells + self.certificate['artifacts'][0]['files'] = {'bundle.zip': '2' * 64} + self.plan = {'source_sha': '8' * 40, 'profiles': profiles} + archive = io.BytesIO() + with zipfile.ZipFile(archive, 'w') as output: + output.writestr('certificate.json', json.dumps(self.certificate)) + data = archive.getvalue() + proof = {'id': 789, 'name': 'release-certificate', 'expired': False, + 'size_in_bytes': len(data), 'workflow_run': {'id': 123, 'head_sha': 'c' * 40}, + 'created_at': '2026-10-11T08:01:00Z', 'digest': 'sha256:' + hashlib.sha256(data).hexdigest()} + self.responses = { + 'releases/latest': {'tag_name': 'v1.3.23', 'draft': False, 'prerelease': False, + 'assets': [{'name': 'bundle.zip', 'digest': 'sha256:' + '2' * 64}]}, + 'git/ref/tags/v1.3.23': {'object': {'type': 'commit', 'sha': 'a' * 40}}, + 'actions/workflows/42/runs?status=success&event=workflow_dispatch&per_page=10': {'workflow_runs': [self.run]}, + 'actions/runs/123/artifacts?per_page=100': {'artifacts': fixture.artifacts['artifacts'] + [proof]}, + 'actions/artifacts/789/zip': data, + 'commits/' + 'a' * 40: fixture.source, + 'commits/' + '8' * 40: {'commit': {'tree': {'sha': '7' * 40}}}, + 'git/trees/' + 'b' * 40 + '?recursive=1': tree({'rust/lib.rs': 'a', 'VERSION': 'a'}), + 'git/trees/' + '7' * 40 + '?recursive=1': tree({'rust/lib.rs': 'a', 'VERSION': 'b'}), + 'contents/.github/plex-compatibility-policy.json?ref=' + '8' * 40: {'content': base64.b64encode(policy).decode()}, + 'actions/workflows/42': fixture.workflow, + } + self.calls = [] + + def check(self): + import json + def get(path, raw=False): + self.calls.append(path) + if path.startswith('contents/') and '?ref=' + 'c' * 40 in path: + return {'sha': '9' * 40} + return self.responses[path] + client = type('Client', (), {'get': staticmethod(get)})() + with patch.dict(os.environ, {'RELEASE_TRUST_JSON': json.dumps(self.trust), 'GH_TOKEN': 'test'}), patch( + 'publish_certified_release.GitHub', return_value=client), patch.object( + compatibility, 'github_api', return_value={'default_branch': 'main'}): + return compatibility.already_published('owner/repo', self.plan) + + def test_latest_published_certificate_differs_from_bootstrap(self): + self.assertTrue(self.check()) + self.assertNotIn('actions/runs/1', self.calls) + + def test_runtime_or_profile_change_requires_certification(self): + self.responses['git/trees/' + '7' * 40 + '?recursive=1'] = tree({'rust/lib.rs': 'b'}) + self.assertFalse(self.check()) + self.setUp() + self.plan['profiles'][0]['fingerprint'] = 'f' * 64 + self.assertFalse(self.check()) + + def test_invalid_archive_attempt_or_asset_is_visible(self): + for mutation in ('archive', 'attempt', 'asset'): + with self.subTest(mutation=mutation): + self.setUp() + if mutation == 'archive': + self.responses['actions/artifacts/789/zip'] = b'changed' + elif mutation == 'attempt': + self.responses['actions/runs/123/artifacts?per_page=100']['artifacts'][-1]['created_at'] = '2026-10-10T08:01:00Z' + else: + self.responses['releases/latest']['assets'][0]['digest'] = 'sha256:' + '3' * 64 + with self.assertRaises(ValueError): + self.check()