Build and release pipeline for the Linux binary dependencies - #1
Merged
Conversation
Ports the FFmpeg, DXVK Native and Steamworks.NET builds out of the Pulsar and Magnetar repos, so each library is built once here and published as a single versioned release archive both consumers download. The archive also carries the proprietary EOS and Steamworks runtimes (committed under Vendor/, no public source or fetchable binary) and the third-party licence texts, so attribution travels with the binaries. The PE-loader wrapper libraries stay in linux-native-wrappers: they change far more often, and re-bundling them here would put a full FFmpeg and DXVK rebuild in front of every wrapper fix. CI pins ubuntu-24.04 so the glibc the libraries link against cannot move under consumers when GitHub rolls the latest label forward.
DXVK Native's meson build refuses to configure without SDL3, SDL2 or GLFW, and ubuntu-24.04 has no libsdl3-dev package, so CI failed at that check. SDL3 is headers-only here: DXVK takes it as a partial_dependency and dlopens libSDL3.so.0 at runtime, so nothing SDL-related links into or ships with the archive. Building one pinned version means CI and local builds compile against identical headers instead of whatever the host happens to have. Verified neutral - DXVK built against the pinned 3.4.12 is byte-identical to DXVK built against a system SDL3 3.5.0.
SDL's cmake hard-errors when neither X11 nor Wayland development libraries are present, which is every clean CI runner. The check guards against accidentally building an SDL that cannot open a window - not a concern here, since nothing links or loads this SDL and DXVK only needs its headers. SDL_UNIX_CONSOLE_BUILD=ON skips it, avoiding a whole X11 and Wayland dev stack on the runner for a library that is then discarded. Verified neutral: DXVK built against the pinned, console-only 3.4.12 is byte-identical to DXVK built against a full system SDL3 3.5.0.
The sed ranges that print each script's header comment as --help were off by a few lines after the header edits, cutting the requirements paragraph mid-sentence in four of the five scripts. Also correct docs/consuming.md now that both consumers are implemented: they extract to different places (Pulsar straight into build/Libraries, Magnetar into a build/linux-deps cache it stages out of), and Magnetar's proprietary blobs were never committed - they were a manual per-developer step and a CI secret, which is the part this change actually removes.
build_ffmpeg.sh promised in a comment that it wiped previously staged FFmpeg files, but never did, and it skipped recreating a symlink that already existed. Bumping FFMPEG_VERSION on an existing tree therefore left the old libraries in place with libavcodec.so still pointing at them, so the archive would have shipped two FFmpeg builds with the unversioned aliases resolving to the stale one. build_dxvk.sh's cache check looked only for the bare .so names while build.sh asserts on the .so.0 aliases too, so a missing alias reported "cached, skipping rebuild" and then failed the caller's assertion. --only/--skip exited 1 even though the header documents the partial staging as expected, making every documented invocation fail. Unknown step names were also accepted silently, skipping every build. The workflow published releases on fork pull requests, where GITHUB_TOKEN is read-only, so external contributions would always go red; and the draft release had no --target, so publishing one would tag main rather than the commit the archive was built from.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Ports the Linux binary dependency build pipeline out of Pulsar (and the overlapping half out of Magnetar) into this repo, so each library is built once and published as a single release archive both consumers download.
What this adds
Scripts/build_ffmpeg.sh— FFmpeg 8.1, configured down to a glibc+zlib-only dependency set,DT_RUNPATH=$ORIGINpatched, SOVERSION andlddassertions intact.Scripts/build_dxvk.sh— DXVK Native 2.7.1 via upstream'spackage-native.sh.Scripts/build_steamworks_net.sh—Steamworks.NET.dllfrom the same pinned commit both repos already used.Vendor/— the proprietary EOS and Steamworks runtimes, moved here from Pulsar. They have no public source and no fetchable binary, so committing them is the only option;.gitignoregained a negation for them.Licenses/— third-party licence texts, moved here so attribution ships with the binaries it covers. The FFmpeg LGPL notice now points at this repo'sbuild_ffmpeg.sh, which is a licence obligation.build.sh— builds everything, stagesbuild/Libraries/, asserts the full expected-file list, packages a reproducibledist/linux-dependencies.tar.gz..github/workflows/build.yml— modelled on the provenlinux-native-wrappersworkflow: push tomainpublishesv1.0.<run_number>as latest, non-draft PRs get a draftpr-<n>release, draft PRs are skipped.docs/— architecture, per-dependency reference, building, the archive contract, consumption, and maintenance. Linked from the README.What this deliberately does not include
The PE-loader wrappers (
libD3DCompiler.so,libHavok.so,libRecastDetour.so,libVRageNative.so) stay inCometWorks/linux-native-wrappers. They change far more often than FFmpeg or DXVK, and re-bundling them here would put a ~15-minute rebuild in front of every wrapper fix. Consumers fetch both releases.Verification
./build.shruns green locally from a cold tree and produces a 23 MB archive. Extracting it round-trips correctly: symlink chains intact,DT_RUNPATH=$ORIGINpresent on every native.so, andlddon the FFmpeg libraries shows only glibc, libz and their siblings.The artefacts match what Pulsar builds today — identical file sizes, the same pinned Steamworks commit embedded in
AssemblyInformationalVersion, the same FFmpeg version string. The remaining hash differences are embedded build paths and the assembly MVID.Follow-up
Pulsar and Magnetar get rewired to fetch this release once it is published; their now-redundant build scripts,
Vendor/andScripts/Licenses/come out at that point.