Fix Android compile: explicit Bionic imports + Bionic-strict pointer signatures - #380
Conversation
weichsel
left a comment
There was a problem hiding this comment.
Thanks for the PR - this is a solid starting point for broader platform support. There are a few build and lint issues, along with some behavioral bugs, that still need to be addressed. Also: What's the main motivation to support Bionic (vs. "vanilla" Android)?
The previous tmpfile() fallback for `funopen`/`fopencookie`-less platforms compiled but didn't behave: writes hit the temp file but were never synced back to MemoryFile.data, so reading the archive returned the original (now stale) bytes. It also silently turned an in-memory API into a disk-backed one — wrong contract. Per review feedback, gate the entire MemoryFile machinery (the class, the Archive(data:) initializer, makeBackingConfiguration(for:data:), and the writing-side memory branches) on `#if swift(>=5.0) && !os(Windows) && !os(Android)`. File-backed archives continue to work everywhere; the in-memory feature is simply unavailable on platforms without a userspace FILE-stream API (Windows MSVC has no equivalent; the Swift Android SDK doesn't expose Bionic's `cookie_io_functions_t`). Drops the now-redundant Bionic-specific fwrite empty-buffer guard in Archive+BackingConfiguration.swift since this code never compiles on Bionic any more. Refs: weichsel#380 (comment)
Modern Swift toolchains pick the version-specific manifest, so linker-setting changes in Package.swift had no effect there. Mirror the per-platform `linkedLibrary` mapping (`z` on Linux/Android, `zlib` on Windows) into the 5.9 manifest, and drop the trailing comma SwiftLint flagged in Package.swift. Refs: weichsel#380 (comment)
|
Pushed three follow-up commits addressing the review:
Bionic is Android's libc — every AOSP device, the NDK, and the swift.org Swift Android SDK go through it; there's no separate "vanilla Android" libc. The two module names in the shim ( |
|
Thanks for the follow up. There are still some issues that break the Linux build (and also the newly added platforms that are currently not addressed by our CI setup):
We should add a Android CI job to cover this. e.g. via: https://github.com/skiptools/swift-android-action
If I understand your comments and this Swift Forums post correctly, Android is a superset of Android’s C standard library and POSIX support. It also appears to be the modern way to interact with Android from Swift, so I’d prefer using it exclusively instead of Bionic.
Thanks for pointing this out: Fixed on |
On Android, `import Foundation` does not transitively re-export the C standard library, so files that touch raw POSIX symbols (`stat`, `lstat`, `S_IFMT`/`S_IFREG`/`S_IFDIR`/`S_IFLNK`, `mode_t`, `errno`, `fopen`/`fclose`/`fread`/`fwrite`/`fseeko`/`ftello`, `fileno`, `ftruncate`, `time_t`/`gmtime`/`timegm`, `timeval`, `timespec`, `fpos_t`, `funopen`, `FILE`) fail to compile with "cannot find … in scope". Add a `canImport(Android)` / `canImport(Bionic)` shim block to every ZIPFoundation source file that uses such symbols. The shim is inert on Apple platforms (Foundation already re-exports Darwin), Linux (Foundation re-exports Glibc), and Windows (Foundation re-exports ucrt) — it only binds extra symbols on Android. `setSymlinkPermissions` and `setSymlinkModificationDate` are now wrapped in the same Apple-only `#if` that already guards their call sites: Bionic ships neither `lchmod` nor `lutimes`, and Linux's `lchmod` is a no-op anyway because the kernel ignores symlink permission bits. Keeping the definitions Apple-only lets the rest of the library compile on Android without changing Apple behaviour. Test target import shims left for a follow-up — this PR only covers the library so downstream packages can build on Android.
Bionic imports `fwrite(const void*, size_t, size_t, FILE*)` with a non-optional `UnsafeRawPointer` first argument. Apple/Linux both bridge the parameter as IUO (`UnsafeRawPointer!`), so today `fwrite(buffer.baseAddress, …)` slips a possibly-nil pointer through. On Bionic the same code fails with "must be unwrapped to a value of type 'UnsafeRawPointer'". Adopt the existing `Data+Serialization.write(chunk:)` pattern: bind `baseAddress` and skip the call when the buffer is empty. Same behaviour on Apple/Linux (the EOCD record is never empty), but compiles on Android.
Apple imports `funopen` with IUO-typed `(UnsafeMutableRawPointer!, UnsafeMutablePointer<Int8>!, Int32) -> Int32` callbacks; the existing stubs with `Optional` parameters bridge fine. Bionic imports the same prototypes with non-Optional pointers, and Swift refuses to coerce `(UnsafeMutableRawPointer?, …)` into `@convention(c) (UnsafeMutableRawPointer, …)` — function pointer types are invariant. Branch the stubs three ways: - Apple → Optional pointer params (unchanged behaviour) - Android → non-Optional pointer params, no nil-guard (Bionic's funopen) - Linux/etc. → fopencookie path (unchanged) This lets Android compile the memory-archive backing without giving up the existing Apple semantics.
Bionic's `funopen` is `__INTRODUCED_IN(28)`, so projects targeting older Android API levels link with `undefined symbol: funopen`. `fopencookie` is available on Bionic since API 23 and matches the Linux/Glibc path the package already takes. Drop the now-redundant Android-funopen stub split — the Linux `fopencookie` stubs (Optional pointer params, `Int` count) work as-is on Bionic too, so let Android fall through to the existing `#else` branch.
clang on MSVC interprets `#import` as the Microsoft type-library directive (not Apple-style once-include), which rejects header file arguments with "#import of type library is an unsupported Microsoft feature". `#include` works identically on every other clang flavour. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CoreFoundation isn't available on Windows (and Android via NDK), so
the unconditional `import CoreFoundation` in Entry.swift errored with
"no such module 'CoreFoundation'". Guard the import via canImport,
and split the `codepage437` static let:
- canImport(CoreFoundation): use CFStringConvertEncodingToNSStringEncoding
as before (Apple + Linux's swift-corelibs-foundation).
- else: provide a raw-value sentinel so equality checks compile;
decoding falls through to the existing cp437Lookup table.
Extend the Linux fallback in `String.init(pathData:encoding:)` and
the `cp437Lookup` table itself to cover Windows + Android too.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds enough Windows-specific shims for ZIPFoundation to compile under
the MSVC-flavoured Swift toolchain. Source-level invasive changes were
kept to a minimum: the bulk of the work lives in a single new file
(`PosixCompat+Windows.swift`) that defines the missing typedefs and
function aliases, plus a tiny refactor of the off_t call sites to
funnel through a `zip_off_t` typealias.
Concretely:
- PosixCompat+Windows.swift: declares `mode_t`, `S_IFMT`, `S_IFREG`,
`S_IFDIR`, `S_IFLNK`, `suseconds_t`, `zip_off_t = Int64`, and
`@inlinable` wrappers `timegm` / `fseeko` / `ftruncate` that map
onto `_mkgmtime` / `_fseeki64` / `_chsize_s`. Linux + Apple unchanged
(still use the libc-provided `off_t`).
- All eight call sites that did `off_t(value)` now use
`zip_off_t(value)`, which expands to `off_t` on Linux/Apple and
`Int64` on Windows. Avoids silent truncation when WinSDK imports
`off_t` as 32-bit.
- `extension stat { var lastAccessDate }` and
`extension timeval { init(timeIntervalSince1970:) }` get gated to
non-Windows. Their only call sites (`setSymlinkModificationDate`)
were already Apple-gated, so no functional change.
- `Date.init(timespec:)` likewise gated to non-Windows.
- `FileManager+ZIP.permissionsForItem` /
`fileModificationDateTimeForItem` / `fileSizeForItem` get a Windows
arm that uses `FileManager.attributesOfItem` instead of `lstat`.
POSIX-permission inspection on Windows returns the type-default
permissions (no real POSIX bits to read).
- `FILEPointer` becomes `OpaquePointer` on Windows too (matches Bionic's
treatment — MSVC also imports `FILE` as opaque).
- `Archive.MemoryFile.open(mode:)` adds a Windows arm using `tmpfile()`
since neither `funopen` nor `fopencookie` exist there. In-memory
archives become disk-backed temp files on Windows; functional but
slower than the userspace FILE-stream APIs the other platforms use.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`zip_off_t = off_t` on non-Windows platforms needs `off_t` in scope. The `import Foundation` we relied on only existed inside the `#if os(Windows)` block, so on macOS / iOS / Linux the typealias couldn't resolve. Add explicit `import Darwin / Glibc / Musl / Android / Bionic` per host inside the non-Windows arm. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two follow-ups to a124cf2: - WinSDK imports `FILE` as `_iobuf` (typed struct), not opaque, so `_fseeki64` declares its first arg as `UnsafeMutablePointer<FILE>`. Match it in our `fseeko` shim and revert FILEPointer = OpaquePointer for Windows (only Bionic still imports FILE opaque). - `FileManager+ZIP.typeForItem(at:)` had a stray `lstat` call we missed in the bare-stat sweep. Add a Windows arm that uses `FileManager.fileExists(atPath:isDirectory:)` instead. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Companion to the fseeko shim. ZIPFoundation calls `ftello` in six places to record current archive offsets; without a shim Windows fails with "cannot find 'ftello' in scope". `_ftelli64` returns `__int64` directly, so the shim returns `Int64` — call sites already wrap with `Int64(ftello(...))` so this works on every platform. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The CZLib modulemap's `link "z"` directive emits `-lz` on every
non-Apple host. lld-link translates that into a search for `z.lib`,
which doesn't exist on Windows where vcpkg installs `zlib.lib`.
- Drop `link "z"` from `Sources/CZLib/module.modulemap`.
- Move the link-library decision to `Package.swift` via per-platform
`linkerSettings` on the `ZIPFoundation` target:
• Linux + Android → `-lz`
• Windows → `zlib`
This requires bumping `swift-tools-version` from 5.0 → 5.3 (first
release with `.when(platforms:)` on linkerSettings).
Apple platforms are unchanged — they take the `canImport(Compression)`
branch and never enter the CZLib path.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The Swift Android SDK doesn't expose `cookie_io_functions_t` through either the `Android` or `Bionic` modules even though Bionic ships `fopencookie`/`cookie_io_functions_t` natively. The Swift compiler just doesn't see the type, so the fopencookie call site fails to build with "cannot find 'cookie_io_functions_t' in scope". Extend the Windows `tmpfile()` arm to also cover Android. Same trade-off applies — costs a write+read round-trip on every operation, but the memory-archive code path compiles and works for the SwiftBash use case. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Bionic imports `fwrite`'s first arg as a non-Optional `UnsafeRawPointer`; `buffer.baseAddress` is `UnsafeRawPointer?`, which doesn't auto-bridge. Guard the unwrap (and the empty-buffer case) explicitly. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Hypothesis: explicit `linkedLibrary("z")` on Android trips the
swift-android-action's CRT recipe, resulting in `__libc_init` undefined
when executables (including the test runner) link. SwiftBash gates
its CZLib pin to exclude Android and links fine. Mirror that here.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This reverts commit 6d1e1ce.
The previous tmpfile() fallback for `funopen`/`fopencookie`-less platforms compiled but didn't behave: writes hit the temp file but were never synced back to MemoryFile.data, so reading the archive returned the original (now stale) bytes. It also silently turned an in-memory API into a disk-backed one — wrong contract. Per review feedback, gate the entire MemoryFile machinery (the class, the Archive(data:) initializer, makeBackingConfiguration(for:data:), and the writing-side memory branches) on `#if swift(>=5.0) && !os(Windows) && !os(Android)`. File-backed archives continue to work everywhere; the in-memory feature is simply unavailable on platforms without a userspace FILE-stream API (Windows MSVC has no equivalent; the Swift Android SDK doesn't expose Bionic's `cookie_io_functions_t`). Drops the now-redundant Bionic-specific fwrite empty-buffer guard in Archive+BackingConfiguration.swift since this code never compiles on Bionic any more. Refs: weichsel#380 (comment)
Modern Swift toolchains pick the version-specific manifest, so linker-setting changes in Package.swift had no effect there. Mirror the per-platform `linkedLibrary` mapping (`z` on Linux/Android, `zlib` on Windows) into the 5.9 manifest, and drop the trailing comma SwiftLint flagged in Package.swift. Refs: weichsel#380 (comment)
CI runs `swiftlint --strict` and our PR introduced four classes of violations: * `PosixCompat+Windows.swift` — POSIX-shaped names (`mode_t`, `S_IFMT`, `timegm`, …) intentionally match the libc spelling so call sites read identically across platforms; bracket the file with `swiftlint:disable identifier_name type_name` / `swiftlint:enable` and rename two short closure params (`tm`, `fd`) that aren't ABI-visible. Also fix the double-space colon spacing on the `S_IF*` constants. * `FileManager+ZIP.swift` was over the 400-line file_length cap after the Windows fallbacks were inlined per-method. Lift the Windows variants of `permissionsForItem(at:)`, `fileModificationDateTimeForItem(at:)`, `fileSizeForItem(at:)`, and `typeForItem(at:)` into a new `FileManager+ZIPWindows.swift` and gate the POSIX implementations with `#if !os(Windows)`. * `Entry.swift` was also over the cap; move the 256-entry `cp437Lookup` table out into a new `Entry+CodePage437.swift` (used only on Linux/Windows/Android, where there's no `CFStringEncoding`). * `Package.swift` had a trailing comma in the new `linkerSettings` array. Pure refactor — no behaviour change on any platform.
…pple `setSymlinkModificationDate` was tightened to Apple-only earlier in this PR (Bionic ships neither `lchmod` nor `lutimes`), so the first `XCTAssertPOSIXError(try fileManager.setSymlinkModificationDate(...))` in `testSymlinkModificationDateTransferErrorConditions` now fails to compile on Linux with "value of type 'FileManager' has no member 'setSymlinkModificationDate'". The test is already gated to Darwin at the runner-registration level (it's only listed in `darwinOnlyTests`), so just match that and wrap the entire function body with the same Apple-only `#if`. Compile-time only; no runtime behaviour change anywhere. Fixes the Linux (5.10.1) failure from CI run 25435022152. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
ebe5b5f to
4a86ab7
Compare
|
@weichsel I've fixed up the build, please approve workflow run so that we see if that makes everything green. About android, we can certainly add an Android CI target if you like. But I propose to do that as a a different PR, because it likely will require fixes outside the scope of this one. |
|
@weichsel ok, all green now |
|
Thanks for fixing builds on the established platforms. However, the issues I outlined above are not addressed for Android and Windows. In particular, the incomplete gating of the memory archive initializer currently prevents the library from building on those platforms, while the remaining issues still break the test targets. (Also, the redundant Bionic-based directives are still in) |
|
@odrobnik Thanks for the fixes. I cleaned up a bit and opened a PR (odrobnik#1). Please check if your use case still works after my changes. If so, we can merge. |
|
I have since replaced ZipFoundation with libarchive, so I cannot test it any more in my code. But I am confident that it would work. |
OK. Can you please merge: odrobnik#1 |
|
I merged it, but why in my fork? |
Thanks.
Didn't realize you had "Maintainer can modify" on. |
The full Windows CLI needs ZIPFoundation (DOCX/Pages readers) to compile, but the released 0.9.20 uses `#import <zlib.h>` in its CZLib shim, which Windows clang rejects as an MSVC COM-import feature. The fix (`#import` -> `#include`, weichsel/ZIPFoundation#380) is on ZIPFoundation's `development` branch but not in any release yet — so pin to that commit (187ee77). Verified locally on macOS: full build + all 443 tests pass, including the DOCX/Pages byte-parity round-trips. CI (Windows CLI job): also provision zlib via vcpkg alongside libxml2, and copy zlib.lib -> z.lib since ZIPFoundation's CZLib links `.linkedLibrary("z")`. The existing -Xcc/-Xlinker flags already cover zlib's header and lib dir. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Closes #46 (the Zip-container half; the PDF-deflate options in that issue are untouched). ZIPFoundation was pinned to an untagged `development` HEAD — the only commit carrying the Windows `#import` fix (weichsel/ZIPFoundation#380), which upstream still hasn't tagged. Replaced with marcprux/swift-archive 3.8.9, which vendors libarchive as C sources, so there is no system library to provision and we are back on a version tag. `GzipSupport` is enabled explicitly: it is off by default upstream, and without zlib libarchive cannot inflate DEFLATE entries. Reading (DOCX parts and media, the iWork `.iwa` container) streams each container once through `ArchiveReader`, since libarchive has no lookup by name. Writing `.docx`/`.epub` goes through the new SwiftTextZip target, which drives the raw `archive_*` C API rather than `ArchiveWriter`. `ArchiveEntry.apply()` always sets uid/gid/mtime, and libarchive then emits `ux`/`UT` extra blocks. Setting none of the three buys a bare `mimetype` header (OCF forbids an extra field there — it is what keeps `application/epub+zip` at offset 38) and reproducible output (libarchive derives DOS timestamps with `localtime()`, where ZIPFoundation used `gmtime()`). `.docx` becomes reproducible too; it was previously stamped with `Date()`. iWork writing stays on the hand-rolled `StoredZipWriter`: `.pages` needs sizes and CRC in the local header, which libarchive's always-streaming writer cannot emit. BREAKING: minimum deployment targets rise to macOS 13 / iOS 15 / tvOS 15 / watchOS 10 (swift-archive's own floor, applied package-wide by SwiftPM). The manifest also moves to `swift-tools-version:6.3` for any-of `.when(traits:)`. Also corrects AGENTS.md, which had been copied from SwiftLEGO and never adapted — it referenced a nonexistent Xcode project and an iOS 26 floor this package has never had. Verified: 496 tests; epubcheck reports 0 errors / 0 warnings; Python's zipfile `testzip()` passes; `textutil` parses the output `.docx`; output is byte-identical across timezones; all seven CI jobs green, including Windows, Linux and Android. Output is ~9% smaller than before.
Summary
Lets ZIPFoundation compile on the Swift Android SDK without changing behaviour on any other platform.
import Foundationdoes not transitively re-export Bionic the way it re-exports Darwin / Glibc / ucrt, so every file that touches a raw libc symbol fails on Android with errors like:Three changes:
Add an Android/Bionic import shim at the top of every source file that uses libc symbols (
stat,lstat,S_IFMT/S_IFREG/S_IFDIR/S_IFLNK,mode_t,fopen/fclose/fread/fwrite/fseeko/ftello,fileno,ftruncate,time_t/gmtime/timegm,timeval/timespec,fpos_t,funopen,FILE,errno):Inert on Apple (Darwin via Foundation), Linux (Glibc via Foundation), and Windows (ucrt via Foundation) — Android is the only platform that gains new bindings.
Wrap
setSymlinkPermissions/setSymlinkModificationDatein their existing Apple-only#if. Both are already only called on Apple (Linux'slchmodis a kernel no-op for symlinks; Bionic ships neitherlchmodnor a usefullutimesfor symlinks). Lifting the same#if os(macOS) || …over the definitions lets Android compile without changing Apple semantics.Fix two strict-signature mismatches that surface only on Bionic:
Archive+BackingConfiguration.swift:fwrite(buffer.baseAddress, …)— Bionic types the first parameter as non-optionalUnsafeRawPointer. Adopt the sameif let baseAddress, count > 0pattern already used inData+Serialization.write(chunk:).Archive+MemoryFile.swiftfunopencallbacks: Apple imports the prototypes with IUO callback parameters ((UnsafeMutableRawPointer!, UnsafeMutablePointer<Int8>!, Int32) -> Int32), Bionic with strictly non-optional pointers. Swift refuses to coerce one@convention(c)signature into the other (function pointer types are invariant), so split the existingfunopenbranch into separate Apple and Android sub-branches with the signatures each platform's libc actually expects. Thefopencookiebranch stays unchanged.Validation
Built locally on macOS (Xcode 26, Swift 6.3) — clean.
Validated on Android via the SwiftPorts CI matrix: with this fork pinned in place of upstream
weichsel/ZIPFoundation,swift buildon the swift.org Android SDK 6.3 (via skiptools/swift-android-action@v2) compiles ZIPFoundation cleanly. The remaining Android failures in that matrix are downstream (libgit2 macros not importing) — none are in ZIPFoundation.Linux + Windows continue to build green as a regression check.
Out of scope
Tests/ZIPFoundationTests/*also touch libc directly. Happy to send a follow-up if you want test-target Android coverage too; left out here so the diff stays focused on the library.