diff --git a/.github/workflows/macadi-host.yml b/.github/workflows/macadi-host.yml new file mode 100644 index 00000000..fccb0121 --- /dev/null +++ b/.github/workflows/macadi-host.yml @@ -0,0 +1,55 @@ +name: Emulated Mac (host) + +# The emulated Mac's native core, built for the host and run against Apple's real macOS CoreADI. +# +# Builds app/src/main/cpp/macadi/ exactly as the app does (MacAdi.cmake is shared), fetches the +# library with the same range request the app makes, checks both pinned hashes, and has the +# emulator load it and answer "not provisioned". Nothing is sent to Apple but that one download: +# provisioning, which would register a machine, is not run. + +on: + pull_request: + branches: [ "main" ] + paths: + - "app/src/main/cpp/macadi/**" + - "app/src/test/macadi-host/**" + - "app/src/main/java/dev/wander/android/opentagviewer/anisette/MacCoreAdi*.java" + - "scripts/locate_mac_coreadi.py" + - ".github/workflows/macadi-host.yml" + push: + branches: [ "main" ] + paths: + - "app/src/main/cpp/macadi/**" + - "app/src/test/macadi-host/**" + - "app/src/main/java/dev/wander/android/opentagviewer/anisette/MacCoreAdi*.java" + - "scripts/locate_mac_coreadi.py" + - ".github/workflows/macadi-host.yml" + workflow_dispatch: + +permissions: + contents: read + +jobs: + host: + runs-on: ubuntu-24.04 + steps: + - uses: actions/checkout@v4 + + - uses: actions/setup-python@v5 + with: + python-version: "3.12" + + - name: Fetch Apple's macOS CoreADI the way the app does, and check both hashes + run: python scripts/locate_mac_coreadi.py --remote --out "$RUNNER_TEMP/CoreADI.x86_64" + + - name: Build the native core for the host + run: | + cmake -S app/src/test/macadi-host -B "$RUNNER_TEMP/macadi" -DCMAKE_BUILD_TYPE=Release + cmake --build "$RUNNER_TEMP/macadi" --parallel + + - name: Load the library and ask whether it is provisioned + run: | + out=$("$RUNNER_TEMP/macadi/macadi_cli" "$RUNNER_TEMP/CoreADI.x86_64" 979a0 /nonexistent \ + 00112233445566778899aabbccddeeff -1 --load-only) + echo "$out" + test "$out" = '{"loaded": true, "provisioned": false}' diff --git a/AGENTS.md b/AGENTS.md index 5fe25e58..c01ebd81 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -88,7 +88,8 @@ python -m venv .venv && .venv/bin/pip install "FindMy==" - **FindMy's own local provider still does not work.** `anisette` needs `unicorn`, a CPU emulator Chaquopy cannot build for Android. `app/stubs/unicorn/` is a stand-in that makes the dependency tree resolve; every method raises. The stub is not a step toward enabling - `aniLocal`, and never will be. + `aniLocal`, and never will be. Unicorn *does* run in the app, but as native code built by + CMake for the emulated Mac (rule 21), never as the Python package. - **The app does run ADI locally, by a different route.** `app/src/main/java/.../anisette/` loads Apple's real Android ADI libraries in-process and produces Anisette here, falling back to a remote server when it cannot. That is unrelated to `aniLocal` and needs no @@ -694,6 +695,49 @@ already sent you, and then reasoning from a guess while you wait. had a phone number blacked out by hand, and the pixels either side of it were not. Quote the error text, never the surrounding screen, and never re-upload the image. +### 21. Anisette comes from an emulated Mac, and it is one machine for the life of the install + +Since 2026-10-12 Apple issues the Find My token (`searchPartyToken`) only to a sign-in whose +Anisette machine was minted by a Mac or an iPhone; an Android-minted one gets `mmeAuthToken` alone +and every report fetch answers 401 ([malmeloo/FindMy.py#278](https://github.com/malmeloo/FindMy.py/issues/278)). +From about 2026-10-20 the edge also answers 503 to any `X-MMe-Client-Info` whose OS token is +`Android`, whatever else is sent ([#261](https://github.com/parawanderer/OpenTagViewer/pull/261)). +So `LocalAnisette`'s Android libraries can no longer reach Find My by any combination of headers. + +`EmulatedMacAnisette` runs Apple's **macOS** CoreADI - the x86_64 build in iTunes 12.8.3 - under +Unicorn, in `libmacadi.so` (`app/src/main/cpp/macadi/`, ported from sacca97/os-findmy-e; see its +`NOTICE.md`). It is what `AppDependencies` hands out. + +- **The library is fetched from Apple, never shipped.** One HTTP range request: in that exact disk + image the package payload is stored uncompressed and contiguous, so its offset is a constant. + `scripts/locate_mac_coreadi.py` re-derives and checks every constant, reading them from the Java + sources. Both the payload and the library are pinned by SHA-256, and the dispatcher offset + (`0x979A0`) is only valid for that one build. +- **One identity, kept forever** - rule 11, with more at stake. `files/anisette/mac/identity.json` + and `store/` belong together: the library hashes the host UUID into the machine id, Apple binds + the provisioning to the device id, and the Find My token only works alongside Anisette from the + machine that obtained it. Apple will provision a device id **once** (a second attempt answers + HTTP 433 / -29002), so a lost store means a new machine and a new device-list entry. +- **Never leave `adi.pb` in the store.** Provisioning writes the legacy whole-machine blob beside + the per-user one, and while it exists the library says "not provisioned". A process that + provisioned and then asked for headers sent Apple an empty machine id, which surfaces as + **`ec -80009`, "MID is invalid"** - and looked like an emulation fault until a restart, which + never reloads the file, made it go away. `CoreAdi::beginSession` drops it before every session. +- **A Mac's device id is not a UUID**, and FindMy.py's CloudKit client parses it as one. The + keychain and CloudKit clients get `icloud_bridge._forCloudKit`'s view of the account instead, + which answers `device_uuid` with the host UUID and forwards everything else. Anything new that + hands the account to CloudKit has to go through it. +- **Never freeze the emulated clock outside a byte-for-byte comparison.** A constant OTP is + accepted for about a minute and then refused. +- **Grand Slam now asks for a second factor with `hsc` 409 and no `au`**, which the pinned + FindMy.py reads as "no 2FA needed". `main._treatHsc409AsSecondFactor` fills `au` in. +- **The APK carries GPL-2.0 code** (Unicorn). Keep it in `libmacadi.so`, not in + `libopentagviewer.so`. + +The emulator suite cannot cover any of this - it swaps the Anisette source for a fake. What does is +`macadi-host.yml` (the library loads in the ported emulator), the JVM tests on the fetcher, the +Python tests in `test_emulated_mac.py`, and a real sign-in on a phone. + --- ## Building and testing diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 5f6c3407..9a705faa 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -646,6 +646,26 @@ static constructor running. No provisioning and nothing sent to Apple beyond dow libraries. See [`app/src/test/adi-on-bionic/README.md`](app/src/test/adi-on-bionic/README.md) for how it works and what it measured. +### The emulated Mac, on the host + +```bash +python scripts/locate_mac_coreadi.py --remote --out /tmp/CoreADI.x86_64 +cmake -S app/src/test/macadi-host -B build/macadi && cmake --build build/macadi --parallel +build/macadi/macadi_cli /tmp/CoreADI.x86_64 979a0 /nonexistent 00112233445566778899aabbccddeeff -1 --load-only +``` + +Since 2026-10-12 Apple gives the Find My token only to a sign-in whose Anisette machine was +minted by a Mac or an iPhone, so the app runs Apple's **macOS** CoreADI under an x86_64 emulator +(`app/src/main/cpp/macadi/`, see `EmulatedMacAnisette`). This builds the same native core for the +machine you are on, through the same `MacAdi.cmake`, and loads Apple's library into it. The first +line downloads 3.5 MB from Apple with exactly the range request the app makes and checks it +against the pinned hashes; `--dmg iTunes12.8.3.dmg` does the same against a full copy of the +installer and shows which of its chunks the range spans. + +Given a provisioned store directory and its host UUID instead of `--load-only`, `macadi_cli` +prints one header set. Pass a fixed Unix time as the fifth argument to compare its output byte +for byte against another implementation; never use a fixed clock for anything sent to Apple. + ### Which Python each tree targets There are three, and they are not the same: @@ -886,6 +906,7 @@ run regardless: each AES entry carries a fresh random salt. | `native-stubs.yml` | `app/src/main/cpp/**`, `app/src/test/cpp/**` changes | Builds the stand-ins for Apple's two stubbed libraries on x86_64 and arm64 Linux and checks what they export and return. Seconds, no Android | | `adi-on-bionic.yml` | `app/src/main/cpp/**`, `adi-libraries.json`, `AdiFunction.java`, its own directory | Loads and initialises Apple's real ADI libraries on x86_64 and arm64 Linux under Android 9 and Android 14 bionic, with no emulator, and checks the pre-#232 stub still fails. About a minute cold, seconds warm | | `check-gsa-edge.yml` | daily, and every PR to `main` | Asks Apple's edge whether it still lets the app's and the exporter's sign-in and provisioning requests through, with their exact headers and no account. A scheduled failure opens an issue. Seconds | +| `macadi-host.yml` | `app/src/main/cpp/macadi/**`, `app/src/test/macadi-host/**`, `MacCoreAdi*.java`, `scripts/locate_mac_coreadi.py` | Fetches Apple's macOS CoreADI with the app's own range request, checks both pinned hashes, builds the emulated Mac's native core for the host and has it load the library. Nothing sent to Apple beyond the download. A few minutes, mostly building Unicorn | | `waiting-on-reporter.yml` | a comment on an issue, daily | Removes the `waiting on reporter` label when the issue's author replies. Daily, comments once on any labelled issue quiet for 14 days, and again only after somebody else has commented. Never closes anything. Both triggers run the copy on `main`, so a PR cannot exercise it; `workflow_dispatch` defaults to a dry run that prints what it would post | The instrumented job needs KVM on the runner; the workflow enables it first. It runs the same diff --git a/app/src/main/cpp/CMakeLists.txt b/app/src/main/cpp/CMakeLists.txt index 66fc6d35..88461c32 100644 --- a/app/src/main/cpp/CMakeLists.txt +++ b/app/src/main/cpp/CMakeLists.txt @@ -42,3 +42,20 @@ include("${CMAKE_CURRENT_SOURCE_DIR}/stubs/AppleStubs.cmake") add_apple_stub(CoreFoundation) add_apple_stub(mediaplatform) + +# -------------------------------------------------------------------------------------------- +# The emulated Mac. +# +# Since 2026-10-12 Apple gives the Find My token only to a Mac or iPhone identity, and a machine +# provisioned by the Android libraries above cannot present as one. So this runs Apple's *macOS* +# CoreADI instead, under an x86_64 emulator - see macadi/MacAdi.cmake and MacCoreAdi.java. +# +# Its own library, not part of libopentagviewer.so: the emulator is GPL-2.0 and several MB, and +# keeping it separate means it is only loaded when the emulated Mac is actually used. +# -------------------------------------------------------------------------------------------- + +include("${CMAKE_CURRENT_SOURCE_DIR}/macadi/MacAdi.cmake") + +add_library(macadi SHARED macadi/jni.cpp) +target_compile_options(macadi PRIVATE -fexceptions) +target_link_libraries(macadi PRIVATE macadi_core log) diff --git a/app/src/main/cpp/macadi/MacAdi.cmake b/app/src/main/cpp/macadi/MacAdi.cmake new file mode 100644 index 00000000..e8880ad6 --- /dev/null +++ b/app/src/main/cpp/macadi/MacAdi.cmake @@ -0,0 +1,40 @@ +# The emulated Mac's Anisette core: Apple's macOS CoreADI, x86_64, run under Unicorn. +# +# Defines the static library `macadi_core`. Included by app/src/main/cpp/CMakeLists.txt for the +# app, and by app/src/test/macadi-host/ for the host check - so both build exactly these sources +# against exactly this Unicorn, rather than two copies of how they are built. +# +# **Unicorn is GPL-2.0.** It is fetched at build time, pinned by hash, and linked only into +# libmacadi.so, never into libopentagviewer.so. An APK built with this carries GPL-2.0 code. +# +# Apple's library is never part of the build or the repository. The app downloads it from +# Apple at runtime - see MacCoreAdiFetcher. + +include_guard(GLOBAL) +include(FetchContent) + +# Only the x86 target: CoreADI is x86_64 code, and every other architecture is megabytes of +# QEMU that would never run. +set(UNICORN_ARCH "x86" CACHE STRING "Enabled unicorn architectures" FORCE) +set(UNICORN_BUILD_TESTS OFF CACHE BOOL "" FORCE) +set(UNICORN_INSTALL OFF CACHE BOOL "" FORCE) +set(UNICORN_LEGACY_STATIC_ARCHIVE OFF CACHE BOOL "" FORCE) + +FetchContent_Declare( + unicorn + URL https://github.com/unicorn-engine/unicorn/archive/refs/tags/2.1.4.tar.gz + URL_HASH SHA256=ea8863f095a0136388694e5a6063afd9bb7650e30243dd6251af59c5ce5601f4) +FetchContent_MakeAvailable(unicorn) + +add_library(macadi_core STATIC + ${CMAKE_CURRENT_LIST_DIR}/macho.cpp + ${CMAKE_CURRENT_LIST_DIR}/vm.cpp + ${CMAKE_CURRENT_LIST_DIR}/core_adi.cpp) +target_include_directories(macadi_core PUBLIC + ${CMAKE_CURRENT_LIST_DIR} + ${unicorn_SOURCE_DIR}/include) +target_compile_features(macadi_core PUBLIC cxx_std_17) +# Errors are C++ exceptions up to the JNI boundary, where they become Java ones. +target_compile_options(macadi_core PRIVATE -fexceptions) +set_target_properties(macadi_core PROPERTIES POSITION_INDEPENDENT_CODE ON) +target_link_libraries(macadi_core PUBLIC unicorn) diff --git a/app/src/main/cpp/macadi/NOTICE.md b/app/src/main/cpp/macadi/NOTICE.md new file mode 100644 index 00000000..935ebfd4 --- /dev/null +++ b/app/src/main/cpp/macadi/NOTICE.md @@ -0,0 +1,43 @@ +# Third-party notices for the emulated Mac + +## sacca97/os-findmy-e + +`macho.*`, `vm.*` and `core_adi.*` are a C++ port of +[sacca97/os-findmy-e](https://github.com/sacca97/os-findmy-e) `native/adi-core/src/{macho,vm,adi}.rs`, +and the provisioning headers in `MacProvisioning.java` and the identity shapes in `MacIdentity.java` +follow its `tools/machoadi/{gsa,mac_anisette}.py`. Its licence: + +``` +MIT License + +Copyright (c) 2026 sacca + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. +``` + +## Unicorn + +[Unicorn](https://github.com/unicorn-engine/unicorn) 2.1.4 is fetched from source at build time +(`MacAdi.cmake`, pinned by SHA-256) and linked into `libmacadi.so`. It is licensed under +**GPL-2.0**, so an APK containing `libmacadi.so` contains GPL-2.0 code. + +## Apple's CoreADI + +Not part of this repository or of any build. The app downloads it at runtime from Apple's own +server - see `MacCoreAdiFetcher.java` and `scripts/locate_mac_coreadi.py`. diff --git a/app/src/main/cpp/macadi/core_adi.cpp b/app/src/main/cpp/macadi/core_adi.cpp new file mode 100644 index 00000000..755cd86b --- /dev/null +++ b/app/src/main/cpp/macadi/core_adi.cpp @@ -0,0 +1,210 @@ +#include "core_adi.h" + +#include + +#include "error.h" + +namespace macadi { +namespace { + +constexpr uint32_t INITIALIZE = 0x12DB31C5; +constexpr uint32_t IS_MACHINE_PROVISIONED = 0xB0EDA7AF; +constexpr uint32_t PROVISIONING_START = 0x716BD86C; +constexpr uint32_t PROVISIONING_END = 0x0B2B3196; +constexpr uint32_t SET_IDMS_ROUTING = 0x632B8D6E; +constexpr uint32_t GET_IDMS_ROUTING = 0x85FE63B0; +constexpr uint32_t SYNCHRONIZE = 0xA54951EC; +constexpr uint32_t REQUEST_OTP = 0xCFE0B46A; +constexpr uint32_t STORAGE_DISPOSE = 0x3E58E7F9; + +void be32(std::vector &m, uint32_t v) { + for (int s = 24; s >= 0; s -= 8) m.push_back(static_cast(v >> s)); +} + +void be64(std::vector &m, uint64_t v) { + for (int s = 56; s >= 0; s -= 8) m.push_back(static_cast(v >> s)); +} + +std::vector dsIdMessage(int64_t dsId) { + std::vector m; + be64(m, static_cast(dsId)); + return m; +} + +const std::string LEGACY_STORE = "adi.pb"; + +} // namespace + +CoreAdi::CoreAdi(const std::vector &image, uint64_t dispatch) + : vm_(image), dispatch_(dispatch) {} + +void CoreAdi::check(const char *op, Reply r) { + if (r.rc < 0 || r.ret < 0) { + throw Error(std::string("adi: ") + op + " failed: " + + std::to_string(r.ret < 0 ? r.ret : r.rc)); + } +} + +CoreAdi::Reply CoreAdi::call(uint32_t code, const std::vector &message) { + std::vector buf(std::max(0x800, 8 + message.size()), 0); + buf[3] = 1; // encryption type: clear + buf[7] = 1; + std::copy(message.begin(), message.end(), buf.begin() + 8); + const uint64_t bufAddr = vm_.scratch(buf.size()); + vm_.write(bufAddr, buf); + + const uint64_t params = vm_.scratch(0x20); + vm_.write(params, std::vector(0x20, 0)); + vm_.writeU64(params, bufAddr); + vm_.writeU32(params + 0x08, static_cast(8 + message.size())); + vm_.writeU32(params + 0x0C, 4); + vm_.writeU32(params + 0x10, 0); + + const int32_t rc = static_cast(static_cast(vm_.invoke(dispatch_, {code, params}))); + const std::vector head = vm_.read(bufAddr, 4); + const int32_t ret = static_cast((static_cast(head[0]) << 24) | + (static_cast(head[1]) << 16) | + (static_cast(head[2]) << 8) | + static_cast(head[3])); + vm_.write(bufAddr + 4, std::vector(4, 0)); + return Reply{rc, ret}; +} + +uint64_t CoreAdi::cell(uint64_t v) { + const uint64_t a = vm_.scratch(8); + vm_.writeU64(a, v); + return a; +} + +uint64_t CoreAdi::input(const std::vector &data) { + const uint64_t a = vm_.scratch(std::max(data.size(), 16)); + vm_.write(a, data); + return a; +} + +std::vector CoreAdi::slice(uint64_t ptr, uint64_t len) { + const uint64_t p = vm_.readU64(ptr); + const uint32_t n = vm_.readU32(len); + if (p == 0 || n == 0) return {}; + return vm_.read(p, n); +} + +void CoreAdi::beginSession() { + // Provisioning writes the legacy whole-machine blob as well as the per-user one, and while it + // is present the library reports the machine unprovisioned (see storePut). A process that + // restarts never sees it, because storePut refuses it; one that provisions and then asks for + // headers in the same process would, and sent Apple an empty machine id (ec -80009). So it + // goes before every session, not only on load. + vm_.files().erase(std::string(STORE_DIR) + "/" + LEGACY_STORE); + vm_.resetSession(); + initialize(); +} + +CoreAdi::Reply CoreAdi::initialize() { + vm_.scratchReset(); + return call(INITIALIZE, {}); +} + +CoreAdi::Reply CoreAdi::isMachineProvisioned(int64_t dsId) { + vm_.scratchReset(); + return call(IS_MACHINE_PROVISIONED, dsIdMessage(dsId)); +} + +CoreAdi::Reply CoreAdi::storageDispose(uint64_t ptr) { + std::vector m; + be64(m, ptr); + return call(STORAGE_DISPOSE, m); +} + +CoreAdi::Reply CoreAdi::setRouting(int64_t dsId, uint64_t routing) { + vm_.scratchReset(); + std::vector m = dsIdMessage(dsId); + be64(m, routing); + return call(SET_IDMS_ROUTING, m); +} + +void CoreAdi::storePut(const std::string &name, const std::vector &data) { + // The legacy whole-machine blob is never restored: whenever it is present, the library + // reports the machine unprovisioned without reading the per-user one (os-findmy-e, + // mac_anisette.py). Restoring it would make every later session look unprovisioned. + if (name == LEGACY_STORE) return; + vm_.files()[std::string(STORE_DIR) + "/" + name] = data; +} + +std::vector>> CoreAdi::storeFiles() { + const std::string prefix = std::string(STORE_DIR) + "/"; + std::vector>> out; + for (const auto &f : vm_.files()) { + if (f.first.rfind(prefix, 0) == 0) out.emplace_back(f.first.substr(prefix.size()), f.second); + } + return out; +} + +Headers CoreAdi::headers(int64_t dsId) { + beginSession(); + if (isMachineProvisioned(dsId).ret != 0) throw NotProvisioned(); + + vm_.scratchReset(); + const uint64_t midP = cell(0), midL = cell(0), otpP = cell(0), otpL = cell(0); + std::vector m = dsIdMessage(dsId); + for (const uint64_t v : {midP, midL, otpP, otpL}) be64(m, v); + const Reply otpReply = call(REQUEST_OTP, m); + Headers h{slice(midP, midL), slice(otpP, otpL), 0}; + // The library hands back pointers into its own storage; release them. + if (!h.mid.empty() && !h.otp.empty()) storageDispose(vm_.readU64(midP)); + check("RequestOTP", otpReply); + + vm_.scratchReset(); + const uint64_t routingCell = cell(0); + std::vector r = dsIdMessage(dsId); + be64(r, routingCell); + check("GetIDMSRouting", call(GET_IDMS_ROUTING, r)); + h.routing = vm_.readU64(routingCell); + return h; +} + +std::pair, uint32_t> CoreAdi::provisioningBegin( + int64_t dsId, const std::vector &spim) { + beginSession(); + if (isMachineProvisioned(dsId).ret == 0) throw AlreadyProvisioned(); + + vm_.scratchReset(); + const uint64_t spimBuf = input(spim); + const uint64_t cpimP = cell(0), cpimL = cell(0), session = cell(0); + std::vector m = dsIdMessage(dsId); + be64(m, spimBuf); + be32(m, static_cast(spim.size())); + for (const uint64_t v : {cpimP, cpimL, session}) be64(m, v); + check("ProvisioningStart", call(PROVISIONING_START, m)); + return {slice(cpimP, cpimL), vm_.readU32(session)}; +} + +void CoreAdi::provisioningFinish(int64_t dsId, uint32_t session, const std::vector &ptm, + const std::vector &tk, uint64_t routing) { + vm_.scratchReset(); + const uint64_t ptmBuf = input(ptm), tkBuf = input(tk); + std::vector m; + be32(m, session); + be64(m, ptmBuf); + be32(m, static_cast(ptm.size())); + be64(m, tkBuf); + be32(m, static_cast(tk.size())); + check("ProvisioningEnd", call(PROVISIONING_END, m)); + check("SetIDMSRouting", setRouting(dsId, routing)); +} + +std::pair, std::vector> CoreAdi::sync( + int64_t dsId, const std::vector &sim) { + beginSession(); + vm_.scratchReset(); + const uint64_t simBuf = input(sim); + const uint64_t midP = cell(0), midL = cell(0), srmP = cell(0), srmL = cell(0); + std::vector m = dsIdMessage(dsId); + be64(m, simBuf); + be32(m, static_cast(sim.size())); + for (const uint64_t v : {midP, midL, srmP, srmL}) be64(m, v); + check("Synchronize", call(SYNCHRONIZE, m)); + return {slice(midP, midL), slice(srmP, srmL)}; +} + +} // namespace macadi diff --git a/app/src/main/cpp/macadi/core_adi.h b/app/src/main/cpp/macadi/core_adi.h new file mode 100644 index 00000000..88618ac6 --- /dev/null +++ b/app/src/main/cpp/macadi/core_adi.h @@ -0,0 +1,80 @@ +// The CoreADI dispatcher, `_vdfut768ig(function_code, CoreADIParameters *)`, and the handful of +// operations Anisette needs from it. +// +// Requests are big-endian; pointers inside a message are 8-byte big-endian addresses *inside the +// emulated process*. After Dadoum/apple-crates adi/src/{core_adi,calls}.rs, by way of +// sacca97/os-findmy-e native/adi-core/src/adi.rs (MIT), which this ports. +// +// **One high-level call is one session.** Each starts by putting the VM back to its post-load +// state and calling Initialize, the way a fresh process would. Nothing accumulates between calls, +// which is what lets a bump allocator stand in for malloc. The one exception is provisioning: +// provisioningBegin and provisioningFinish must share a session, with nothing in between. + +#pragma once + +#include +#include +#include +#include + +#include "vm.h" + +namespace macadi { + +struct Headers { + std::vector mid; // X-Apple-I-MD-M, raw + std::vector otp; // X-Apple-I-MD, raw + uint64_t routing; // X-Apple-I-MD-RINFO +}; + +class CoreAdi { +public: + // `dispatch` is the offset of _vdfut768ig in this build. Java knows it, by the build's sha256, + // and refuses any file it does not recognise before this is ever called. + CoreAdi(const std::vector &image, uint64_t dispatch); + + void setHostUuid(const uint8_t uuid[16]) { vm_.setHostUuid(uuid); } + void setFrozenTime(int64_t secs) { vm_.setFrozenTime(secs); } + + // The store, as (file name, bytes). Persist it after every call that can change it - which is + // every call, since RequestOTP rewrites it. + void storePut(const std::string &name, const std::vector &data); + std::vector>> storeFiles(); + + // One session: what a machine puts on one request. + Headers headers(int64_t dsId); + + // Start provisioning with Apple's spim; returns (cpim, session). + std::pair, uint32_t> provisioningBegin(int64_t dsId, + const std::vector &spim); + // Finish with what Apple returned for the cpim. Same session as provisioningBegin. + void provisioningFinish(int64_t dsId, uint32_t session, const std::vector &ptm, + const std::vector &tk, uint64_t routing); + + // Answer Apple's resync request (a 434 carrying X-Apple-I-MD-DATA): returns (mid, srm). + std::pair, std::vector> sync(int64_t dsId, + const std::vector &sim); + +private: + struct Reply { + int32_t rc; // the dispatcher's status + int32_t ret; // the operation's own, negative on failure + }; + + void beginSession(); + Reply call(uint32_t code, const std::vector &message); + uint64_t cell(uint64_t v); + uint64_t input(const std::vector &data); + std::vector slice(uint64_t ptr, uint64_t len); + static void check(const char *op, Reply r); + + Reply initialize(); + Reply isMachineProvisioned(int64_t dsId); + Reply storageDispose(uint64_t ptr); + Reply setRouting(int64_t dsId, uint64_t routing); + + Vm vm_; + uint64_t dispatch_; +}; + +} // namespace macadi diff --git a/app/src/main/cpp/macadi/error.h b/app/src/main/cpp/macadi/error.h new file mode 100644 index 00000000..e9b70ed5 --- /dev/null +++ b/app/src/main/cpp/macadi/error.h @@ -0,0 +1,28 @@ +#pragma once + +#include +#include + +namespace macadi { + +// Everything this library can fail with. The message is what reaches Java, and through it the +// log the user is asked to attach, so it says which layer failed: the Mach-O, the emulation, or +// Apple's library answering with an error of its own. +class Error : public std::runtime_error { +public: + explicit Error(const std::string &message) : std::runtime_error(message) {} +}; + +// The library is not provisioned yet. Distinct because it is the one failure that has a remedy: +// provisioning. +class NotProvisioned : public Error { +public: + NotProvisioned() : Error("the machine is not provisioned") {} +}; + +class AlreadyProvisioned : public Error { +public: + AlreadyProvisioned() : Error("the machine is already provisioned") {} +}; + +} // namespace macadi diff --git a/app/src/main/cpp/macadi/jni.cpp b/app/src/main/cpp/macadi/jni.cpp new file mode 100644 index 00000000..754aa7ce --- /dev/null +++ b/app/src/main/cpp/macadi/jni.cpp @@ -0,0 +1,172 @@ +// JNI face of the emulated Mac: dev.wander.android.opentagviewer.anisette.MacCoreAdi. +// +// A handle is a CoreAdi* passed to Java as a long. Java owns its lifetime and uses it from one +// thread at a time, under its own lock. Every native failure becomes an IllegalStateException +// carrying the C++ message, except "not provisioned", which has its own type because it is the +// one failure with a remedy. + +#include + +#include +#include +#include + +#include "core_adi.h" +#include "error.h" + +namespace { + +constexpr const char *STATE_EXCEPTION = "java/lang/IllegalStateException"; +constexpr const char *NOT_PROVISIONED = + "dev/wander/android/opentagviewer/anisette/MacCoreAdi$NotProvisionedException"; + +macadi::CoreAdi *adi(jlong handle) { return reinterpret_cast(handle); } + +std::vector bytes(JNIEnv *env, jbyteArray array) { + if (array == nullptr) return {}; + const jsize n = env->GetArrayLength(array); + std::vector out(static_cast(n)); + if (n > 0) env->GetByteArrayRegion(array, 0, n, reinterpret_cast(out.data())); + return out; +} + +jbyteArray array(JNIEnv *env, const std::vector &data) { + jbyteArray out = env->NewByteArray(static_cast(data.size())); + if (out != nullptr && !data.empty()) { + env->SetByteArrayRegion(out, 0, static_cast(data.size()), + reinterpret_cast(data.data())); + } + return out; +} + +void raise(JNIEnv *env, const char *type, const char *message) { + jclass cls = env->FindClass(type); + if (cls != nullptr) env->ThrowNew(cls, message); +} + +// Runs `body`, turning any C++ exception into a Java one. Nothing may unwind past a JNI frame. +template +T guarded(JNIEnv *env, T onError, F body) { + try { + return body(); + } catch (const macadi::NotProvisioned &e) { + raise(env, NOT_PROVISIONED, e.what()); + } catch (const std::exception &e) { + raise(env, STATE_EXCEPTION, e.what()); + } catch (...) { + raise(env, STATE_EXCEPTION, "unknown native failure"); + } + return onError; +} + +} // namespace + +extern "C" { + +JNIEXPORT jlong JNICALL +Java_dev_wander_android_opentagviewer_anisette_MacCoreAdi_nativeOpen( + JNIEnv *env, jclass, jbyteArray image, jlong dispatch) { + return guarded(env, 0, [&]() { + auto opened = std::make_unique(bytes(env, image), + static_cast(dispatch)); + return reinterpret_cast(opened.release()); + }); +} + +JNIEXPORT void JNICALL +Java_dev_wander_android_opentagviewer_anisette_MacCoreAdi_nativeClose(JNIEnv *, jclass, + jlong handle) { + delete adi(handle); +} + +JNIEXPORT void JNICALL +Java_dev_wander_android_opentagviewer_anisette_MacCoreAdi_nativeSetHostUuid( + JNIEnv *env, jclass, jlong handle, jbyteArray uuid) { + guarded(env, 0, [&]() { + const std::vector raw = bytes(env, uuid); + if (raw.size() != 16) throw macadi::Error("a host UUID is 16 bytes"); + adi(handle)->setHostUuid(raw.data()); + return 0; + }); +} + +JNIEXPORT void JNICALL +Java_dev_wander_android_opentagviewer_anisette_MacCoreAdi_nativeSetFrozenTime( + JNIEnv *, jclass, jlong handle, jlong secs) { + adi(handle)->setFrozenTime(secs); +} + +JNIEXPORT void JNICALL +Java_dev_wander_android_opentagviewer_anisette_MacCoreAdi_nativeStorePut( + JNIEnv *env, jclass, jlong handle, jstring name, jbyteArray data) { + guarded(env, 0, [&]() { + const char *chars = env->GetStringUTFChars(name, nullptr); + const std::string n = chars == nullptr ? std::string() : chars; + if (chars != nullptr) env->ReleaseStringUTFChars(name, chars); + adi(handle)->storePut(n, bytes(env, data)); + return 0; + }); +} + +// The store as alternating {name bytes (UTF-8), contents} entries. +JNIEXPORT jobjectArray JNICALL +Java_dev_wander_android_opentagviewer_anisette_MacCoreAdi_nativeStoreFiles(JNIEnv *env, jclass, + jlong handle) { + return guarded(env, nullptr, [&]() -> jobjectArray { + const auto files = adi(handle)->storeFiles(); + jclass bytesClass = env->FindClass("[B"); + jobjectArray out = env->NewObjectArray(static_cast(files.size() * 2), bytesClass, nullptr); + if (out == nullptr) return nullptr; + for (size_t i = 0; i < files.size(); i++) { + const std::vector name(files[i].first.begin(), files[i].first.end()); + jbyteArray n = array(env, name); + jbyteArray c = array(env, files[i].second); + env->SetObjectArrayElement(out, static_cast(i * 2), n); + env->SetObjectArrayElement(out, static_cast(i * 2 + 1), c); + env->DeleteLocalRef(n); + env->DeleteLocalRef(c); + } + return out; + }); +} + +// {mid, otp}; routing goes into routingOut[0]. +JNIEXPORT jobjectArray JNICALL +Java_dev_wander_android_opentagviewer_anisette_MacCoreAdi_nativeHeaders( + JNIEnv *env, jclass, jlong handle, jlong dsId, jlongArray routingOut) { + return guarded(env, nullptr, [&]() -> jobjectArray { + const macadi::Headers h = adi(handle)->headers(dsId); + const jlong routing = static_cast(h.routing); + env->SetLongArrayRegion(routingOut, 0, 1, &routing); + jobjectArray out = env->NewObjectArray(2, env->FindClass("[B"), nullptr); + if (out == nullptr) return nullptr; + env->SetObjectArrayElement(out, 0, array(env, h.mid)); + env->SetObjectArrayElement(out, 1, array(env, h.otp)); + return out; + }); +} + +// The cpim; the session goes into sessionOut[0]. +JNIEXPORT jbyteArray JNICALL +Java_dev_wander_android_opentagviewer_anisette_MacCoreAdi_nativeProvisioningBegin( + JNIEnv *env, jclass, jlong handle, jlong dsId, jbyteArray spim, jintArray sessionOut) { + return guarded(env, nullptr, [&]() { + const auto started = adi(handle)->provisioningBegin(dsId, bytes(env, spim)); + const jint session = static_cast(started.second); + env->SetIntArrayRegion(sessionOut, 0, 1, &session); + return array(env, started.first); + }); +} + +JNIEXPORT void JNICALL +Java_dev_wander_android_opentagviewer_anisette_MacCoreAdi_nativeProvisioningFinish( + JNIEnv *env, jclass, jlong handle, jlong dsId, jint session, jbyteArray ptm, + jbyteArray tk, jlong routing) { + guarded(env, 0, [&]() { + adi(handle)->provisioningFinish(dsId, static_cast(session), bytes(env, ptm), + bytes(env, tk), static_cast(routing)); + return 0; + }); +} + +} // extern "C" diff --git a/app/src/main/cpp/macadi/macho.cpp b/app/src/main/cpp/macadi/macho.cpp new file mode 100644 index 00000000..cd401773 --- /dev/null +++ b/app/src/main/cpp/macadi/macho.cpp @@ -0,0 +1,178 @@ +#include "macho.h" + +#include +#include + +#include "error.h" + +namespace macadi { +namespace { + +constexpr uint32_t MH_MAGIC_64 = 0xFEEDFACF; +constexpr uint32_t LC_SEGMENT_64 = 0x19; +constexpr uint32_t LC_DYLD_INFO = 0x22; +constexpr uint32_t LC_DYLD_INFO_ONLY = 0x80000022; + +uint32_t u32At(const std::vector &d, size_t o) { + if (o + 4 > d.size()) throw Error("mach-o: truncated"); + uint32_t v; + std::memcpy(&v, d.data() + o, 4); + return v; // little-endian host on every ABI this is built for +} + +uint64_t u64At(const std::vector &d, size_t o) { + if (o + 8 > d.size()) throw Error("mach-o: truncated"); + uint64_t v; + std::memcpy(&v, d.data() + o, 8); + return v; +} + +uint64_t uleb(const std::vector &d, size_t &o) { + uint64_t v = 0; + unsigned shift = 0; + for (;;) { + if (o >= d.size()) throw Error("mach-o: truncated leb"); + const uint8_t b = d[o++]; + if (shift < 64) v |= static_cast(b & 0x7F) << shift; + shift += 7; + if ((b & 0x80) == 0) return v; + } +} + +int64_t sleb(const std::vector &d, size_t &o) { + int64_t v = 0; + unsigned shift = 0; + for (;;) { + if (o >= d.size()) throw Error("mach-o: truncated leb"); + const uint8_t b = d[o++]; + if (shift < 64) v |= static_cast(b & 0x7F) << shift; + shift += 7; + if ((b & 0x80) == 0) { + if (shift < 64 && (b & 0x40) != 0) v |= -(static_cast(1) << shift); + return v; + } + } +} + +} // namespace + +MachO MachO::parse(const std::vector &d) { + if (u32At(d, 0) != MH_MAGIC_64) throw Error("mach-o: not a thin 64-bit Mach-O"); + const uint32_t ncmds = u32At(d, 16); + + MachO m; + size_t bindOff = 0, bindLen = 0, lazyOff = 0, lazyLen = 0; + size_t off = 32; + for (uint32_t i = 0; i < ncmds; i++) { + const uint32_t cmd = u32At(d, off); + const uint32_t size = u32At(d, off + 4); + if (cmd == LC_SEGMENT_64) { + m.segments_.push_back(Segment{ + u64At(d, off + 24), u64At(d, off + 32), u64At(d, off + 40), + u64At(d, off + 48), (u32At(d, off + 60) & 2) != 0}); + } else if (cmd == LC_DYLD_INFO || cmd == LC_DYLD_INFO_ONLY) { + bindOff = u32At(d, off + 16); + bindLen = u32At(d, off + 20); + lazyOff = u32At(d, off + 32); + lazyLen = u32At(d, off + 36); + } + if (size == 0) throw Error("mach-o: zero-sized load command"); + off += size; + } + + std::vector binds = m.readBinds(d, bindOff, bindLen, false); + std::vector lazy = m.readBinds(d, lazyOff, lazyLen, true); + binds.insert(binds.end(), lazy.begin(), lazy.end()); + m.binds_ = std::move(binds); + return m; +} + +std::vector MachO::readBinds(const std::vector &d, size_t start, size_t len, + bool lazy) const { + std::vector out; + if (len == 0) return out; + if (start + len > d.size()) throw Error("mach-o: bind stream outside the file"); + + const size_t end = start + len; + size_t o = start; + std::string symbol; + int64_t addend = 0; + size_t seg = 0; + uint64_t segOff = 0; + + auto push = [&]() { + if (seg >= segments_.size()) throw Error("mach-o: bad bind segment"); + out.push_back(Bind{segments_[seg].vmaddr + segOff, symbol, addend}); + }; + + while (o < end) { + const uint8_t byte = d[o++]; + const uint8_t op = byte & 0xF0; + const uint64_t imm = byte & 0x0F; + switch (op) { + case 0x00: // DONE - but lazy streams carry one per entry, so keep going there + if (!lazy) return out; + break; + case 0x10: // SET_DYLIB_ORDINAL_IMM + case 0x30: // SET_DYLIB_SPECIAL_IMM + case 0x50: // SET_TYPE_IMM + break; + case 0x20: // SET_DYLIB_ORDINAL_ULEB + uleb(d, o); + break; + case 0x40: { // SET_SYMBOL_TRAILING_FLAGS_IMM + const void *nul = std::memchr(d.data() + o, 0, end - o); + if (nul == nullptr) throw Error("mach-o: bad symbol"); + const size_t z = static_cast(nul) - (d.data() + o); + symbol.assign(reinterpret_cast(d.data() + o), z); + o += z + 1; + break; + } + case 0x60: // SET_ADDEND_SLEB + addend = sleb(d, o); + break; + case 0x70: // SET_SEGMENT_AND_OFFSET_ULEB + seg = static_cast(imm); + segOff = uleb(d, o); + break; + case 0x80: // ADD_ADDR_ULEB + segOff += uleb(d, o); + break; + case 0x90: // DO_BIND + push(); + segOff += 8; + break; + case 0xA0: // DO_BIND_ADD_ADDR_ULEB + push(); + segOff += uleb(d, o) + 8; + break; + case 0xB0: // DO_BIND_ADD_ADDR_IMM_SCALED + push(); + segOff += imm * 8 + 8; + break; + case 0xC0: { // DO_BIND_ULEB_TIMES_SKIPPING_ULEB + const uint64_t count = uleb(d, o); + const uint64_t skip = uleb(d, o); + for (uint64_t i = 0; i < count; i++) { + push(); + segOff += skip + 8; + } + break; + } + default: + throw Error("mach-o: unsupported bind opcode " + std::to_string(op)); + } + } + return out; +} + +std::vector MachO::bindSymbols() const { + std::vector out; + std::unordered_set seen; + for (const Bind &b : binds_) { + if (seen.insert(b.symbol).second) out.push_back(b.symbol); + } + return out; +} + +} // namespace macadi diff --git a/app/src/main/cpp/macadi/macho.h b/app/src/main/cpp/macadi/macho.h new file mode 100644 index 00000000..818bcf78 --- /dev/null +++ b/app/src/main/cpp/macadi/macho.h @@ -0,0 +1,50 @@ +// A minimal reader for the one Mach-O this library loads: Apple's CoreADI, x86_64 slice, as it +// ships in iTunes 12.8.3 for Mac. Segments to map and the dyld bind stream to stub - nothing else. +// +// Classic LC_DYLD_INFO only. The builds this is for predate chained fixups, and a build that used +// them would be refused by the sha256 check in Java long before it got here. + +#pragma once + +#include +#include +#include + +namespace macadi { + +struct Segment { + uint64_t vmaddr; + uint64_t vmsize; + uint64_t fileoff; + uint64_t filesize; + bool writable; +}; + +struct Bind { + uint64_t address; + std::string symbol; + int64_t addend; +}; + +class MachO { +public: + // Throws macadi::Error on anything that is not a thin little-endian 64-bit Mach-O with a + // bind stream this reader understands. + static MachO parse(const std::vector &image); + + const std::vector &segments() const { return segments_; } + const std::vector &binds() const { return binds_; } + + // Distinct imported symbols, in first-seen order: binds, then lazy binds. The index of a symbol + // in this list is what places its stub, so the order is part of the memory layout. + std::vector bindSymbols() const; + +private: + std::vector readBinds(const std::vector &d, size_t start, size_t len, + bool lazy) const; + + std::vector segments_; + std::vector binds_; +}; + +} // namespace macadi diff --git a/app/src/main/cpp/macadi/vm.cpp b/app/src/main/cpp/macadi/vm.cpp new file mode 100644 index 00000000..e53c2e44 --- /dev/null +++ b/app/src/main/cpp/macadi/vm.cpp @@ -0,0 +1,663 @@ +#include "vm.h" + +#include +#include +#include + +#include "error.h" +#include "macho.h" + +namespace macadi { + +const char *const STORE_DIR = "/Users/Shared/adi"; + +namespace { + +constexpr uint64_t PAGE = 0x1000; +constexpr uint64_t RETURN_ADDRESS = 0x40000000; +constexpr uint64_t HEAP_ADDRESS = 0x40010000; +constexpr uint64_t HEAP_SIZE = 0x01000000; +constexpr uint64_t STACK_ADDRESS = 0x50000000; +constexpr uint64_t STACK_SIZE = 0x00200000; +constexpr uint64_t STUB_ADDRESS = 0x60000000; +constexpr uint64_t STUB_SIZE = 0x20000; +constexpr uint64_t DATA_ADDRESS = 0x60020000; +constexpr uint64_t DATA_SIZE = 0x20000; +constexpr uint64_t SCRATCH_ADDRESS = 0x60040000; +constexpr uint64_t SCRATCH_SIZE = 0x40000; +constexpr uint64_t OBJECT_ADDRESS = 0x60080000; +constexpr uint64_t OBJECT_SIZE = 0x40000; +constexpr uint64_t SLOT = 0x10; + +constexpr size_t STAT_SIZE = 144; +constexpr size_t STATFS_SIZE = 2168; +constexpr uint16_t S_IFDIR_ = 0040000; +constexpr uint16_t S_IFREG_ = 0100000; +constexpr uint64_t O_CREAT_ = 0x0200; // Darwin's values, not Linux's +constexpr uint64_t O_TRUNC_ = 0x0400; +constexpr uint64_t ENOENT_ = 2; +constexpr uint64_t EBADF_ = 9; +constexpr uint64_t EEXIST_ = 17; +constexpr uint64_t EISDIR_ = 21; +constexpr uint64_t FAILED = ~static_cast(0); + +const char *const DATA_SYMBOLS[] = { + "stack_chk_guard", "kCFAllocatorDefault", "kCFBooleanFalse", + "kCFBooleanTrue", "kCFErrorDomainOSStatus", "kCFURLFileSecurityKey", + "kCFURLIsDirectoryKey", "kCFURLIsHiddenKey", "kCFURLTypeIdentifierKey", + "kUTTypePlainText", +}; + +const int ARG_REGS[] = {UC_X86_REG_RDI, UC_X86_REG_RSI, UC_X86_REG_RDX, + UC_X86_REG_RCX, UC_X86_REG_R8, UC_X86_REG_R9}; + +std::string norm(const std::string &s) { + const size_t i = s.find_first_not_of('_'); + return i == std::string::npos ? std::string() : s.substr(i); +} + +bool isDataSymbol(const std::string &normalised) { + for (const char *d : DATA_SYMBOLS) { + if (normalised == d) return true; + } + return false; +} + +// CRC-32, as the Python harness does (zlib.crc32), so inodes are stable per path. +uint64_t inode(const std::string &path) { + uint32_t crc = ~0u; + for (const unsigned char b : path) { + crc ^= b; + for (int i = 0; i < 8; i++) crc = (crc >> 1) ^ (0xEDB88320u & (0u - (crc & 1u))); + } + return 0x10000 + ((~crc) & 0xFFFFF); +} + +void put16(uint8_t *b, uint16_t v) { std::memcpy(b, &v, 2); } +void put32(uint8_t *b, uint32_t v) { std::memcpy(b, &v, 4); } +void put64(uint8_t *b, uint64_t v) { std::memcpy(b, &v, 8); } + +} // namespace + +bool Vm::handlerFor(const std::string &n, Handler &out) { + auto set = [&](Kind k, uint64_t v = 0) { + out = Handler{k, v}; + return true; + }; + if (n == "stack_chk_fail") return set(Kind::Abort); + if (n == "pthread_rwlock_init" || n == "pthread_rwlock_rdlock" || + n == "pthread_rwlock_wrlock" || n == "pthread_rwlock_unlock" || + n == "pthread_rwlock_destroy") + return set(Kind::Ret, 0); + if (n == "getuid") return set(Kind::Ret, 501); + if (n == "umask") return set(Kind::Ret, 0022); + if (n == "arc4random") return set(Kind::Ret, 0x12345678); + if (n == "arc4random_buf") return set(Kind::Arc4Buf); + if (n == "gethostuuid") return set(Kind::HostUuid); + if (n == "gettimeofday") return set(Kind::TimeOfDay); + if (n == "error") return set(Kind::Errno); // ___error, i.e. errno's address + if (n == "objc_msgSend") return set(Kind::ObjcMsgSend); + if (n == "NSVersionOfRunTimeLibrary") return set(Kind::Ret, 0x100000000ull); + if (n == "CFErrorGetCode" || n == "CFErrorGetDomain" || n == "CFRelease" || + n == "CSBackupIsItemExcluded" || n == "notify_post") + return set(Kind::Ret, 0); + if (n == "CSBackupSetItemExcluded" || n == "CFFileSecuritySetMode" || + n == "CFURLSetResourcePropertyForKey") + return set(Kind::Ret, 1); + if (n == "malloc") return set(Kind::Malloc); + if (n == "free" || n == "fchmod" || n == "fcntl" || n == "lockf") return set(Kind::Ret, 0); + if (n == "open") return set(Kind::Open); + if (n == "close") return set(Kind::Close); + if (n == "read") return set(Kind::Read); + if (n == "write") return set(Kind::Write); + if (n == "lstat$INODE64" || n == "stat$INODE64") return set(Kind::Lstat); + if (n == "fstat$INODE64") return set(Kind::Fstat); + if (n == "statfs$INODE64") return set(Kind::Statfs); + if (n == "mkdir") return set(Kind::Mkdir); + if (n == "unlink") return set(Kind::Unlink); + if (n == "chmod") return set(Kind::Chmod); + if (n == "ftruncate") return set(Kind::Ftruncate); + if (n == "getenv") return set(Kind::Getenv); + if (n == "CFURLCreateFromFileSystemRepresentation") return set(Kind::CfurlCreate); + if (n == "CFURLCopyResourcePropertyForKey") return set(Kind::CfurlCopyProp); + return false; +} + +void Vm::check(uc_err err, const char *what) const { + if (err != UC_ERR_OK) { + throw Error(std::string("unicorn: ") + what + ": " + uc_strerror(err)); + } +} + +Vm::Vm(const std::vector &image) { + const MachO macho = MachO::parse(image); + check(uc_open(UC_ARCH_X86, UC_MODE_64, &uc_), "open"); + + try { + const std::pair regions[] = { + {RETURN_ADDRESS, PAGE}, {HEAP_ADDRESS, HEAP_SIZE}, + {STACK_ADDRESS, STACK_SIZE}, {STUB_ADDRESS, STUB_SIZE}, + {DATA_ADDRESS, DATA_SIZE}, {SCRATCH_ADDRESS, SCRATCH_SIZE}, + {OBJECT_ADDRESS, OBJECT_SIZE}, + }; + for (const auto &r : regions) check(uc_mem_map(uc_, r.first, r.second, UC_PROT_ALL), "map"); + + for (const Segment &seg : macho.segments()) { + const uint64_t size = (seg.vmsize + PAGE - 1) / PAGE * PAGE; + if (size == 0) continue; + check(uc_mem_map(uc_, seg.vmaddr, size, UC_PROT_ALL), "map segment"); + if (seg.filesize != 0) { + if (seg.fileoff + seg.filesize > image.size()) { + throw Error("mach-o: segment outside the file"); + } + check(uc_mem_write(uc_, seg.vmaddr, image.data() + seg.fileoff, seg.filesize), + "load segment"); + } + } + const std::vector rets(STUB_SIZE, 0xC3); + check(uc_mem_write(uc_, STUB_ADDRESS, rets.data(), rets.size()), "write stubs"); + + std::unordered_map stubAddr; + std::unordered_map dataCell; + const std::vector symbols = macho.bindSymbols(); + for (size_t i = 0; i < symbols.size(); i++) { + const std::string &sym = symbols[i]; + const std::string n = norm(sym); + if (isDataSymbol(n)) { + const uint64_t cell = DATA_ADDRESS + i * SLOT; + const uint64_t obj = OBJECT_ADDRESS + i * SLOT; + writeU64(cell, obj); + dataCell[sym] = cell; + objectOf_[n] = obj; + keyByObject_[obj] = sym; + } else { + const uint64_t addr = STUB_ADDRESS + i * SLOT; + stubAddr[sym] = addr; + Handler h{}; + if (handlerFor(n, h)) handlers_[addr] = h; + } + } + for (const Bind &b : macho.binds()) { + uint64_t target; + if (auto it = dataCell.find(b.symbol); it != dataCell.end()) { + target = it->second; + } else if (auto st = stubAddr.find(b.symbol); st != stubAddr.end()) { + target = st->second; + } else { + throw Error("mach-o: unbound symbol " + b.symbol); + } + writeU64(b.address, target + static_cast(b.addend)); + } + + heapCur_ = HEAP_ADDRESS; + scratchCur_ = SCRATCH_ADDRESS; + objCur_ = OBJECT_ADDRESS; + nextUrl_ = OBJECT_ADDRESS; + nextFd_ = 3; + const uint8_t defaultUuid[16] = {0x00, 0x11, 0x22, 0x33, 0x44, 0x55, 0x66, 0x77, + 0x88, 0x99, 0xAA, 0xBB, 0xCC, 0xDD, 0xEE, 0xFF}; + std::memcpy(hostUuid_, defaultUuid, 16); + dirs_ = {"/", "/Users", "/Users/Shared", STORE_DIR}; + + // Long-lived cells (errno, getenv strings) live below the per-call scratch base. + errnoAddr_ = bumpScratch(8); + const std::pair env[] = { + {"HOME", "/Users/aditest"}, {"USER", "aditest"}, {"TMPDIR", "/tmp/"}}; + std::vector> cells; + for (const auto &kv : env) { + const std::string v = kv.second; + const uint64_t a = bumpScratch(v.size() + 1); + getenvAddr_[kv.first] = a; + cells.emplace_back(a, v); + } + scratchBase_ = scratchCur_; + for (const auto &c : cells) { + std::vector bytes(c.second.begin(), c.second.end()); + bytes.push_back(0); + write(c.first, bytes); + } + + check(uc_hook_add(uc_, &stubHook_, UC_HOOK_CODE, reinterpret_cast(&Vm::onStub), + this, STUB_ADDRESS, STUB_ADDRESS + STUB_SIZE - 1), + "hook stubs"); + check(uc_hook_add(uc_, &unmappedHook_, UC_HOOK_MEM_UNMAPPED, + reinterpret_cast(&Vm::onUnmapped), this, 1, 0), + "hook unmapped"); + + for (const Segment &seg : macho.segments()) { + if (seg.writable && seg.vmsize != 0) { + snapshot_.segments.emplace_back(seg.vmaddr, read(seg.vmaddr, seg.vmsize)); + } + } + snapshot_.cells = read(DATA_ADDRESS, PAGE); + snapshot_.heapCur = heapCur_; + snapshot_.objCur = objCur_; + snapshot_.nextUrl = nextUrl_; + snapshot_.nextFd = nextFd_; + zeros_.assign(STACK_SIZE, 0); + } catch (...) { + uc_close(uc_); + uc_ = nullptr; + throw; + } +} + +Vm::~Vm() { + if (uc_ != nullptr) uc_close(uc_); +} + +void Vm::onStub(uc_engine *uc, uint64_t address, uint32_t, void *self) { + Vm *vm = static_cast(self); + const auto it = vm->handlers_.find(address); + if (it == vm->handlers_.end()) return; // an import nothing needed answering: just `ret` + // Nothing may throw past this frame: it is called from inside Unicorn's C code. + try { + uint64_t result = 0; + if (vm->call(it->second, result)) { + uc_reg_write(uc, UC_X86_REG_RAX, &result); + } + } catch (...) { + vm->fault_ = "a stand-in for an imported function failed"; + uc_emu_stop(uc); + } +} + +bool Vm::onUnmapped(uc_engine *uc, uc_mem_type type, uint64_t address, int size, int64_t, + void *self) { + Vm *vm = static_cast(self); + uint64_t rip = 0; + uc_reg_read(uc, UC_X86_REG_RIP, &rip); + const char *kind = type == UC_MEM_READ_UNMAPPED ? "read" + : type == UC_MEM_WRITE_UNMAPPED ? "write" + : "fetch"; + char message[160]; + std::snprintf(message, sizeof message, "%s-unmapped at %#llx (size %d) rip=%#llx", kind, + static_cast(address), size, + static_cast(rip)); + vm->fault_ = message; + return false; +} + +uint64_t Vm::arg(int i) const { + uint64_t v = 0; + uc_reg_read(uc_, ARG_REGS[i], &v); + return v; +} + +std::string Vm::cstr(uint64_t addr) const { + std::string out; + while (out.size() < 4096) { + uint8_t b = 0; + if (uc_mem_read(uc_, addr + out.size(), &b, 1) != UC_ERR_OK || b == 0) break; + out.push_back(static_cast(b)); + } + return out; +} + +uint64_t Vm::fail(uint64_t err) { + poke32(errnoAddr_, static_cast(err)); + return FAILED; +} + +// Writes made on the library's behalf from inside a hook. Failure is ignored, as in the reference: +// the library supplied the address, and an unmapped one faults on its own at the next access. +void Vm::poke32(uint64_t addr, uint32_t v) { uc_mem_write(uc_, addr, &v, 4); } + +void Vm::poke64(uint64_t addr, uint64_t v) { uc_mem_write(uc_, addr, &v, 8); } + +void Vm::fillStat(uint64_t addr, const std::string &path, bool isDir, int64_t size) { + uint8_t b[STAT_SIZE] = {}; + put32(b + 0, 0x1000003); + put16(b + 4, static_cast((isDir ? S_IFDIR_ : S_IFREG_) | 0755)); + put16(b + 6, isDir ? 2 : 1); + put64(b + 8, inode(path)); + put32(b + 16, 501); + put32(b + 20, 20); + put64(b + 96, static_cast(size)); + put64(b + 104, static_cast((size + 511) / 512)); + put32(b + 112, 4096); + uc_mem_write(uc_, addr, b, sizeof b); +} + +uint64_t Vm::bumpScratch(uint64_t size) { + const uint64_t addr = scratchCur_; + scratchCur_ += (size + 0xF) & ~static_cast(0xF); + if (scratchCur_ > SCRATCH_ADDRESS + SCRATCH_SIZE) throw Error("emulation: scratch exhausted"); + return addr; +} + +bool Vm::call(const Handler &h, uint64_t &r) { + switch (h.kind) { + case Kind::Abort: + fault_ = "stack check failed"; + uc_emu_stop(uc_); + return false; + case Kind::Ret: + r = h.value; + return true; + case Kind::Arc4Buf: { + const uint64_t buf = arg(0); + const size_t n = static_cast(arg(1)); + std::vector bytes(n); + for (size_t i = 0; i < n; i++) bytes[i] = static_cast(i * 7 + 3); + uc_mem_write(uc_, buf, bytes.data(), bytes.size()); + r = 0; + return true; + } + case Kind::HostUuid: + uc_mem_write(uc_, arg(0), hostUuid_, 16); + r = 0; + return true; + case Kind::TimeOfDay: { + const uint64_t tv = arg(0); + if (tv != 0) { + uint64_t s, us; + if (frozenTime_ >= 0) { + s = static_cast(frozenTime_); + us = 0; + } else { + timespec now{}; + clock_gettime(CLOCK_REALTIME, &now); + s = static_cast(now.tv_sec); + us = static_cast(now.tv_nsec / 1000); + } + poke64(tv, s); + poke64(tv + 8, us); + } + r = 0; + return true; + } + case Kind::Errno: + r = errnoAddr_; + return true; + case Kind::ObjcMsgSend: { + const std::string sel = cstr(arg(1)); + if (sel == "processInfo") { + r = objCur_; + objCur_ += 0x100; + } else if (sel == "operatingSystem") { + r = 0x0D; + } else if (sel == "isOperatingSystemAtLeastVersion:") { + r = 1; + } else { + r = 0; + } + return true; + } + case Kind::Malloc: { + // A bump allocator, like the harness Apple has accepted. Nothing is freed: the VM is + // put back to its post-load snapshot between sessions instead. + const uint64_t size = (std::max(arg(0), 0x20) + 0xF) & ~static_cast(0xF); + const uint64_t addr = heapCur_; + if (addr + size > HEAP_ADDRESS + HEAP_SIZE) { + fault_ = "emulated heap exhausted"; + uc_emu_stop(uc_); + return false; + } + heapCur_ = addr + size; + r = addr; + return true; + } + case Kind::Open: { + std::string path = cstr(arg(0)); + const uint64_t flags = arg(1); + if (auto it = files_.find(path); it != files_.end()) { + if (flags & O_TRUNC_) it->second.clear(); + } else if (flags & O_CREAT_) { + files_[path] = {}; + } else if (dirs_.count(path) != 0) { + r = fail(EISDIR_); + return true; + } else { + r = fail(ENOENT_); + return true; + } + const uint64_t fd = nextFd_++; + fds_[fd] = Fd{std::move(path), 0}; + r = fd; + return true; + } + case Kind::Close: + fds_.erase(arg(0)); + r = 0; + return true; + case Kind::Read: { + const uint64_t fd = arg(0), buf = arg(1); + const size_t n = static_cast(arg(2)); + auto it = fds_.find(fd); + if (it == fds_.end()) { + r = fail(EBADF_); + return true; + } + const auto f = files_.find(it->second.path); + static const std::vector empty; + const std::vector &data = f == files_.end() ? empty : f->second; + const size_t start = std::min(it->second.off, data.size()); + const size_t len = std::min(n, data.size() - start); + if (len != 0) uc_mem_write(uc_, buf, data.data() + start, len); + it->second.off += len; + r = len; + return true; + } + case Kind::Write: { + const uint64_t fd = arg(0), buf = arg(1); + const size_t n = static_cast(arg(2)); + auto it = fds_.find(fd); + if (it == fds_.end()) { + r = fail(EBADF_); + return true; + } + std::vector payload(n); + if (n != 0 && uc_mem_read(uc_, buf, payload.data(), n) != UC_ERR_OK) payload.clear(); + std::vector &data = files_[it->second.path]; + const size_t off = it->second.off; + if (off > data.size()) data.resize(off, 0); + const size_t end = off + payload.size(); + if (end > data.size()) data.resize(end, 0); + std::copy(payload.begin(), payload.end(), data.begin() + static_cast(off)); + it->second.off = end; + r = n; + return true; + } + case Kind::Lstat: { + const std::string path = cstr(arg(0)); + const uint64_t buf = arg(1); + if (auto f = files_.find(path); f != files_.end()) { + fillStat(buf, path, false, static_cast(f->second.size())); + r = 0; + } else if (dirs_.count(path) != 0) { + fillStat(buf, path, true, 0); + r = 0; + } else { + r = fail(ENOENT_); + } + return true; + } + case Kind::Fstat: { + const uint64_t fd = arg(0), buf = arg(1); + auto it = fds_.find(fd); + if (it == fds_.end()) { + r = fail(EBADF_); + return true; + } + const auto f = files_.find(it->second.path); + const int64_t size = f == files_.end() ? 0 : static_cast(f->second.size()); + const std::string path = it->second.path; + fillStat(buf, path, false, size); + r = 0; + return true; + } + case Kind::Statfs: { + std::vector raw(STATFS_SIZE, 0); + put32(raw.data(), 0x1000003); + uc_mem_write(uc_, arg(1), raw.data(), raw.size()); + r = 0; + return true; + } + case Kind::Mkdir: { + std::string path = cstr(arg(0)); + if (files_.count(path) != 0 || dirs_.count(path) != 0) { + r = fail(EEXIST_); + return true; + } + dirs_.insert(std::move(path)); + r = 0; + return true; + } + case Kind::Unlink: { + const std::string path = cstr(arg(0)); + r = files_.erase(path) != 0 ? 0 : fail(ENOENT_); + return true; + } + case Kind::Chmod: { + const std::string path = cstr(arg(0)); + r = (files_.count(path) != 0 || dirs_.count(path) != 0) ? 0 : fail(ENOENT_); + return true; + } + case Kind::Ftruncate: { + const uint64_t fd = arg(0); + const size_t len = static_cast(arg(1)); + auto it = fds_.find(fd); + if (it == fds_.end()) { + r = fail(EBADF_); + return true; + } + files_[it->second.path].resize(len, 0); + r = 0; + return true; + } + case Kind::Getenv: { + const auto it = getenvAddr_.find(cstr(arg(0))); + r = it == getenvAddr_.end() ? 0 : it->second; + return true; + } + case Kind::CfurlCreate: { + const uint64_t buf = arg(1); + const size_t len = static_cast(arg(2)); + std::string path(len, '\0'); + if (len != 0 && uc_mem_read(uc_, buf, path.data(), len) != UC_ERR_OK) path.clear(); + nextUrl_ += 0x10; + urls_[nextUrl_] = path; + r = nextUrl_; + return true; + } + case Kind::CfurlCopyProp: { + const uint64_t url = arg(0), key = arg(1), out = arg(2); + const auto u = urls_.find(url); + const std::string path = u == urls_.end() ? std::string() : u->second; + const auto k = keyByObject_.find(key); + const std::string name = k == keyByObject_.end() ? std::string() : norm(k->second); + auto obj = [&](const char *n) { + const auto it = objectOf_.find(n); + return it == objectOf_.end() ? 0 : it->second; + }; + if (name == "kCFURLIsDirectoryKey") { + poke64(out, dirs_.count(path) != 0 ? obj("kCFBooleanTrue") : obj("kCFBooleanFalse")); + r = 1; + } else if (name == "kCFURLIsHiddenKey") { + poke64(out, obj("kCFBooleanFalse")); + r = 1; + } else if (name == "kCFURLFileSecurityKey" || name == "kCFURLTypeIdentifierKey") { + poke64(out, OBJECT_ADDRESS); + r = 1; + } else { + r = 0; + } + return true; + } + } + r = 0; + return true; +} + +uint64_t Vm::invoke(uint64_t address, const std::vector &args) { + for (size_t i = 0; i < args.size() && i < 6; i++) { + check(uc_reg_write(uc_, ARG_REGS[i], &args[i]), "set argument"); + } + uint64_t rsp = STACK_ADDRESS + STACK_SIZE - 0x1000; + writeU64(rsp, RETURN_ADDRESS); + check(uc_reg_write(uc_, UC_X86_REG_RSP, &rsp), "set rsp"); + uint64_t zero = 0; + check(uc_reg_write(uc_, UC_X86_REG_RBP, &zero), "set rbp"); + fault_.clear(); + + const uc_err err = uc_emu_start(uc_, address, RETURN_ADDRESS, 0, 0); + if (!fault_.empty()) { + const std::string message = fault_; + fault_.clear(); + throw Error("emulation: " + message); + } + if (err != UC_ERR_OK) { + uint64_t rip = 0; + uc_reg_read(uc_, UC_X86_REG_RIP, &rip); + char message[160]; + std::snprintf(message, sizeof message, "emulation failed at rip=%#llx: %s", + static_cast(rip), uc_strerror(err)); + throw Error(message); + } + uint64_t rax = 0; + check(uc_reg_read(uc_, UC_X86_REG_RAX, &rax), "read rax"); + return rax; +} + +void Vm::resetSession() { + for (const auto &seg : snapshot_.segments) write(seg.first, seg.second); + write(DATA_ADDRESS, snapshot_.cells); + check(uc_mem_unmap(uc_, HEAP_ADDRESS, HEAP_SIZE), "unmap heap"); + check(uc_mem_map(uc_, HEAP_ADDRESS, HEAP_SIZE, UC_PROT_ALL), "map heap"); + check(uc_mem_write(uc_, STACK_ADDRESS, zeros_.data(), zeros_.size()), "clear stack"); + const uint64_t base = scratchBase_; + const size_t dirty = static_cast(scratchCur_ - scratchBase_); + heapCur_ = snapshot_.heapCur; + objCur_ = snapshot_.objCur; + nextUrl_ = snapshot_.nextUrl; + nextFd_ = snapshot_.nextFd; + scratchCur_ = scratchBase_; + fds_.clear(); + urls_.clear(); + fault_.clear(); + writeU64(errnoAddr_, 0); + if (dirty != 0) { + check(uc_mem_write(uc_, base, zeros_.data(), std::min(dirty, zeros_.size())), + "clear scratch"); + } +} + +uint64_t Vm::scratch(size_t size) { return bumpScratch(size); } + +void Vm::scratchReset() { scratchCur_ = scratchBase_; } + +void Vm::write(uint64_t addr, const std::vector &data) { + if (data.empty()) return; + check(uc_mem_write(uc_, addr, data.data(), data.size()), "write"); +} + +void Vm::writeU32(uint64_t addr, uint32_t v) { + check(uc_mem_write(uc_, addr, &v, 4), "write"); +} + +void Vm::writeU64(uint64_t addr, uint64_t v) { + check(uc_mem_write(uc_, addr, &v, 8), "write"); +} + +std::vector Vm::read(uint64_t addr, size_t len) const { + std::vector out(len); + if (len != 0) check(uc_mem_read(uc_, addr, out.data(), len), "read"); + return out; +} + +uint32_t Vm::readU32(uint64_t addr) const { + uint32_t v = 0; + check(uc_mem_read(uc_, addr, &v, 4), "read"); + return v; +} + +uint64_t Vm::readU64(uint64_t addr) const { + uint64_t v = 0; + check(uc_mem_read(uc_, addr, &v, 8), "read"); + return v; +} + +void Vm::setHostUuid(const uint8_t uuid[16]) { std::memcpy(hostUuid_, uuid, 16); } + +} // namespace macadi diff --git a/app/src/main/cpp/macadi/vm.h b/app/src/main/cpp/macadi/vm.h new file mode 100644 index 00000000..3d685fef --- /dev/null +++ b/app/src/main/cpp/macadi/vm.h @@ -0,0 +1,136 @@ +// Apple's macOS CoreADI, x86_64, run under Unicorn - plus the few pieces of macOS userland it +// asks for. +// +// The dylib's preferred base is 0, so every segment is mapped at its literal vmaddr and nothing +// is rebased. Each imported function gets a stub address holding a single `ret`; a code hook on +// the stub page runs the handler in C++, sets rax, and the `ret` returns to the caller. Imported +// data symbols point at per-symbol cells. +// +// Ported from sacca97/os-findmy-e (MIT), native/adi-core/src/vm.rs, which in turn mirrors the +// Python harness its results were measured with. Behaviour is kept identical on purpose, down to +// the deterministic arc4random: that harness is what Apple has been seen to accept, and a +// "tidier" stand-in is an untested one. + +#pragma once + +#include +#include +#include +#include +#include +#include + +#include + +namespace macadi { + +// Where the library keeps its provisioning state. A virtual directory: the files live in memory +// here, and Java persists them between sessions. +extern const char *const STORE_DIR; + +class Vm { +public: + explicit Vm(const std::vector &image); + ~Vm(); + Vm(const Vm &) = delete; + Vm &operator=(const Vm &) = delete; + + // Run address(args...) until it returns to our sentinel; the result is rax. + uint64_t invoke(uint64_t address, const std::vector &args); + + // Back to the state right after loading, as if a fresh process had started. The store, the + // host UUID and the clock are kept; library memory, descriptors and allocator counters are + // not. Translated code stays cached, which is what makes this cheap. Between sessions only - + // a provisioning session id lives in library memory. + void resetSession(); + + // Per-call scratch for request buffers and out-cells, reset between top-level calls. + uint64_t scratch(size_t size); + void scratchReset(); + + void write(uint64_t addr, const std::vector &data); + void writeU32(uint64_t addr, uint32_t v); + void writeU64(uint64_t addr, uint64_t v); + std::vector read(uint64_t addr, size_t len) const; + uint32_t readU32(uint64_t addr) const; + uint64_t readU64(uint64_t addr) const; + + std::map> &files() { return files_; } + + void setHostUuid(const uint8_t uuid[16]); + + // Pin gettimeofday to `secs` for tests; negative is the real clock. + // + // Never in the app. A frozen clock makes the OTP a constant, and Apple then accepts the + // machine for about a minute and refuses it after (os-findmy-e EVIDENCE.md sections 8-9). + void setFrozenTime(int64_t secs) { frozenTime_ = secs; } + +private: + enum class Kind { + Abort, Ret, Arc4Buf, HostUuid, TimeOfDay, Errno, ObjcMsgSend, Malloc, Open, Close, Read, + Write, Lstat, Fstat, Statfs, Mkdir, Unlink, Chmod, Ftruncate, Getenv, CfurlCreate, + CfurlCopyProp, + }; + struct Handler { + Kind kind; + uint64_t value; + }; + struct Fd { + std::string path; + size_t off; + }; + struct Snapshot { + std::vector>> segments; + std::vector cells; + uint64_t heapCur; + uint64_t objCur; + uint64_t nextUrl; + uint64_t nextFd; + }; + + static bool handlerFor(const std::string &name, Handler &out); + static void onStub(uc_engine *uc, uint64_t address, uint32_t size, void *self); + static bool onUnmapped(uc_engine *uc, uc_mem_type type, uint64_t address, int size, + int64_t value, void *self); + + // Runs the stand-in for one imported function. Returns false when it stopped the emulation + // instead of returning a value. + bool call(const Handler &h, uint64_t &result); + + uint64_t arg(int i) const; + std::string cstr(uint64_t addr) const; + uint64_t fail(uint64_t err); + void poke32(uint64_t addr, uint32_t v); + void poke64(uint64_t addr, uint64_t v); + void fillStat(uint64_t addr, const std::string &path, bool isDir, int64_t size); + uint64_t bumpScratch(uint64_t size); + void check(uc_err err, const char *what) const; + + uc_engine *uc_ = nullptr; + uc_hook stubHook_ = 0; + uc_hook unmappedHook_ = 0; + + std::unordered_map handlers_; + std::unordered_map objectOf_; + std::unordered_map keyByObject_; + uint64_t heapCur_ = 0; + uint64_t scratchCur_ = 0; + uint64_t scratchBase_ = 0; + uint64_t objCur_ = 0; + uint64_t nextUrl_ = 0; + uint64_t errnoAddr_ = 0; + std::map getenvAddr_; + std::map> files_; + std::set dirs_; + std::map fds_; + uint64_t nextFd_ = 3; + std::map urls_; + uint8_t hostUuid_[16]; + int64_t frozenTime_ = -1; + std::string fault_; + + Snapshot snapshot_; + std::vector zeros_; +}; + +} // namespace macadi diff --git a/app/src/main/java/dev/wander/android/opentagviewer/anisette/AnisetteSource.java b/app/src/main/java/dev/wander/android/opentagviewer/anisette/AnisetteSource.java index ba5bf60e..9096e6c7 100644 --- a/app/src/main/java/dev/wander/android/opentagviewer/anisette/AnisetteSource.java +++ b/app/src/main/java/dev/wander/android/opentagviewer/anisette/AnisetteSource.java @@ -94,6 +94,17 @@ public interface AnisetteSource { */ String serial(); + /** + * {@code X-Apple-I-MD-RINFO}: the routing number the machine was provisioned under. + * + *

FindMy.py's own providers send a constant - 17106176, which is what Android-backed + * Anisette servers use. That is right for {@link LocalAnisette}, which is one, and so is the + * default here. {@link EmulatedMacAnisette} answers with the number Apple assigned its Mac. + */ + default String router() { + return "17106176"; + } + /** Short human-readable state, for logs and diagnostics. */ String describe(); } diff --git a/app/src/main/java/dev/wander/android/opentagviewer/anisette/EmulatedMacAnisette.java b/app/src/main/java/dev/wander/android/opentagviewer/anisette/EmulatedMacAnisette.java new file mode 100644 index 00000000..37fa58ca --- /dev/null +++ b/app/src/main/java/dev/wander/android/opentagviewer/anisette/EmulatedMacAnisette.java @@ -0,0 +1,351 @@ +package dev.wander.android.opentagviewer.anisette; + +import android.content.Context; +import android.content.SharedPreferences; +import android.util.Base64; +import android.util.Log; + +import dev.wander.android.opentagviewer.db.repo.model.UserSettings; + +import org.json.JSONException; +import org.json.JSONObject; + +import java.io.File; +import java.io.FileOutputStream; +import java.io.IOException; +import java.util.Locale; +import java.util.Map; + +/** + * Anisette from an emulated Mac on this device: Apple's macOS CoreADI under an x86_64 emulator. + * + *

Why this replaced {@link LocalAnisette}. That one runs Apple's Android libraries, + * which mint Android machines. Since 2026-10-12 Apple issues the Find My token only to a sign-in + * made with a Mac- or iPhone-minted machine, and from about 2026-10-20 its edge refuses any client + * that admits to being Android - so an Android-minted machine can no longer sign in to Find My by + * any combination of headers. A Mac-minted one can, and this mints them. See {@link MacCoreAdi}. + * + *

The machine must be the same one for the life of the session, and not only at + * sign-in: the Find My token only works alongside Anisette from the machine that obtained it + * (os-findmy-e EVIDENCE.md section 6). So everything this hands out comes from one identity, + * stored under {@code files/anisette/mac/}, and the same identity answers before and after it is + * ready. + * + *

Once per process. The emulator, the library and the identity are static, for the same + * reason {@link LocalAnisette}'s are: {@code AppDependencies} builds a new source for every caller, + * and two emulators over one store would each think they owned it. + */ +public final class EmulatedMacAnisette implements AnisetteSource { + private static final String TAG = "EmulatedMacAnisette"; + + private static final String PREFERENCES = "anisette-emulated-mac"; + private static final String KEY_SESSION_WAS_LOCAL = "sessionEstablishedLocally"; + + /** How long one generated set is reused. See {@code LocalAnisette.CACHE_FOR_MS}. */ + private static final long CACHE_FOR_MS = 5_000; + + /** Guards everything static below, and every call into the emulator. */ + private static final Object LOCK = new Object(); + private static MacCoreAdi adi; + private static MacIdentity identity; + private static Map cached; + private static long cachedAt; + + private final Context context; + private final UserSettings settings; + private final boolean hasExistingSession; + private volatile String unavailableReason; + + public EmulatedMacAnisette(Context context, UserSettings settings, boolean hasExistingSession) { + this.context = context.getApplicationContext(); + this.settings = settings; + this.hasExistingSession = hasExistingSession; + } + + private File directory() { + return new File(this.context.getFilesDir(), "anisette/mac"); + } + + private File storeDirectory() { + return new File(directory(), "store"); + } + + @Override + public boolean ensureReady() { + synchronized (LOCK) { + if (adi != null) { + return true; + } + if (!this.settings.usesLocalAnisette(this.hasExistingSession)) { + this.unavailableReason = "a remote Anisette server is selected in Settings"; + return false; + } + + MacCoreAdi opened = null; + try { + final byte[] library = MacCoreAdiFetcher.ensure( + new File(directory(), "CoreADI-4.19-x86_64")); + + MacIdentity current = loadOrCreateIdentity(); + opened = open(library, current); + + if (current.provisionedAtMillis() == 0) { + current = provision(opened, current); + } else if (!isProvisioned(opened)) { + // The identity says provisioned but the store has lost it. Apple will not + // provision the same device id twice (HTTP 433), so this is a new machine: + // a second device-list entry, which is better than never signing in again. + Log.w(TAG, "the emulated Mac's store no longer holds its provisioning;" + + " registering a new machine"); + opened.close(); + current = MacIdentity.generate(); + current.save(directory()); + clearStore(); + opened = open(library, current); + current = provision(opened, current); + } + + adi = opened; + identity = current; + cached = null; + this.unavailableReason = null; + Log.i(TAG, "ready as an emulated Mac, device " + current.deviceId()); + return true; + } catch (final Exception | LinkageError e) { + // LinkageError too: an UnsatisfiedLinkError loading libmacadi.so is the likeliest + // failure on a device nobody has tried, and it must fall back, not crash sign-in. + if (opened != null) { + opened.close(); + } + this.unavailableReason = e.getMessage() == null ? e.toString() : e.getMessage(); + Log.w(TAG, "the emulated Mac is unavailable, falling back to a remote server: " + + this.unavailableReason, e); + return false; + } + } + } + + private MacCoreAdi open(byte[] library, MacIdentity current) throws IOException { + final MacCoreAdi opened = new MacCoreAdi(library); + opened.setHostUuid(current.hostUuidBytes()); + final File[] stored = storeDirectory().listFiles(); + if (stored != null) { + for (final File file : stored) { + if (file.isFile() && !file.getName().endsWith(".partial")) { + opened.storePut(file.getName(), MacCoreAdiFetcher.readFile(file)); + } + } + } + return opened; + } + + private static boolean isProvisioned(MacCoreAdi opened) { + try { + opened.headers(MacCoreAdi.ANONYMOUS_DS_ID); + return true; + } catch (final MacCoreAdi.NotProvisionedException e) { + return false; + } + } + + /** + * Register this machine with Apple and keep the result. + * + *

The store is written before the identity is marked provisioned. If the process dies in + * between, the next launch finds a provisioned store under an unprovisioned identity, the + * library refuses to provision again, and the failure is visible - rather than the other way + * round, an identity marked provisioned over an empty store, which costs a second device. + */ + private MacIdentity provision(MacCoreAdi opened, MacIdentity current) throws Exception { + final long routing = new MacProvisioning(current, opened).provision(); + saveStore(opened); + final MacIdentity done = current.provisioned(routing, System.currentTimeMillis()); + done.save(directory()); + return done; + } + + private void saveStore(MacCoreAdi opened) throws IOException { + final File dir = storeDirectory(); + if (!dir.isDirectory() && !dir.mkdirs()) { + throw new IOException("could not create " + dir); + } + for (final Map.Entry file : opened.storeFiles().entrySet()) { + // The legacy whole-machine blob makes the library report "not provisioned" whenever + // it is present, and is never read back - see CoreAdi::beginSession. + if ("adi.pb".equals(file.getKey())) { + continue; + } + final File target = new File(dir, file.getKey()); + final File partial = new File(dir, file.getKey() + ".partial"); + try (FileOutputStream out = new FileOutputStream(partial)) { + out.write(file.getValue()); + out.getFD().sync(); + } + if (!partial.renameTo(target)) { + throw new IOException("could not store " + target); + } + } + } + + private void clearStore() { + final File[] stored = storeDirectory().listFiles(); + if (stored != null) { + for (final File file : stored) { + if (!file.delete()) { + Log.w(TAG, "could not delete " + file); + } + } + } + } + + /** + * The identity, read or - on a fresh install - created and saved. Never provisions: that + * needs the library, and this has to answer without it. + */ + private MacIdentity loadOrCreateIdentity() throws IOException { + synchronized (LOCK) { + if (identity != null) { + return identity; + } + final MacIdentity stored = MacIdentity.load(directory()); + if (stored != null) { + return stored; + } + final MacIdentity fresh = MacIdentity.generate(); + fresh.save(directory()); + Log.i(TAG, "generated a new emulated Mac identity - this must now be kept"); + return fresh; + } + } + + private MacIdentity currentIdentity() { + try { + return loadOrCreateIdentity(); + } catch (final IOException e) { + throw new IllegalStateException("could not read or create the emulated Mac", e); + } + } + + @Override + public String unavailableReason() { + return this.unavailableReason; + } + + @Override + public boolean isChangingMachineIdentity() { + return wasSessionEstablishedLocally() && !ensureReady(); + } + + @Override + public String otp() { + return header("X-Apple-I-MD"); + } + + @Override + public String machine() { + return header("X-Apple-I-MD-M"); + } + + /** + * The routing number the library reports, which is the one Apple assigned at provisioning. + * Not FindMy.py's constant: that is an Android or Windows value, and this is a Mac. + */ + @Override + public String router() { + final String value = header("X-Apple-I-MD-RINFO"); + return value.isEmpty() ? AnisetteSource.super.router() : value; + } + + private String header(String name) { + if (!ensureReady()) { + return ""; + } + synchronized (LOCK) { + final long now = System.currentTimeMillis(); + if (cached == null || now - cachedAt > CACHE_FOR_MS) { + try { + final MacCoreAdi.Headers generated = + adi.headers(MacCoreAdi.ANONYMOUS_DS_ID); + // RequestOTP rewrites the store, and a store that goes back in time is a + // machine Apple has seen move backwards. + saveStore(adi); + cached = Map.of( + "X-Apple-I-MD-M", + Base64.encodeToString(generated.machineId, Base64.NO_WRAP), + "X-Apple-I-MD", + Base64.encodeToString(generated.oneTimePassword, Base64.NO_WRAP), + "X-Apple-I-MD-RINFO", Long.toString(generated.routing)); + cachedAt = now; + } catch (final Exception e) { + Log.w(TAG, "could not produce Anisette data", e); + this.unavailableReason = String.valueOf(e.getMessage()); + return ""; + } + } + final String value = cached.get(name); + return value == null ? "" : value; + } + } + + @Override + public void recordSessionProvenance(boolean establishedLocally) { + preferences().edit().putBoolean(KEY_SESSION_WAS_LOCAL, establishedLocally).apply(); + Log.i(TAG, "session established with " + (establishedLocally ? "the emulated Mac" : "remote") + + " Anisette"); + } + + @Override + public boolean wasSessionEstablishedLocally() { + return preferences().getBoolean(KEY_SESSION_WAS_LOCAL, false); + } + + @Override + public String hardwareProfileJson() { + return MacIdentity.hardwareJson(); + } + + @Override + public String deviceIdsJson() { + final MacIdentity current = currentIdentity(); + try { + return new JSONObject() + .put("uid", current.localUserUuid()) + .put("devid", current.deviceId()) + .toString(); + } catch (final JSONException e) { + throw new IllegalStateException("could not describe the emulated Mac", e); + } + } + + @Override + public String serial() { + return currentIdentity().serial(); + } + + /** + * What CloudKit is told this device is, as an uppercase UUID: the emulated Mac's host UUID. + * + *

Called from Python's {@code icloud_bridge._forCloudKit}. FindMy.py's CloudKit client + * parses the account's device id as a UUID, and a Mac's is not one. Not on + * {@link AnisetteSource}: no other source has a device id that needs it. + */ + public String cloudKitDeviceUuid() { + final String h = currentIdentity().hostUuid().toUpperCase(Locale.ROOT); + return h.substring(0, 8) + "-" + h.substring(8, 12) + "-" + h.substring(12, 16) + "-" + + h.substring(16, 20) + "-" + h.substring(20); + } + + @Override + public String describe() { + synchronized (LOCK) { + if (identity == null) { + return "emulated Mac not ready: " + this.unavailableReason; + } + return "ready as an emulated Mac, device " + identity.deviceId(); + } + } + + private SharedPreferences preferences() { + return this.context.getSharedPreferences(PREFERENCES, Context.MODE_PRIVATE); + } +} diff --git a/app/src/main/java/dev/wander/android/opentagviewer/anisette/MacCoreAdi.java b/app/src/main/java/dev/wander/android/opentagviewer/anisette/MacCoreAdi.java new file mode 100644 index 00000000..a12e3634 --- /dev/null +++ b/app/src/main/java/dev/wander/android/opentagviewer/anisette/MacCoreAdi.java @@ -0,0 +1,161 @@ +package dev.wander.android.opentagviewer.anisette; + +import java.io.Closeable; +import java.nio.charset.StandardCharsets; +import java.util.LinkedHashMap; +import java.util.Map; + +/** + * Apple's macOS CoreADI, running under an x86_64 emulator in {@code libmacadi.so}. + * + *

Why a Mac, on an Android phone. Since 2026-10-12 Apple issues the Find My token + * ({@code searchPartyToken}) only to a sign-in whose Anisette machine was minted by a Mac or an + * iPhone, and since about 2026-10-20 its edge refuses any client that says it is Android. The + * Android libraries {@link LocalAnisette} loads mint Android machines, so they can no longer + * reach Find My at all. This runs the Mac library instead - the one in iTunes 12.8.3 for Mac - + * which mints Mac machines. See sacca97/os-findmy-e, which this is ported from, for the evidence. + * + *

One call is one emulated session. The native side puts the emulator back to its + * post-load state before each of {@link #headers} and {@link #provisioningBegin}, so nothing + * builds up however long the app runs. Provisioning is the exception: + * {@link #provisioningBegin} and {@link #provisioningFinish} share one session and nothing may run + * between them. + * + *

Not thread-safe. Callers hold their own lock; {@link EmulatedMacAnisette} does. + */ +public final class MacCoreAdi implements Closeable { + + /** The one build this knows how to drive: iTunes 12.8.3, CoreADI 4.19, x86_64 slice. */ + public static final String BUILD_SHA256 = + "2eafc3c79c7f41a8afbfbeaaf23ee5dab5ad5bda55ec8e54dbc0ddf1d20ce630"; + + /** + * Where {@code _vdfut768ig}, the library's single dispatcher, sits in that build. + * + *

Tied to {@link #BUILD_SHA256}: the dispatcher's offset is a property of those exact bytes, + * and a different build at this offset would run arbitrary code as if it were the dispatcher. + */ + static final long DISPATCH_OFFSET = 0x979A0L; + + /** Apple's "no account" pseudo-DSID, which machine-level Anisette is always asked for. */ + public static final long ANONYMOUS_DS_ID = -2L; + + /** The library's own answer that this machine has never been provisioned. */ + public static final class NotProvisionedException extends IllegalStateException { + public NotProvisionedException(String message) { + super(message); + } + } + + /** One Anisette header set. */ + public static final class Headers { + /** {@code X-Apple-I-MD-M}, raw. */ + public final byte[] machineId; + /** {@code X-Apple-I-MD}, raw. */ + public final byte[] oneTimePassword; + /** {@code X-Apple-I-MD-RINFO}. */ + public final long routing; + + Headers(byte[] machineId, byte[] oneTimePassword, long routing) { + this.machineId = machineId; + this.oneTimePassword = oneTimePassword; + this.routing = routing; + } + } + + static { + System.loadLibrary("macadi"); + } + + private long handle; + + /** + * Load the library's bytes into a fresh emulator. + * + *

The caller has already checked them against {@link #BUILD_SHA256}; + * {@link MacCoreAdiFetcher} will not hand over anything else. + */ + public MacCoreAdi(byte[] image) { + this.handle = nativeOpen(image, DISPATCH_OFFSET); + } + + /** What the library hashes into the machine id. Fixed per machine; see {@link MacIdentity}. */ + public void setHostUuid(byte[] uuid) { + nativeSetHostUuid(live(), uuid); + } + + /** Tests only. A frozen clock makes the OTP a constant, which Apple refuses within a minute. */ + void setFrozenTime(long unixSeconds) { + nativeSetFrozenTime(live(), unixSeconds); + } + + public void storePut(String name, byte[] contents) { + nativeStorePut(live(), name, contents); + } + + /** The library's provisioning state, by file name. Persist it after every call. */ + public Map storeFiles() { + final Object[] flat = nativeStoreFiles(live()); + final Map out = new LinkedHashMap<>(); + for (int i = 0; i + 1 < flat.length; i += 2) { + out.put(new String((byte[]) flat[i], StandardCharsets.UTF_8), (byte[]) flat[i + 1]); + } + return out; + } + + /** @throws NotProvisionedException if this machine has never been provisioned */ + public Headers headers(long dsId) { + final long[] routing = new long[1]; + final byte[][] both = nativeHeaders(live(), dsId, routing); + return new Headers(both[0], both[1], routing[0]); + } + + /** + * Turn Apple's {@code spim} into the {@code cpim} to send back. + * + * @param sessionOut receives the session id {@link #provisioningFinish} needs + */ + public byte[] provisioningBegin(long dsId, byte[] spim, int[] sessionOut) { + return nativeProvisioningBegin(live(), dsId, spim, sessionOut); + } + + public void provisioningFinish(long dsId, int session, byte[] persistentTokenMetadata, + byte[] trustKey, long routing) { + nativeProvisioningFinish(live(), dsId, session, persistentTokenMetadata, trustKey, routing); + } + + @Override + public void close() { + if (this.handle != 0) { + nativeClose(this.handle); + this.handle = 0; + } + } + + private long live() { + if (this.handle == 0) { + throw new IllegalStateException("the emulated Mac has been closed"); + } + return this.handle; + } + + private static native long nativeOpen(byte[] image, long dispatch); + + private static native void nativeClose(long handle); + + private static native void nativeSetHostUuid(long handle, byte[] uuid); + + private static native void nativeSetFrozenTime(long handle, long secs); + + private static native void nativeStorePut(long handle, String name, byte[] data); + + private static native Object[] nativeStoreFiles(long handle); + + private static native byte[][] nativeHeaders(long handle, long dsId, long[] routingOut); + + private static native byte[] nativeProvisioningBegin( + long handle, long dsId, byte[] spim, int[] sessionOut); + + private static native void nativeProvisioningFinish( + long handle, long dsId, int session, byte[] ptm, byte[] tk, long routing); +} diff --git a/app/src/main/java/dev/wander/android/opentagviewer/anisette/MacCoreAdiFetcher.java b/app/src/main/java/dev/wander/android/opentagviewer/anisette/MacCoreAdiFetcher.java new file mode 100644 index 00000000..84893284 --- /dev/null +++ b/app/src/main/java/dev/wander/android/opentagviewer/anisette/MacCoreAdiFetcher.java @@ -0,0 +1,301 @@ +package dev.wander.android.opentagviewer.anisette; + +import android.util.Log; + +import java.io.ByteArrayInputStream; +import java.io.ByteArrayOutputStream; +import java.io.File; +import java.io.FileInputStream; +import java.io.FileOutputStream; +import java.io.IOException; +import java.io.InputStream; +import java.net.HttpURLConnection; +import java.net.URL; +import java.nio.charset.StandardCharsets; +import java.security.MessageDigest; +import java.security.NoSuchAlgorithmException; +import java.util.zip.GZIPInputStream; + +/** + * Apple's macOS CoreADI, fetched from Apple: 3.5 MB of a 291 MB installer, in one request. + * + *

Where it is. The library ships in iTunes 12.8.3 for Mac, which Apple still serves + * (KB DL1977). It sits several layers down - the disk image's HFS partition, then + * {@code Install iTunes.pkg}, then {@code CoreADI.pkg/Payload}, a gzipped cpio archive. But + * the image is a fixed file, and in it the payload happens to be stored uncompressed and in one + * piece, so its position is a constant: bytes {@link #PAYLOAD_OFFSET} to + * {@code PAYLOAD_OFFSET + PAYLOAD_LENGTH}. {@code scripts/locate_mac_coreadi.py} derives those + * numbers from the image and checks them, and can be rerun by anybody who doubts them. + * + *

Nothing is trusted that is not pinned. The payload is checked against + * {@link #PAYLOAD_SHA256} before it is unpacked, and the library against + * {@link MacCoreAdi#BUILD_SHA256} before it is written - so a server that answered with + * something else, or ignored the range, produces an error rather than a library. + * + *

Apple's code is never redistributed by this project: it is fetched, at runtime, from Apple. + */ +public final class MacCoreAdiFetcher { + private static final String TAG = "MacCoreAdiFetcher"; + + public static final String DMG_URL = "https://secure-appldnld.apple.com/itunes12/" + + "041-56365-20210413-79A5AC6D-6B0A-46EC-B399-5A381183E3BE/iTunes12.8.3.dmg"; + + static final long PAYLOAD_OFFSET = 184_678_769L; + static final int PAYLOAD_LENGTH = 3_471_595; + static final String PAYLOAD_SHA256 = + "8a6c068c95036326eb2d2885fa60912ed0f7229d0939ae92934a70a3bb6cf025"; + + /** The library inside the payload. Apple's cpio names carry a leading {@code ./}. */ + static final String FRAMEWORK_PATH = + "System/Library/PrivateFrameworks/CoreADI.framework/Versions/A/CoreADI"; + + private static final int CPU_TYPE_X86_64 = 0x01000007; + + private MacCoreAdiFetcher() { + } + + /** Thrown when what came back is not the library this app knows how to run. */ + public static final class FetchException extends Exception { + public FetchException(String message) { + super(message); + } + + public FetchException(String message, Throwable cause) { + super(message, cause); + } + } + + /** + * Make sure {@code target} holds the library, fetching it if not. + * + * @return the library's bytes + */ + public static byte[] ensure(File target) throws FetchException { + return ensure(target, DMG_URL); + } + + /** As {@link #ensure(File)}, from {@code url}. Tests serve a fixture here. */ + static byte[] ensure(File target, String url) throws FetchException { + if (target.isFile()) { + try { + final byte[] existing = readFile(target); + if (MacCoreAdi.BUILD_SHA256.equals(sha256(existing))) { + return existing; + } + Log.w(TAG, "the stored library is not the expected build; fetching it again"); + } catch (final IOException e) { + Log.w(TAG, "could not read the stored library; fetching it again", e); + } + } + + final byte[] payload = fetchPayload(url); + final byte[] library = extract(payload); + write(target, library); + Log.i(TAG, String.format("fetched Apple's macOS CoreADI (%.1f MB downloaded)", + payload.length / 1e6)); + return library; + } + + private static byte[] fetchPayload(String url) throws FetchException { + HttpURLConnection connection = null; + try { + connection = (HttpURLConnection) new URL(url).openConnection(); + connection.setConnectTimeout(30_000); + connection.setReadTimeout(60_000); + connection.setRequestProperty("Range", "bytes=" + PAYLOAD_OFFSET + "-" + + (PAYLOAD_OFFSET + PAYLOAD_LENGTH - 1)); + + // Anything but a partial response means the range was not honoured, and reading on + // would download the whole 291 MB image to throw nearly all of it away. + final int status = connection.getResponseCode(); + if (status != HttpURLConnection.HTTP_PARTIAL) { + throw new FetchException("Apple answered HTTP " + status + + " to a range request for the library; expected 206"); + } + + final byte[] payload = readAtMost(connection.getInputStream(), PAYLOAD_LENGTH + 1); + if (payload.length != PAYLOAD_LENGTH) { + throw new FetchException("expected " + PAYLOAD_LENGTH + " bytes of the installer," + + " got " + payload.length); + } + return payload; + } catch (final IOException e) { + throw new FetchException("could not download the library from Apple: " + + e.getClass().getSimpleName() + ": " + e.getMessage(), e); + } finally { + if (connection != null) { + connection.disconnect(); + } + } + } + + /** + * The x86_64 library out of the installer's payload. No network; pure bytes in, bytes out. + * + * @throws FetchException if either the payload or the library is not the pinned one + */ + static byte[] extract(byte[] payload) throws FetchException { + final String payloadHash = sha256(payload); + if (!PAYLOAD_SHA256.equals(payloadHash)) { + throw new FetchException("the installer bytes are not the expected ones (sha256 " + + payloadHash + ")"); + } + + final byte[] cpio; + try (GZIPInputStream in = new GZIPInputStream(new ByteArrayInputStream(payload))) { + cpio = readAtMost(in, 64 << 20); + } catch (final IOException e) { + throw new FetchException("could not decompress the installer payload", e); + } + + final byte[] framework = fileFromCpio(cpio, FRAMEWORK_PATH); + final byte[] library = slice(framework, CPU_TYPE_X86_64); + + final String libraryHash = sha256(library); + if (!MacCoreAdi.BUILD_SHA256.equals(libraryHash)) { + throw new FetchException("the library is not the expected build (sha256 " + + libraryHash + ")"); + } + return library; + } + + /** + * One file's contents from an "odc" (old ASCII) cpio archive, which is what Apple's package + * payloads are. + * + *

Each entry: the magic {@code 070707}, then dev, ino, mode, uid, gid, nlink, rdev as six + * octal digits each, mtime as eleven, namesize as six and filesize as eleven - 76 bytes - then + * the name with its NUL, then the data. No padding. + */ + static byte[] fileFromCpio(byte[] cpio, String wanted) throws FetchException { + int pos = 0; + while (pos + 76 <= cpio.length) { + final String magic = new String(cpio, pos, 6, StandardCharsets.US_ASCII); + if (!"070707".equals(magic)) { + throw new FetchException("not an odc cpio archive at offset " + pos); + } + final int nameSize = (int) octal(cpio, pos + 59, 6); + final long fileSize = octal(cpio, pos + 65, 11); + final int nameStart = pos + 76; + if (nameSize < 1 || nameStart + nameSize > cpio.length) { + throw new FetchException("truncated cpio entry at offset " + pos); + } + String name = new String(cpio, nameStart, nameSize - 1, StandardCharsets.UTF_8); + final int dataStart = nameStart + nameSize; + if (fileSize < 0 || dataStart + fileSize > cpio.length) { + throw new FetchException("truncated cpio data for " + name); + } + if ("TRAILER!!!".equals(name)) { + break; + } + if (name.startsWith("./")) { + name = name.substring(2); + } + // A hard link's twins carry no data; the entry that matters is the one that does. + if (wanted.equals(name) && fileSize > 0) { + final byte[] out = new byte[(int) fileSize]; + System.arraycopy(cpio, dataStart, out, 0, out.length); + return out; + } + pos = dataStart + (int) fileSize; + } + throw new FetchException(wanted + " is not in the installer payload"); + } + + /** The slice for {@code cpuType} out of a Mach-O universal binary. */ + static byte[] slice(byte[] fat, int cpuType) throws FetchException { + if (fat.length < 8 || be32(fat, 0) != 0xCAFEBABE) { + throw new FetchException("the library is not a universal binary"); + } + final int count = be32(fat, 4); + for (int i = 0; i < count; i++) { + final int entry = 8 + 20 * i; + if (entry + 20 > fat.length) { + break; + } + if (be32(fat, entry) == cpuType) { + final int offset = be32(fat, entry + 8); + final int size = be32(fat, entry + 12); + if (offset < 0 || size < 0 || (long) offset + size > fat.length) { + throw new FetchException("the library's slice lies outside the file"); + } + final byte[] out = new byte[size]; + System.arraycopy(fat, offset, out, 0, size); + return out; + } + } + throw new FetchException("the library has no x86_64 slice"); + } + + private static void write(File target, byte[] library) throws FetchException { + final File dir = target.getParentFile(); + if (dir != null && !dir.isDirectory() && !dir.mkdirs()) { + throw new FetchException("could not create " + dir); + } + // Written beside and renamed over, so a process killed mid-write never leaves a + // truncated library that the hash check would then reject on every launch until cleared. + final File partial = new File(target.getPath() + ".partial"); + try (FileOutputStream out = new FileOutputStream(partial)) { + out.write(library); + out.getFD().sync(); + } catch (final IOException e) { + throw new FetchException("could not store the library", e); + } + if (!partial.renameTo(target)) { + throw new FetchException("could not store the library at " + target); + } + } + + /** + * A whole file. Not {@code java.nio.file.Files}, which is API 26 and which core library + * desugaring does not cover - minSdk is 24. + */ + static byte[] readFile(File file) throws IOException { + try (InputStream in = new FileInputStream(file)) { + return readAtMost(in, Integer.MAX_VALUE - 1); + } + } + + private static long octal(byte[] b, int offset, int length) throws FetchException { + long v = 0; + for (int i = 0; i < length; i++) { + final int c = b[offset + i]; + if (c < '0' || c > '7') { + throw new FetchException("bad octal field in the cpio archive"); + } + v = v * 8 + (c - '0'); + } + return v; + } + + private static int be32(byte[] b, int offset) { + return ((b[offset] & 0xFF) << 24) | ((b[offset + 1] & 0xFF) << 16) + | ((b[offset + 2] & 0xFF) << 8) | (b[offset + 3] & 0xFF); + } + + private static byte[] readAtMost(InputStream in, int limit) throws IOException { + final ByteArrayOutputStream out = new ByteArrayOutputStream(); + final byte[] buffer = new byte[64 * 1024]; + int read; + while ((read = in.read(buffer)) != -1) { + out.write(buffer, 0, read); + if (out.size() > limit) { + throw new IOException("more data than expected"); + } + } + return out.toByteArray(); + } + + static String sha256(byte[] data) { + try { + final byte[] digest = MessageDigest.getInstance("SHA-256").digest(data); + final StringBuilder hex = new StringBuilder(digest.length * 2); + for (final byte b : digest) { + hex.append(String.format("%02x", b)); + } + return hex.toString(); + } catch (final NoSuchAlgorithmException e) { + throw new IllegalStateException("SHA-256 is always available", e); + } + } +} diff --git a/app/src/main/java/dev/wander/android/opentagviewer/anisette/MacIdentity.java b/app/src/main/java/dev/wander/android/opentagviewer/anisette/MacIdentity.java new file mode 100644 index 00000000..4c219b54 --- /dev/null +++ b/app/src/main/java/dev/wander/android/opentagviewer/anisette/MacIdentity.java @@ -0,0 +1,205 @@ +package dev.wander.android.opentagviewer.anisette; + +import org.json.JSONException; +import org.json.JSONObject; + +import java.io.File; +import java.io.FileOutputStream; +import java.io.IOException; +import java.nio.charset.StandardCharsets; +import java.security.SecureRandom; +import java.util.Locale; + +/** + * Who the emulated Mac says it is - every value Apple sees about the machine, in one place. + * + *

Rule 11 applies with extra force. The library hashes {@link #hostUuid()} into the + * machine id it mints, and Apple binds the provisioning to the device id, local user id and serial + * presented with it. So these are generated once, stored beside the library's own state + * ({@code identity.json} next to {@code store/}), and never regenerated: a second set is a second + * Mac on the user's Apple account, and the session established with the first stops working. + * + *

The shapes are a real Mac's, and the values are the ones that have been seen to + * work. They follow sacca97/os-findmy-e's {@code mac_anisette.py} and {@code gsa.py}, which + * is the configuration Apple has been observed to issue a Find My token to. Where that source is + * inconsistent with itself - provisioning sends Darwin 22.3.0 and the login 22.2.0 - it is copied + * rather than corrected, because the consistent version is the untested one. + * + *

The serial is this app's own {@code 0PENTAGV} serial, so the device-list entry stays + * recognisable (rule 11). Apple has not been seen to check serials at all. + */ +public final class MacIdentity { + + static final String MODEL = "MacBookPro13,2"; + static final String OS_NAME = "macOS"; + static final String OS_VERSION = "13.1"; + static final String OS_BUILD = "22C65"; + static final String CFNETWORK = "1404.0.5"; + + /** What the sign-in reports, through FindMy.py's {@code DeviceIdentity}. */ + static final String LOGIN_DARWIN = "22.2.0"; + + /** {@code X-MMe-Client-Info} for provisioning. FindMy.py composes the same string at login. */ + static final String CLIENT_INFO = "<" + MODEL + "> <" + OS_NAME + ";" + OS_VERSION + ";" + + OS_BUILD + "> "; + + /** The provisioning requests' User-Agent - Darwin 22.3.0 here, as in the reference. */ + static final String PROVISIONING_USER_AGENT = "akd/1.0 CFNetwork/" + CFNETWORK + + " Darwin/22.3.0"; + + /** + * A Mac's {@code X-Mme-Device-Id} is its provisioning UDID - eight hex, a dash, sixteen hex - + * and not its hardware UUID. + */ + private static final String UDID_PREFIX = "00006000"; + + private static final String FILE = "identity.json"; + + private final String deviceId; + private final String localUserUuid; + private final String hostUuid; + private final String serial; + private final long routing; + private final long provisionedAtMillis; + + private MacIdentity(String deviceId, String localUserUuid, String hostUuid, String serial, + long routing, long provisionedAtMillis) { + this.deviceId = deviceId; + this.localUserUuid = localUserUuid; + this.hostUuid = hostUuid; + this.serial = serial; + this.routing = routing; + this.provisionedAtMillis = provisionedAtMillis; + } + + /** A fresh, unprovisioned machine. Call once, {@link #save}, never call again. */ + static MacIdentity generate() { + final SecureRandom random = new SecureRandom(); + return new MacIdentity( + UDID_PREFIX + "-" + hex(random, 8).toUpperCase(Locale.ROOT), + hex(random, 16).toUpperCase(Locale.ROOT), + hex(random, 16), + AdiDeviceIdentity.generateSerial(random), + 0, + 0); + } + + /** The stored identity, or null if this install has never had one. */ + static MacIdentity load(File directory) throws IOException { + final File file = new File(directory, FILE); + if (!file.isFile()) { + return null; + } + try { + final JSONObject json = new JSONObject( + new String(MacCoreAdiFetcher.readFile(file), StandardCharsets.UTF_8)); + return new MacIdentity( + json.getString("deviceId"), + json.getString("localUserUuid"), + json.getString("hostUuid"), + json.getString("serial"), + json.optLong("routing", 0), + json.optLong("provisionedAtMillis", 0)); + } catch (final JSONException e) { + throw new IOException("the emulated Mac's identity is unreadable", e); + } + } + + void save(File directory) throws IOException { + if (!directory.isDirectory() && !directory.mkdirs()) { + throw new IOException("could not create " + directory); + } + final String text; + try { + text = new JSONObject() + .put("deviceId", this.deviceId) + .put("localUserUuid", this.localUserUuid) + .put("hostUuid", this.hostUuid) + .put("serial", this.serial) + .put("routing", this.routing) + .put("provisionedAtMillis", this.provisionedAtMillis) + .toString(2); + } catch (final JSONException e) { + throw new IOException("could not describe the emulated Mac", e); + } + final File partial = new File(directory, FILE + ".partial"); + try (FileOutputStream out = new FileOutputStream(partial)) { + out.write(text.getBytes(StandardCharsets.UTF_8)); + out.getFD().sync(); + } + if (!partial.renameTo(new File(directory, FILE))) { + throw new IOException("could not store the emulated Mac's identity"); + } + } + + /** This identity, provisioned, with the routing Apple assigned it. */ + MacIdentity provisioned(long assignedRouting, long atMillis) { + return new MacIdentity(this.deviceId, this.localUserUuid, this.hostUuid, this.serial, + assignedRouting, atMillis); + } + + /** {@code X-Mme-Device-Id}. */ + public String deviceId() { + return this.deviceId; + } + + /** + * The local user id. Sent raw at provisioning and as {@code base64} of this at login, by + * FindMy.py - which is what the reference does, and what Apple accepted. + */ + public String localUserUuid() { + return this.localUserUuid; + } + + /** What {@code gethostuuid} returns inside the emulator, as 32 hex characters. */ + public String hostUuid() { + return this.hostUuid; + } + + byte[] hostUuidBytes() { + final byte[] out = new byte[16]; + for (int i = 0; i < 16; i++) { + out[i] = (byte) Integer.parseInt(this.hostUuid.substring(i * 2, i * 2 + 2), 16); + } + return out; + } + + public String serial() { + return this.serial; + } + + /** {@code X-Apple-I-MD-RINFO}, as Apple assigned it at provisioning. 0 before that. */ + public long routing() { + return this.routing; + } + + public long provisionedAtMillis() { + return this.provisionedAtMillis; + } + + /** The six values FindMy.py's {@code DeviceIdentity.from_json} takes. See rule 11. */ + static String hardwareJson() { + try { + return new JSONObject() + .put("model", MODEL) + .put("os_name", OS_NAME) + .put("os_version", OS_VERSION) + .put("os_build", OS_BUILD) + .put("cfnetwork", CFNETWORK) + .put("darwin", LOGIN_DARWIN) + .toString(); + } catch (final JSONException e) { + throw new IllegalStateException("could not describe the emulated Mac", e); + } + } + + private static String hex(SecureRandom random, int bytes) { + final byte[] buffer = new byte[bytes]; + random.nextBytes(buffer); + final StringBuilder out = new StringBuilder(bytes * 2); + for (final byte b : buffer) { + out.append(String.format("%02x", b)); + } + return out.toString(); + } +} diff --git a/app/src/main/java/dev/wander/android/opentagviewer/anisette/MacProvisioning.java b/app/src/main/java/dev/wander/android/opentagviewer/anisette/MacProvisioning.java new file mode 100644 index 00000000..4c3cc0f3 --- /dev/null +++ b/app/src/main/java/dev/wander/android/opentagviewer/anisette/MacProvisioning.java @@ -0,0 +1,218 @@ +package dev.wander.android.opentagviewer.anisette; + +import android.util.Base64; +import android.util.Log; + +import java.io.ByteArrayOutputStream; +import java.io.IOException; +import java.io.InputStream; +import java.net.HttpURLConnection; +import java.net.URL; +import java.nio.charset.StandardCharsets; +import java.text.SimpleDateFormat; +import java.util.Date; +import java.util.HashMap; +import java.util.Locale; +import java.util.Map; +import java.util.TimeZone; + +/** + * Registering the emulated Mac with Apple: the same three requests as {@link AdiProvisioning}, + * presented the way a Mac's {@code akd} presents them. + * + *

    + *
  1. GET the URL bag
    2. + *
  2. POST {@code midStartProvisioning}, which answers the server's metadata (spim)
    3. + *
  3. POST {@code midFinishProvisioning} with what the library made of it (cpim), which + * answers a persistent token, a trust key and the machine's routing number
    4. + *
+ * + *

The headers are copied from sacca97/os-findmy-e {@code tools/machoadi/gsa.py}, the + * configuration Apple has been seen to accept, and not tidied. + * + *

Apple allows one provisioning per device id. A second answers HTTP 433 / -29002 + * whatever else changes, so a failure after {@code midFinishProvisioning} succeeded cannot be + * retried with the same identity. That is why the caller saves the store and the identity the + * moment this returns. + */ +final class MacProvisioning { + private static final String TAG = "MacProvisioning"; + + private static final String URL_BAG = "https://gsa.apple.com/grandslam/GsService2/lookup"; + + private static final String START_REQUEST = + "\n" + + "\n" + + "" + + "HeaderRequest" + + ""; + + private static final String FINISH_REQUEST = + "\n" + + "\n" + + "" + + "Header" + + "Requestcpim%s" + + ""; + + private final MacIdentity identity; + private final MacCoreAdi adi; + + MacProvisioning(MacIdentity identity, MacCoreAdi adi) { + this.identity = identity; + this.adi = adi; + } + + /** @return the routing number Apple assigned the machine */ + long provision() throws AdiProvisioning.ProvisioningException { + final Map urls = loadUrlBag(); + final String startUrl = urls.get("midStartProvisioning"); + final String finishUrl = urls.get("midFinishProvisioning"); + if (startUrl == null || finishUrl == null) { + throw new AdiProvisioning.ProvisioningException( + "the URL bag has no provisioning endpoints. Got: " + urls.keySet()); + } + + final Object started = parse(request(startUrl, START_REQUEST), "midStartProvisioning"); + final byte[] spim = decode(started, "spim"); + + final int[] session = new int[1]; + final byte[] cpim; + try { + cpim = this.adi.provisioningBegin(MacCoreAdi.ANONYMOUS_DS_ID, spim, session); + } catch (final IllegalStateException e) { + throw new AdiProvisioning.ProvisioningException( + "the emulated Mac could not start provisioning: " + e.getMessage(), e); + } + + final Object finished = parse(request(finishUrl, String.format( + FINISH_REQUEST, Base64.encodeToString(cpim, Base64.NO_WRAP))), + "midFinishProvisioning"); + final byte[] persistentToken = decode(finished, "ptm"); + final byte[] trustKey = decode(finished, "tk"); + final String routingText = SimplePlist.string(finished, "Response", "X-Apple-I-MD-RINFO"); + long routing = 0; + if (routingText != null && !routingText.isEmpty()) { + try { + routing = Long.parseLong(routingText.trim()); + } catch (final NumberFormatException e) { + throw new AdiProvisioning.ProvisioningException( + "Apple's routing number is not a number: " + routingText); + } + } + + try { + this.adi.provisioningFinish(MacCoreAdi.ANONYMOUS_DS_ID, session[0], persistentToken, + trustKey, routing); + } catch (final IllegalStateException e) { + throw new AdiProvisioning.ProvisioningException( + "the emulated Mac could not finish provisioning: " + e.getMessage(), e); + } + Log.i(TAG, "the emulated Mac is provisioned, routing " + routing); + return routing; + } + + private Map loadUrlBag() throws AdiProvisioning.ProvisioningException { + final Object plist = parse(request(URL_BAG, null), "the URL bag"); + if (!(plist instanceof Map)) { + throw new AdiProvisioning.ProvisioningException("the URL bag is not a dictionary"); + } + final Object urls = ((Map) plist).get("urls"); + if (!(urls instanceof Map) || ((Map) urls).isEmpty()) { + throw new AdiProvisioning.ProvisioningException("the URL bag came back empty"); + } + final Map out = new HashMap<>(); + for (final Map.Entry entry : ((Map) urls).entrySet()) { + if (entry.getValue() instanceof String) { + out.put(String.valueOf(entry.getKey()), (String) entry.getValue()); + } + } + return out; + } + + private static Object parse(String body, String what) + throws AdiProvisioning.ProvisioningException { + try { + return SimplePlist.parse(body); + } catch (final Exception e) { + throw new AdiProvisioning.ProvisioningException("could not read " + what, e); + } + } + + private static byte[] decode(Object plist, String key) + throws AdiProvisioning.ProvisioningException { + final String value = SimplePlist.string(plist, "Response", key); + if (value == null) { + throw new AdiProvisioning.ProvisioningException("Apple's response has no Response/" + + key + " - the request was refused, or the protocol has changed"); + } + return Base64.decode(value, Base64.DEFAULT); + } + + private String request(String url, String body) throws AdiProvisioning.ProvisioningException { + HttpURLConnection connection = null; + try { + connection = (HttpURLConnection) new URL(url).openConnection(); + connection.setConnectTimeout(30_000); + connection.setReadTimeout(30_000); + connection.setRequestProperty("User-Agent", MacIdentity.PROVISIONING_USER_AGENT); + // AuthKit really does label an XML plist body as a form. + connection.setRequestProperty("Content-Type", "application/x-www-form-urlencoded"); + connection.setRequestProperty("Connection", "keep-alive"); + connection.setRequestProperty("Accept", "*/*"); + connection.setRequestProperty("X-Mme-Device-Id", this.identity.deviceId()); + connection.setRequestProperty("X-MMe-Client-Info", MacIdentity.CLIENT_INFO); + connection.setRequestProperty("X-Apple-I-MD-LU", this.identity.localUserUuid()); + connection.setRequestProperty("X-Apple-Client-App-Name", "Setup"); + connection.setRequestProperty("X-Apple-I-SRL-NO", this.identity.serial()); + + if (body != null) { + connection.setRequestProperty("X-Apple-I-Client-Time", now()); + connection.setRequestMethod("POST"); + connection.setDoOutput(true); + connection.getOutputStream().write(body.getBytes(StandardCharsets.UTF_8)); + } + + final int status = connection.getResponseCode(); + final String response = readAll(status >= 400 + ? connection.getErrorStream() : connection.getInputStream()); + if (status >= 400) { + throw new AdiProvisioning.ProvisioningException( + "Apple returned HTTP " + status + " for " + url + ": " + response); + } + return response; + } catch (final AdiProvisioning.ProvisioningException e) { + throw e; + } catch (final IOException e) { + throw new AdiProvisioning.ProvisioningException("could not reach " + url + ": " + + e.getClass().getSimpleName() + ": " + e.getMessage(), e); + } finally { + if (connection != null) { + connection.disconnect(); + } + } + } + + /** ISO 8601, whole seconds. Apple rejects anything with milliseconds on it. */ + private static String now() { + final SimpleDateFormat format = + new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss'Z'", Locale.ROOT); + format.setTimeZone(TimeZone.getTimeZone("UTC")); + return format.format(new Date()); + } + + private static String readAll(InputStream stream) throws IOException { + if (stream == null) { + return ""; + } + final ByteArrayOutputStream out = new ByteArrayOutputStream(); + final byte[] buffer = new byte[8192]; + int read; + while ((read = stream.read(buffer)) != -1) { + out.write(buffer, 0, read); + } + return out.toString("UTF-8"); + } +} diff --git a/app/src/main/java/dev/wander/android/opentagviewer/python/AppDependencies.java b/app/src/main/java/dev/wander/android/opentagviewer/python/AppDependencies.java index e92fec1f..62c3ca76 100644 --- a/app/src/main/java/dev/wander/android/opentagviewer/python/AppDependencies.java +++ b/app/src/main/java/dev/wander/android/opentagviewer/python/AppDependencies.java @@ -13,6 +13,7 @@ import java.util.function.Supplier; import dev.wander.android.opentagviewer.anisette.AnisetteSource; +import dev.wander.android.opentagviewer.anisette.EmulatedMacAnisette; import dev.wander.android.opentagviewer.anisette.LocalAnisette; import dev.wander.android.opentagviewer.ble.AccessorySoundTrigger; import dev.wander.android.opentagviewer.ble.BleAccessorySoundTrigger; @@ -54,8 +55,12 @@ private AppDependencies() {} * How to build Anisette for a given settings object. A factory rather than an instance * because the real one needs a Context and the current settings, and neither exists when * this class is loaded. + * + *

The emulated Mac, not {@link LocalAnisette}. The Android libraries that one runs + * mint Android machines, which Apple no longer lets reach Find My - see + * {@link EmulatedMacAnisette}. */ - private static AnisetteFactory anisetteFactory = LocalAnisette::new; + private static AnisetteFactory anisetteFactory = EmulatedMacAnisette::new; /** Builds the Anisette source for a screen, given where it is running and who is signed in. */ public interface AnisetteFactory { @@ -293,7 +298,7 @@ public static void replaceAnisette(final Function @VisibleForTesting public static void reset() { authService = new PythonAppleAuthService(); - anisetteFactory = LocalAnisette::new; + anisetteFactory = EmulatedMacAnisette::new; serverTesterFactory = AnisetteServerTesterService::new; hardwareDescriber = new ChaquopyHardwareDescriber(); accessoryMacResolver = new ChaquopyAccessoryMacResolver(); diff --git a/app/src/main/python/icloud_bridge.py b/app/src/main/python/icloud_bridge.py index 2b85beb8..1665251a 100644 --- a/app/src/main/python/icloud_bridge.py +++ b/app/src/main/python/icloud_bridge.py @@ -40,6 +40,7 @@ import sys import time import traceback +import uuid from typing import Any import identity as app_identity @@ -423,7 +424,7 @@ def open(self) -> str: try: client = self._awaiting( - lambda: icloud.open_client(self._async, self._identity), + lambda: icloud.open_client(_forCloudKit(self._async), self._identity), "opening the Find My client") self._awaiting(client.__aenter__, "starting the keychain session") self._client = client @@ -890,6 +891,54 @@ def _plist(mapping: Any) -> str | None: return plistlib.dumps(dict(mapping), fmt=plistlib.FMT_XML).decode("utf-8") +class _CloudKitDeviceView: + """ + The account, except that `device_uuid` is a UUID - for the keychain and CloudKit clients only. + + FindMy.py's CloudKit client does `uuid.UUID(account.device_uuid)` for the device identifier it + sends. An emulated Mac's device id is a Mac provisioning UDID, `00006000-` and sixteen hex, + which is not a UUID, so opening the client failed with "badly formed hexadecimal UUID string" + before anything was sent. CloudKit's identifier is a different field from `X-Mme-Device-Id`, + so this answers it with another stable value and forwards everything else to the real + account - Grand Slam, tokens and PET renewals keep the real device id. After + sacca97/os-findmy-e `tag_keys.CloudKitIdentity`, which read tags this way. + """ + + def __init__(self, account: Any, deviceUuid: str) -> None: + self._account = account + self._deviceUuid = deviceUuid + + @property + def device_uuid(self) -> str: + return self._deviceUuid + + def __getattr__(self, name: str) -> Any: + return getattr(self._account, name) + + +def _forCloudKit(asyncAccount: Any) -> Any: + """`asyncAccount`, or a view of it whose device id CloudKit can parse.""" + deviceId = getattr(asyncAccount, "device_uuid", None) + if not isinstance(deviceId, str): + return asyncAccount + try: + uuid.UUID(deviceId) + return asyncAccount + except ValueError: + pass + + # The emulated Mac's host UUID when Java can say, as os-findmy-e used; otherwise one derived + # from the device id, which is as stable as the device id is. + bridge = getattr(getattr(asyncAccount, "_anisette", None), "_bridge", None) + ask = getattr(bridge, "cloudKitDeviceUuid", None) + if ask is not None: + stable = str(ask()) + else: + stable = str(uuid.uuid5(uuid.NAMESPACE_URL, f"opentagviewer:device:{deviceId}")).upper() + print(f"The device id is not a UUID; CloudKit is told {stable} instead") + return _CloudKitDeviceView(asyncAccount, stable) + + def openSession(account: Any) -> ICloudSession | None: """ Start an iCloud conversation on the account the app is already signed in with. diff --git a/app/src/main/python/main.py b/app/src/main/python/main.py index 5bab609f..8e2a51fd 100644 --- a/app/src/main/python/main.py +++ b/app/src/main/python/main.py @@ -206,11 +206,33 @@ def __init__(self, bridge: Any, fallbackServerUrl: str, **identityKwargs: Any) - @property def otp(self) -> str: - return str(self._bridge.otp()) + return self._said("one-time password", str(self._bridge.otp())) @property def machine(self) -> str: - return str(self._bridge.machine()) + return self._said("machine id", str(self._bridge.machine())) + + def _said(self, what: str, value: str) -> str: + """Pass the value on, saying why when Java had none. + + Empty is still what goes to Apple - the contract is not to throw into a login - but + Apple's answer to it is only `ec -80009`, which names no cause. The reason is in Java. + """ + if not value: + print(f"Local Anisette produced no {what}: {self._bridge.unavailableReason()}") + return value + + @property + def router(self) -> str: + """`X-Apple-I-MD-RINFO`, from Java rather than FindMy.py's Android constant. + + The emulated Mac is provisioned under a routing number Apple assigns, and has to present + that one. A bridge without the method - an older fake in a test - keeps the library's. + """ + router = getattr(self._bridge, "router", None) + if router is None: + return super().router + return str(router()) def to_json(self, dst=None, /): """Deliberately the remote mapping - see the class docstring. @@ -318,6 +340,7 @@ def loginSync(email: str, password: str, anisetteServerUrl: str, timeout=LOGIN_TIMEOUT_SECONDS, **app_identity.deviceIdsForNewSession(localAnisette), ) + _treatHsc409AsSecondFactor(acc) state = acc.login(email, password) @@ -759,6 +782,38 @@ def acceptTerms(account: AppleAccount, pageId: str) -> str: }) +def _treatHsc409AsSecondFactor(acc: AppleAccount) -> None: + """Read Grand Slam's new way of asking for a second factor as asking for one. + + Since 2026-10-12, a password Apple accepts but wants a code for is answered at SRP `complete` + with `hsc` 409 and **no `au`** - the field the pinned FindMy.py reads to decide that 2FA is + needed. Seeing no `au`, the library concludes the sign-in is complete, skips the code, and the + `mobileme` step after it fails with `status=1`. Reported in malmeloo/FindMy.py#278 and on + parawanderer/OpenTagViewer#248; the remedy, setting `au` to `trustedDeviceSecondaryAuth`, is + the one sacca97/os-findmy-e signed in with. + + Wraps the instance's `_gsa_request`, which `_gsa_authenticate` looks up on the instance. If a + later FindMy.py renames it, this says so and changes nothing, and the 2FA bug comes back. + """ + inner = getattr(acc, "_asyncacc", None) + original = getattr(inner, "_gsa_request", None) if inner is not None else None + if inner is None or original is None: + print("FindMy's account internals have changed; hsc 409 will not be read as 2FA") + return + + async def _gsa_request(parameters: Any, *args: Any, **kwargs: Any) -> Any: + response = await original(parameters, *args, **kwargs) + status = response.get("Status") if isinstance(response, dict) else None + if (isinstance(parameters, dict) and parameters.get("o") == "complete" + and isinstance(status, dict) and status.get("hsc") == 409 + and not status.get("au")): + print("Grand Slam answered hsc 409 with no au: treating it as a second factor") + status["au"] = "trustedDeviceSecondaryAuth" + return response + + inner._gsa_request = _gsa_request + + def _preferLocalAnisette(acc: AppleAccount, localAnisette: Any) -> None: """Swap a restored account's anisette provider for the local one, if it is usable. @@ -839,6 +894,8 @@ def getAccount( acc = AppleAccount.from_json(data) _preferLocalAnisette(acc, localAnisette) + # A restored session re-authenticates on its own after a 401, through the same exchange. + _treatHsc409AsSecondFactor(acc) print(f"Restored account, login state: {acc.login_state}") diff --git a/app/src/test/java/dev/wander/android/opentagviewer/anisette/MacCoreAdiFetcherTest.java b/app/src/test/java/dev/wander/android/opentagviewer/anisette/MacCoreAdiFetcherTest.java new file mode 100644 index 00000000..6856e268 --- /dev/null +++ b/app/src/test/java/dev/wander/android/opentagviewer/anisette/MacCoreAdiFetcherTest.java @@ -0,0 +1,174 @@ +package dev.wander.android.opentagviewer.anisette; + +import static org.junit.Assert.assertArrayEquals; +import static org.junit.Assert.assertThrows; +import static org.junit.Assert.assertTrue; + +import org.junit.Rule; +import org.junit.Test; +import org.junit.rules.TemporaryFolder; + +import java.io.ByteArrayOutputStream; +import java.io.File; +import java.io.FileOutputStream; +import java.nio.charset.StandardCharsets; +import java.util.Arrays; + +/** + * Getting Apple's macOS CoreADI out of the iTunes installer, without Apple's bytes. + * + *

The real payload is Apple's and is not in the repository, so the pinned hashes cannot be + * satisfied here; {@code scripts/locate_mac_coreadi.py} checks those against the real image. What + * this covers is everything between the hashes - the cpio walk, the universal-binary slice - and + * that a server which ignores the range is refused before the 291 MB arrive. + */ +public class MacCoreAdiFetcherTest { + + @Rule + public TemporaryFolder folder = new TemporaryFolder(); + + /** One "odc" cpio entry, as Apple's package payloads write them. */ + private static void entry(ByteArrayOutputStream out, String name, byte[] data) { + final byte[] nameBytes = (name + "\0").getBytes(StandardCharsets.UTF_8); + final String header = "070707" + + "000001" + "000002" + "100644" + "000000" + "000000" + "000001" + "000000" + + String.format("%011o", 0) + + String.format("%06o", nameBytes.length) + + String.format("%011o", data.length); + out.write(header.getBytes(StandardCharsets.US_ASCII), 0, header.length()); + out.write(nameBytes, 0, nameBytes.length); + out.write(data, 0, data.length); + } + + private static byte[] cpio(Object... namesAndData) { + final ByteArrayOutputStream out = new ByteArrayOutputStream(); + for (int i = 0; i < namesAndData.length; i += 2) { + entry(out, (String) namesAndData[i], (byte[]) namesAndData[i + 1]); + } + entry(out, "TRAILER!!!", new byte[0]); + return out.toByteArray(); + } + + private static byte[] bytes(String s) { + return s.getBytes(StandardCharsets.US_ASCII); + } + + @Test + public void theLibraryIsFoundUnderItsDotSlashName() throws Exception { + final byte[] archive = cpio( + "./System/Library/PrivateFrameworks/CoreADI.framework/Versions/A/adid", bytes("adid"), + "./" + MacCoreAdiFetcher.FRAMEWORK_PATH, bytes("the library")); + + assertArrayEquals(bytes("the library"), + MacCoreAdiFetcher.fileFromCpio(archive, MacCoreAdiFetcher.FRAMEWORK_PATH)); + } + + @Test + public void aHardLinkTwinWithNoDataIsPassedOver() throws Exception { + final byte[] archive = cpio( + "./" + MacCoreAdiFetcher.FRAMEWORK_PATH, new byte[0], + "./" + MacCoreAdiFetcher.FRAMEWORK_PATH, bytes("the data-bearing one")); + + assertArrayEquals(bytes("the data-bearing one"), + MacCoreAdiFetcher.fileFromCpio(archive, MacCoreAdiFetcher.FRAMEWORK_PATH)); + } + + @Test + public void aPayloadWithoutTheLibrarySaysSo() { + final byte[] archive = cpio("./something/else", bytes("x")); + + final MacCoreAdiFetcher.FetchException thrown = assertThrows( + MacCoreAdiFetcher.FetchException.class, + () -> MacCoreAdiFetcher.fileFromCpio(archive, MacCoreAdiFetcher.FRAMEWORK_PATH)); + assertTrue(thrown.getMessage(), thrown.getMessage().contains("is not in the installer")); + } + + @Test + public void anArchiveThatIsNotOdcIsRefused() { + final byte[] newc = new byte[206]; + Arrays.fill(newc, (byte) '0'); + System.arraycopy(bytes("070701"), 0, newc, 0, 6); + + assertThrows(MacCoreAdiFetcher.FetchException.class, + () -> MacCoreAdiFetcher.fileFromCpio(newc, MacCoreAdiFetcher.FRAMEWORK_PATH)); + } + + /** A universal binary with an i386 slice and an x86_64 one, in that order, as Apple's is. */ + private static byte[] fat(byte[] i386, byte[] x8664) { + final ByteArrayOutputStream out = new ByteArrayOutputStream(); + final int first = 8 + 2 * 20; + writeBe32(out, 0xCAFEBABE); + writeBe32(out, 2); + for (final int[] arch : new int[][]{ + {0x00000007, first, i386.length}, + {0x01000007, first + i386.length, x8664.length}}) { + writeBe32(out, arch[0]); + writeBe32(out, 3); + writeBe32(out, arch[1]); + writeBe32(out, arch[2]); + writeBe32(out, 0); + } + out.write(i386, 0, i386.length); + out.write(x8664, 0, x8664.length); + return out.toByteArray(); + } + + private static void writeBe32(ByteArrayOutputStream out, int v) { + out.write(v >>> 24); + out.write(v >>> 16); + out.write(v >>> 8); + out.write(v); + } + + @Test + public void theX8664SliceIsTheOneTaken() throws Exception { + final byte[] universal = fat(bytes("thirty-two bit"), bytes("sixty-four bit")); + + assertArrayEquals(bytes("sixty-four bit"), MacCoreAdiFetcher.slice(universal, 0x01000007)); + } + + @Test + public void aThinBinaryIsNotMistakenForAUniversalOne() { + assertThrows(MacCoreAdiFetcher.FetchException.class, + () -> MacCoreAdiFetcher.slice(bytes("not a fat header"), 0x01000007)); + } + + @Test + public void bytesThatAreNotThePinnedPayloadAreNeverUnpacked() { + final MacCoreAdiFetcher.FetchException thrown = assertThrows( + MacCoreAdiFetcher.FetchException.class, + () -> MacCoreAdiFetcher.extract(bytes("anything at all"))); + assertTrue(thrown.getMessage(), thrown.getMessage().contains("not the expected ones")); + } + + @Test + public void aServerThatIgnoresTheRangeIsRefusedRatherThanRead() throws Exception { + try (FakeApkServer server = FakeApkServer.ignoringRanges(new byte[64 * 1024])) { + final File target = new File(this.folder.getRoot(), "CoreADI"); + + final MacCoreAdiFetcher.FetchException thrown = assertThrows( + MacCoreAdiFetcher.FetchException.class, + () -> MacCoreAdiFetcher.ensure(target, server.url())); + + assertTrue(thrown.getMessage(), thrown.getMessage().contains("expected 206")); + assertTrue("nothing should have been written", !target.exists()); + } + } + + @Test + public void aStoredFileOfTheWrongBuildIsFetchedAgainRatherThanUsed() throws Exception { + final File target = new File(this.folder.getRoot(), "CoreADI"); + try (FileOutputStream out = new FileOutputStream(target)) { + out.write(bytes("some other build")); + } + try (FakeApkServer server = FakeApkServer.ignoringRanges(new byte[16])) { + final MacCoreAdiFetcher.FetchException thrown = assertThrows( + MacCoreAdiFetcher.FetchException.class, + () -> MacCoreAdiFetcher.ensure(target, server.url())); + // Refused by the server's answer, which means it went to the server: the stored + // file of the wrong build was not handed out. + assertTrue(thrown.getMessage(), thrown.getMessage().contains("expected 206")); + assertArrayEquals(bytes("some other build"), MacCoreAdiFetcher.readFile(target)); + } + } +} diff --git a/app/src/test/macadi-host/CMakeLists.txt b/app/src/test/macadi-host/CMakeLists.txt new file mode 100644 index 00000000..920aa8b7 --- /dev/null +++ b/app/src/test/macadi-host/CMakeLists.txt @@ -0,0 +1,11 @@ +# Host build of the emulated Mac's Anisette core, for checking it against a reference without a +# device. See README.md. + +cmake_minimum_required(VERSION 3.22.1) +project(macadi_host CXX C) + +set(CMAKE_POSITION_INDEPENDENT_CODE ON) +include(${CMAKE_CURRENT_SOURCE_DIR}/../../main/cpp/macadi/MacAdi.cmake) + +add_executable(macadi_cli macadi_cli.cpp) +target_link_libraries(macadi_cli PRIVATE macadi_core) diff --git a/app/src/test/macadi-host/macadi_cli.cpp b/app/src/test/macadi-host/macadi_cli.cpp new file mode 100644 index 00000000..4f861d88 --- /dev/null +++ b/app/src/test/macadi-host/macadi_cli.cpp @@ -0,0 +1,89 @@ +// One Anisette header set from the emulated Mac, on the host, printed as JSON. +// +// macadi_cli +// [--load-only] +// +// The store directory is read and never written: this is for comparing output, and a check that +// advanced the machine's state would make the next comparison against a different machine. +// +// --load-only stops after loading and one Initialize + IsMachineProvisioned, which needs no +// provisioned store and so can run anywhere the library can be downloaded. + +#include +#include +#include +#include +#include +#include +#include + +#include "core_adi.h" +#include "error.h" + +namespace { + +std::vector readFile(const std::string &path) { + std::ifstream in(path, std::ios::binary); + if (!in) throw macadi::Error("cannot read " + path); + return {std::istreambuf_iterator(in), std::istreambuf_iterator()}; +} + +std::string hex(const std::vector &b) { + static const char *digits = "0123456789abcdef"; + std::string out; + for (const uint8_t c : b) { + out.push_back(digits[c >> 4]); + out.push_back(digits[c & 15]); + } + return out; +} + +} // namespace + +int main(int argc, char **argv) { + if (argc < 6) { + std::fprintf(stderr, "usage: %s coreadi dispatch_hex store_dir host_uuid_hex frozen [--load-only]\n", + argv[0]); + return 2; + } + try { + const std::vector image = readFile(argv[1]); + macadi::CoreAdi adi(image, std::strtoull(argv[2], nullptr, 16)); + + const std::string uuidHex = argv[4]; + if (uuidHex.size() != 32) throw macadi::Error("host uuid must be 32 hex characters"); + uint8_t uuid[16]; + for (int i = 0; i < 16; i++) uuid[i] = static_cast(std::stoul(uuidHex.substr(i * 2, 2), nullptr, 16)); + adi.setHostUuid(uuid); + adi.setFrozenTime(std::stoll(argv[5])); + + if (argc > 6 && std::string(argv[6]) == "--load-only") { + try { + adi.headers(-2); + std::printf("{\"loaded\": true, \"provisioned\": true}\n"); + } catch (const macadi::NotProvisioned &) { + std::printf("{\"loaded\": true, \"provisioned\": false}\n"); + } + return 0; + } + + if (DIR *dir = opendir(argv[3])) { + while (const dirent *e = readdir(dir)) { + const std::string name = e->d_name; + if (name == "." || name == "..") continue; + adi.storePut(name, readFile(std::string(argv[3]) + "/" + name)); + } + closedir(dir); + } else { + throw macadi::Error(std::string("cannot open store ") + argv[3]); + } + + const macadi::Headers h = adi.headers(-2); + std::printf("{\"mid\": \"%s\", \"otp\": \"%s\", \"routing\": %llu}\n", hex(h.mid).c_str(), + hex(h.otp).c_str(), static_cast(h.routing)); + return 0; + } catch (const std::exception &e) { + std::fprintf(stderr, "error: %s\n", e.what()); + return 1; + } +} diff --git a/app/src/test/python/test_emulated_mac.py b/app/src/test/python/test_emulated_mac.py new file mode 100644 index 00000000..d8d8b313 --- /dev/null +++ b/app/src/test/python/test_emulated_mac.py @@ -0,0 +1,154 @@ +""" +The two Python halves of signing in as the emulated Mac. + +**Against the real `AppleAccount`**, with only Grand Slam's answers replaced: the 409 shim works +by replacing `_gsa_request` on the instance, and a fake account would keep that name after a +rename (rule 3). + +**Not covered here:** that the library, given `au`, then asks for a code. That decision is in +`_gsa_authenticate` after the SRP exchange, which needs Apple's half to reach. It is the line +`if au in ("secondaryAuth", "trustedDeviceSecondaryAuth")` in the pinned `account.py`. +""" + +from __future__ import annotations + +import asyncio +import uuid +from collections.abc import Iterator +from types import SimpleNamespace +from typing import Any + +import pytest + +from findmy.reports import AppleAccount, RemoteAnisetteProvider + +import icloud_bridge +import main + + +class Bridge: + """What Java hands Python as `localAnisette`, as far as these tests need.""" + + def __init__(self, router: str | None = None) -> None: + if router is not None: + self.router = lambda: router + + def otp(self) -> str: + return "b3Rw" + + def machine(self) -> str: + return "bWFjaGluZQ==" + + +class TestTheRoutingNumber: + def test_it_is_the_one_java_reports(self) -> None: + provider = main.LocalAnisetteProvider(Bridge(router="12345678"), "https://example.invalid") + assert provider.router == "12345678" + + def test_a_bridge_without_one_keeps_the_librarys(self) -> None: + provider = main.LocalAnisetteProvider(Bridge(), "https://example.invalid") + assert provider.router == RemoteAnisetteProvider("https://example.invalid").router + + +@pytest.fixture +def account() -> Iterator[AppleAccount]: + made = AppleAccount(RemoteAnisetteProvider("https://example.invalid")) + yield made + main.closeAccount(made) + + +def answersFromGrandSlam(account: AppleAccount, status: dict[str, Any]) -> list[dict]: + """ + Make the account's Grand Slam exchange answer every request with `status`, without SRP: the + password check itself needs Apple's half, and the shim only reads what comes back. + """ + asked: list[dict] = [] + + async def gsa_request(parameters: dict[str, Any]) -> dict[str, Any]: + asked.append(parameters) + return {"Status": dict(status)} + + account._asyncacc._gsa_request = gsa_request + return asked + + +class TestHsc409WithNoAu: + def test_a_complete_answered_409_without_au_is_read_as_a_second_factor( + self, account: AppleAccount) -> None: + asked = answersFromGrandSlam(account, {"ec": 0, "hsc": 409}) + main._treatHsc409AsSecondFactor(account) + + response = asyncio.run(account._asyncacc._gsa_request({"o": "complete"})) + + assert asked == [{"o": "complete"}] + assert response["Status"]["au"] == "trustedDeviceSecondaryAuth" + + @pytest.mark.parametrize("status", [ + {"ec": 0, "hsc": 200}, + {"ec": 0, "hsc": 409, "au": "secondaryAuth"}, + ]) + def test_anything_else_is_left_as_apple_sent_it( + self, account: AppleAccount, status: dict) -> None: + answersFromGrandSlam(account, status) + main._treatHsc409AsSecondFactor(account) + + response = asyncio.run(account._asyncacc._gsa_request({"o": "complete"})) + + assert response["Status"] == status + + def test_an_init_answered_409_is_not_touched(self, account: AppleAccount) -> None: + answersFromGrandSlam(account, {"ec": 0, "hsc": 409}) + main._treatHsc409AsSecondFactor(account) + + response = asyncio.run(account._asyncacc._gsa_request({"o": "init"})) + + assert "au" not in response["Status"] + + +MAC_DEVICE_ID = "00006000-0123456789ABCDEF" + + +class TestWhatCloudKitIsToldTheDeviceIs: + def test_a_device_id_that_is_already_a_uuid_is_passed_through_untouched(self) -> None: + account = SimpleNamespace(device_uuid="9F1C2D3E-4B5A-4C6D-8E7F-001122334455") + assert icloud_bridge._forCloudKit(account) is account + + def test_a_mac_device_id_gets_the_host_uuid_java_reports(self) -> None: + bridge = SimpleNamespace(cloudKitDeviceUuid=lambda: "00112233-4455-6677-8899-AABBCCDDEEFF") + account = SimpleNamespace(device_uuid=MAC_DEVICE_ID, + _anisette=SimpleNamespace(_bridge=bridge), serial="0PENTAGVTEST") + + view = icloud_bridge._forCloudKit(account) + + assert view.device_uuid == "00112233-4455-6677-8899-AABBCCDDEEFF" + # What the CloudKit client does with it, and what failed before. + assert uuid.UUID(view.device_uuid).hex.upper() == "00112233445566778899AABBCCDDEEFF" + # Everything else is the real account's. + assert view.serial == "0PENTAGVTEST" + + def test_without_a_bridge_it_is_derived_and_stable(self) -> None: + first = icloud_bridge._forCloudKit(SimpleNamespace(device_uuid=MAC_DEVICE_ID)) + again = icloud_bridge._forCloudKit(SimpleNamespace(device_uuid=MAC_DEVICE_ID)) + + uuid.UUID(first.device_uuid) + assert first.device_uuid == again.device_uuid + + def test_a_method_reached_through_the_view_acts_on_the_real_account(self) -> None: + """request_pet swaps login state on `self` - it must be the account's self, not the view's.""" + class Account: + device_uuid = MAC_DEVICE_ID + state = "before" + + def change(self) -> None: + self.state = "after" + + real = Account() + icloud_bridge._forCloudKit(real).change() + assert real.state == "after" + + +class TestWhenTheInternalsMove: + def test_an_account_without_gsa_request_is_left_alone(self, capsys) -> None: + account = SimpleNamespace(_asyncacc=SimpleNamespace()) + main._treatHsc409AsSecondFactor(account) # type: ignore[arg-type] + assert "hsc 409 will not be read as 2FA" in capsys.readouterr().out diff --git a/scripts/locate_mac_coreadi.py b/scripts/locate_mac_coreadi.py new file mode 100644 index 00000000..2be0fe27 --- /dev/null +++ b/scripts/locate_mac_coreadi.py @@ -0,0 +1,179 @@ +#!/usr/bin/env python3 +""" +Check where Apple's macOS CoreADI sits in the iTunes 12.8.3 installer, and optionally extract it. + +The app fetches the library with one HTTP range request into a 291 MB disk image +(`MacCoreAdiFetcher`). That only works because, in that exact image, the package payload holding +the library is stored uncompressed and in one piece, so its byte range is a constant. This script +is how anybody can confirm the constants rather than take them on trust. + +The constants are read from the Java sources, not copied here, so the two cannot drift apart: +`PAYLOAD_OFFSET`, `PAYLOAD_LENGTH`, `PAYLOAD_SHA256` and `DMG_URL` from `MacCoreAdiFetcher.java`, +`BUILD_SHA256` from `MacCoreAdi.java`. + + # Ask Apple for the range, the way the app does, and check both hashes + python scripts/locate_mac_coreadi.py --remote + + # The same against a full copy of the image, and show which chunks of it the range spans + python scripts/locate_mac_coreadi.py --dmg iTunes12.8.3.dmg + + # Also write the x86_64 library out (the host check in app/src/test/macadi-host/ uses it) + python scripts/locate_mac_coreadi.py --remote --out /tmp/CoreADI.x86_64 + +Apple's library is never written into the repository. `--out` is for a scratch location. +""" + +from __future__ import annotations + +import argparse +import gzip +import hashlib +import plistlib +import re +import struct +import sys +import urllib.request +from pathlib import Path +from typing import NamedTuple + +ROOT = Path(__file__).resolve().parent.parent +JAVA = ROOT / "app/src/main/java/dev/wander/android/opentagviewer/anisette" +FRAMEWORK_PATH = "System/Library/PrivateFrameworks/CoreADI.framework/Versions/A/CoreADI" +CPU_TYPE_X86_64 = 0x01000007 + + +def constant(source: str, name: str) -> str: + """The value of `NAME = ...;` in a Java source, with string concatenation and `_` folded.""" + match = re.search(rf"\b{name}\s*=\s*(.*?);", source, re.DOTALL) + if match is None: + raise SystemExit(f"{name} not found in the Java sources") + text = match.group(1) + strings = re.findall(r'"([^"]*)"', text) + if strings: + return "".join(strings) + return text.strip().rstrip("L").replace("_", "") + + +class Constants(NamedTuple): + url: str + offset: int + length: int + payload_sha256: str + build_sha256: str + + +def load_constants() -> Constants: + fetcher = (JAVA / "MacCoreAdiFetcher.java").read_text() + adi = (JAVA / "MacCoreAdi.java").read_text() + return Constants( + url=constant(fetcher, "DMG_URL"), + offset=int(constant(fetcher, "PAYLOAD_OFFSET")), + length=int(constant(fetcher, "PAYLOAD_LENGTH")), + payload_sha256=constant(fetcher, "PAYLOAD_SHA256"), + build_sha256=constant(adi, "BUILD_SHA256"), + ) + + +def fetch_range(url: str, offset: int, length: int) -> bytes: + request = urllib.request.Request( + url, headers={"Range": f"bytes={offset}-{offset + length - 1}"}) + with urllib.request.urlopen(request, timeout=120) as response: + if response.status != 206: + raise SystemExit(f"Apple answered HTTP {response.status} to the range; expected 206") + return response.read() + + +def file_from_cpio(cpio: bytes, wanted: str) -> bytes: + """One file out of an odc cpio archive - the same walk as MacCoreAdiFetcher.fileFromCpio.""" + pos = 0 + while pos + 76 <= len(cpio): + if cpio[pos:pos + 6] != b"070707": + raise SystemExit(f"not an odc cpio archive at offset {pos}") + name_size = int(cpio[pos + 59:pos + 65], 8) + file_size = int(cpio[pos + 65:pos + 76], 8) + name = cpio[pos + 76:pos + 76 + name_size - 1].decode() + data = pos + 76 + name_size + if name == "TRAILER!!!": + break + if name.removeprefix("./") == wanted and file_size > 0: + return cpio[data:data + file_size] + pos = data + file_size + raise SystemExit(f"{wanted} is not in the payload") + + +def x86_64_slice(fat: bytes) -> bytes: + magic, count = struct.unpack(">II", fat[:8]) + if magic != 0xCAFEBABE: + raise SystemExit("the library is not a universal binary") + for i in range(count): + cpu, _, offset, size, _ = struct.unpack(">IIIII", fat[8 + 20 * i:28 + 20 * i]) + if cpu == CPU_TYPE_X86_64: + return fat[offset:offset + size] + raise SystemExit("the library has no x86_64 slice") + + +def chunks_spanned(dmg: Path, offset: int, length: int) -> list[str]: + """Which UDIF chunks the range lies in, and that they are stored raw and back to back.""" + with dmg.open("rb") as f: + f.seek(-512, 2) + koly = f.read(512) + if koly[:4] != b"koly": + raise SystemExit("not a UDIF disk image") + xml_offset, xml_length = struct.unpack(">QQ", koly[0xD8:0xE8]) + f.seek(xml_offset) + resources = plistlib.loads(f.read(xml_length)) + + spanned = [] + for part in resources["resource-fork"]["blkx"]: + table = part["Data"] + count = struct.unpack(">I", table[200:204])[0] + for i in range(count): + kind, _, _, _, at, size = struct.unpack(">IIQQQQ", table[204 + 40 * i:244 + 40 * i]) + if at < offset + length and at + size > offset: + if kind != 1: + raise SystemExit(f"the range crosses a compressed chunk (type {kind:#x})") + spanned.append(f"raw chunk at {at}, {size} bytes") + return spanned + + +def main() -> int: + parser = argparse.ArgumentParser(description=__doc__, + formatter_class=argparse.RawDescriptionHelpFormatter) + source = parser.add_mutually_exclusive_group(required=True) + source.add_argument("--remote", action="store_true", help="range-fetch from Apple") + source.add_argument("--dmg", type=Path, help="a local copy of iTunes12.8.3.dmg") + parser.add_argument("--out", type=Path, help="write the x86_64 library here") + args = parser.parse_args() + + c = load_constants() + print(f"payload: bytes {c.offset}..{c.offset + c.length - 1} of the image") + + if args.remote: + payload = fetch_range(c.url, c.offset, c.length) + else: + for line in chunks_spanned(args.dmg, c.offset, c.length): + print(f" {line}") + with args.dmg.open("rb") as f: + f.seek(c.offset) + payload = f.read(c.length) + + payload_hash = hashlib.sha256(payload).hexdigest() + print(f"payload sha256 {payload_hash}") + if payload_hash != c.payload_sha256: + raise SystemExit("payload hash does not match PAYLOAD_SHA256") + + library = x86_64_slice(file_from_cpio(gzip.decompress(payload), FRAMEWORK_PATH)) + library_hash = hashlib.sha256(library).hexdigest() + print(f"library sha256 {library_hash} ({len(library)} bytes)") + if library_hash != c.build_sha256: + raise SystemExit("library hash does not match BUILD_SHA256") + + if args.out: + args.out.write_bytes(library) + print(f"wrote {args.out}") + print("ok") + return 0 + + +if __name__ == "__main__": + sys.exit(main())