Skip to content

Add experimental self-hosted one-way push on Android - #1631

Merged
ryanbr merged 9 commits into
ryanbr:mainfrom
bhelm:feature/self-hosted-push
Aug 28, 2026
Merged

ryanbr merged 9 commits into
ryanbr:mainfrom
bhelm:feature/self-hosted-push

Conversation

@bhelm

@bhelm bhelm commented Aug 25, 2026 •

Copy link
Copy Markdown

Summary

Adds an Android-first, Experimental export path for keeping a fresh copy of NOOP data on a machine the user owns.

After a productive strap offload, NOOP schedules an independent Wi-Fi-gated worker that sends fresh local data without blocking BLE offload, persistence, analytics, or the UI. App launch and a manual catch-up action provide recovery paths.

This follows the boundary agreed in #1314:

  • disabled by default; without a configured endpoint, the worker does not run and no push egress occurs;
  • the endpoint and bearer token are supplied by the user;
  • export is strictly one-way and the on-device database remains authoritative;
  • receiver failure cannot block or degrade normal NOOP behavior;
  • delivery stays outside the strap-sync path in an independent background job.

docs/SCOPE.md records this narrow Experimental exception.

What it adds

  • An Advanced Settings screen to configure and test a destination, enable export, run a catch-up, and inspect bounded diagnostics.
  • A documented, versioned PUSH_PROTOCOL 1.0 with authenticated capability discovery and a fixed client-side stream registry.
  • Incremental high-water marks for append-only streams.
  • Authoritative rolling-window replacement for editable streams, including deletion propagation and daily hashes to skip unchanged windows.
  • Bounded gzip-compressed NDJSON batches, deterministic retry IDs, exact acknowledgements, endpoint-scoped progress, and receiver-state continuity.
  • Serialized background retries, trigger coalescing, endpoint-change fencing, and productive WHOOP 4 timeout-offload handling.
  • Encrypted storage for the bearer token and push progress.

The protocol is receiver-neutral: NOOP commits to an interoperable contract, not to a particular server implementation.

Security and failure boundaries

  • Redirects are disabled so credentials are never forwarded to another destination.
  • Plain HTTP is limited to numeric local/private destinations; normal remote use requires HTTPS.
  • Capability and acknowledgement bodies, request sizes, record counts, database reads, deadlines, retries, and per-run work are bounded.
  • Invalid capability responses fail closed before health data is read or uploaded.
  • Changing or disabling the destination cancels and fences stale work before another batch is sent.
  • Receiver responses cannot add schemas, issue commands, or change phone state.

Non-goals

  • No receiver or hosted NOOP service is included.
  • No account, cloud identity, telemetry, or third-party destination.
  • No restore, read-back, remote writes, conflict resolution, or two-way sync.
  • No .noopbak, arbitrary-file, or debug-bundle upload in this first version.
  • No Apple client yet; the protocol is platform-neutral, but this implementation is Android-only.

Possible follow-ups

The same explicitly configured self-hosted destination could later gain separate, capability-gated
support for uploading .noopbak snapshots and for a generic artifact store used by manually shared
debug logs or diagnostic bundles. Those are deliberately not part of this PR or PUSH_PROTOCOL 1.0.

Tests

The Android Full Debug unit suite passes:

./android/gradlew -p android testFullDebugUnitTest --no-daemon

Coverage includes default-off and stale-worker gates, endpoint policy and rotation, capability negotiation, stream allowlisting, cursor replay, mutable replacement and deletion, multipart retries, acknowledgements, payload bounds, structured failures, WorkManager serialization, and productive-offload triggering.

Closes #1314
Closes #1622

bhelm added 6 commits August 20, 2026 17:08
# Conflicts:
#	android/app/src/main/java/com/noop/ui/AppRoot.kt
#	android/app/src/main/java/com/noop/ui/SettingsScreen.kt
#	android/app/src/main/res/values-de/strings.xml
#	android/app/src/main/res/values-es/strings.xml
#	android/app/src/main/res/values-fr/strings.xml
#	android/app/src/main/res/values-pl/strings.xml
#	android/app/src/main/res/values-pt-rPT/strings.xml
#	android/app/src/main/res/values-zh/strings.xml
@bhelm

bhelm commented Aug 25, 2026

Copy link
Copy Markdown
Author

One follow-up on the Experimental lifecycle:

SCOPE.md says this should remain Experimental until the Android background behavior is proven quiet and reliable in real-world use. What evidence or time horizon would you want for that graduation — for example, a number of releases/testers, no observed battery or strap-sync regressions, or something else? And when it graduates, should it simply move out of the Experimental settings section, or is the self-hosted network exception expected to retain that label longer-term?

Also, when would you prefer iOS parity: only after the Android implementation has graduated, or as a separate follow-up once the protocol and Android client have had some field testing? I am happy to keep this PR Android-only and follow whichever sequencing best protects the current sync path.

@ryanbr

ryanbr commented Aug 26, 2026

Copy link
Copy Markdown
Owner

Reviewed this against the conditions in #1314 rather than the summary, and the safety-critical parts hold up — I checked each one in the code:

  • enabledEndpoint() really is a plain-pref gate that returns before Room or the Keystore, which matters a lot given the trigger runs on the BLE thread after every offload. The disabled path costs one cached getBoolean.
  • WhoopBleClient takes an injected successfulOffloadSink and has no knowledge of push at all.
  • followRedirects(false) and followSslRedirects(false), and plain HTTP is validated down to a numeric private address before DNS.
  • The token stays out of Worker input data, with a comment saying why.
  • 91 com.noop.push tests, 0 failures locally.

One small thing, and I want to be clear it's a nit rather than a blocker.

isPushWifiAvailable checks TRANSPORT_WIFI:

return capabilities.hasTransport(NetworkCapabilities.TRANSPORT_WIFI)

That is satisfied by a metered Wi-Fi — a phone hotspot, or a network the user has marked metered — so it isn't quite the "Wi-Fi-gated" promise the summary and the UI make. A user reasonably reads "Wi-Fi" as "this won't spend my mobile data", and on a hotspot that reading is wrong.

The exposure is genuinely narrow, which is why I'd treat it as minor: a private-IP receiver isn't reachable over a hotspot anyway, so it fails rather than uploads, and payloads are gzipped and incremental. It only really bites with a public HTTPS receiver on a metered link.

Two lines make the code match the promise:

  • NetworkCapabilities.NET_CAPABILITY_NOT_METERED instead of (or alongside) TRANSPORT_WIFI
  • NetworkType.UNMETERED instead of NetworkType.CONNECTED in the constraint, so WorkManager stops waking the worker on cellular purely to decline

Separately, and for @ryanbr rather than for you: #1314 asked to keep the first PR small — the pusher behind the toggle plus the format doc. This carries rolling-window replacement, deletion propagation, daily window hashes, capability negotiation and an encrypted progress store as well. All coherent and tested, but it's a lot more surface to judge "stays quiet and reliable in the background" against, so worth a deliberate call on scope.

@bhelm

bhelm commented Aug 26, 2026

Copy link
Copy Markdown
Author

Update pushed in 5b07c0781.

I addressed the network-policy finding by making it an explicit Wi-Fi only setting, enabled by default:

  • enabled: WorkManager waits for an unmetered network and the runtime gate additionally requires actual Wi-Fi, so metered Wi-Fi/hotspots do not upload;
  • disabled: any connected network is allowed, including mobile data or a phone VPN (so a private receiver reachable through the VPN still works);
  • changing the setting cancels/requeues pending work and connection tests use the same policy.

The endpoint policy already supports the deployment case I use: public HTTPS domains are allowed. Only public/plain HTTP is rejected; numeric private, loopback, link-local, and ULA addresses remain permitted over HTTP, including when reached through a VPN.

I added regression coverage for settings persistence/defaults, scheduler constraints, runtime gating, and the endpoint boundary. The full com.noop.push.* suite is green: 94 tests, 0 failures.

On scope: the protocol surface is deliberately fairly complete for 1.0. I would rather make the initial version internally coherent and durable than introduce several protocol revisions immediately afterward that every receiver implementation would then need to track. The feature remains default-off and Experimental, so Android field experience can still drive lifecycle/UI decisions without destabilizing the 1.0 wire contract.

bhelm and others added 2 commits August 26, 2026 20:16
The conflict was semantic, not textual. Since this branch was cut, main documented a
third network path — the update check — and gave it §1.1c, the same number this branch
assigns to self-hosted push.

Resolved by renumbering push to §1.1d (8 references) and leaving §1.1c to the update
check, then correcting the counts main and this branch each state from their own side:
four network paths total, of which three are opt-in.

Main's absolute claims had to be weakened rather than kept, because push makes them
false: "your data never leaves the device", "only two kinds of network request", and the
Coach being "the one feature that can ever send anything about your data". Each now says
nothing leaves unless the user switches on a feature that sends it.

The threat-model egress row deliberately still reads "three explicit paths" — main
excludes the update check there too, since it sends nothing about the user.
@ryanbr
ryanbr merged commit d88fd19 into ryanbr:main Aug 28, 2026
4 checks passed
@bhelm
bhelm deleted the feature/self-hosted-push branch September 15, 2026 22:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

NOOP Data Export for External Consumption Scope question: Backup & one way Sync: self-hosted destination

2 participants