Skip to content

Port the SDNN index to Android - #1535

Merged
ryanbr merged 1 commit into
ryanbr:mainfrom
bhelm:codex/issue18-android-avgsdnn
Aug 22, 2026
Merged

ryanbr merged 1 commit into
ryanbr:mainfrom
bhelm:codex/issue18-android-avgsdnn

Conversation

@bhelm

@bhelm bhelm commented Aug 21, 2026

Copy link
Copy Markdown

Why

Android currently stores nightly RMSSD but lacks the separate five-minute SDNN index already computed and persisted on iOS. This closes that data and schema gap so native Android nights and cross-platform backups can carry the same distinct metric.

Scope

  • port the cleaned, segmented SDNN-index calculation to Kotlin
  • compute it from R-R intervals inside matched sleep sessions
  • add nullable DailyMetric.avgSdnn and preserve it through merge/coalescing paths
  • bump Room 31 to 32 with one additive nullable column
  • map Apple Health SDNN to both the legacy avgHrv compatibility field and the new semantically correct avgSdnn
  • keep Health Connect RMSSD in avgHrv; it must not be relabelled as SDNN
  • remove the resolved Android schema-oracle divergence

Existing rows remain null. There is deliberately no RMSSD-to-SDNN backfill.

Verification

  • exact Kotlin SDNN vectors cover empty/sparse input, cleaning, segment boundaries, inter-segment drift, and qualifying-segment averaging
  • analytics sleep-scope, Apple import, merge/union, migration contract, and both schema-oracle copies are covered
  • Demo and Full Android unit-test suites pass (4,199 tests in Demo; 6 intentional skips)
  • two independent frozen-delta reviews report P0=0 and P1=0

Migration verification is static plus Room KSP schema verification. The repository currently has no Room runtime migration-test harness, so this PR does not claim an exercised on-device SQLite 31-to-32 upgrade.

Non-goals

No UI, score, baseline, Health Connect export, or trust-gating behavior changes. Shared trust hardening remains a separate cross-platform concern.

Closes bhelm#18

@bhelm

bhelm commented Aug 22, 2026

Copy link
Copy Markdown
Author

Android integration verification

Tested end to end on a real Android staging installation after merging this exact reviewed commit into bhelm/integration (74807ae9). The resulting app backup was exported and inspected read-only.

  • app: 10.5.1-staging (build 349)
  • manifest and Room schema version: 32
  • SQLite PRAGMA quick_check: ok
  • SQLite PRAGMA integrity_check: ok
  • dailyMetric.avgSdnn REAL is present as the trailing nullable column
  • 3 recent nights contain distinct, non-null RMSSD and SDNN values:
Day avgHrv (RMSSD) avgSdnn
2026-08-22 40.00 ms 55.82 ms
2026-08-21 51.71 ms 67.78 ms
2026-08-20 46.16 ms 61.09 ms

This verifies the production Android calculation, persistence, Room schema, and .noopbak export path with real app data. Older rows remaining NULL are expected because the migration deliberately performs no synthetic RMSSD-to-SDNN backfill. Exact algorithm semantics remain covered by the mirrored vectors and the green automated suites.

@bhelm

bhelm commented Aug 22, 2026

Copy link
Copy Markdown
Author

Clarification on historical population: the Room 31→32 migration itself performs no SQL/value backfill; it only adds nullable avgSdnn. After upgrade, the app’s normal analyzeRecent pass can recompute up to 21 prior days from retained raw R-R data and matched sleep sessions. That is why the 2026-08-20 row received SDNN even though the feature was installed later. Older NULL rows mean the rescore had no qualifying retained inputs (or no qualifying segment), not necessarily that their date predates the migration.

@ryanbr

ryanbr commented Aug 22, 2026

Copy link
Copy Markdown
Owner

Reviewed. This is the Android twin the schema oracle has been asking for by name, and it holds up — I checked the parity chain end to end rather than trusting the description, and tested the merged result rather than the branch alone.

The parity question that decides it

Your Kotlin buckets samples in one pass; Swift re-filters the whole night per segment. Those are only equivalent if cleaning happens after windowing — Malik rejection compares each interval to its predecessor, so if Swift cleaned across the night first, its boundary intervals would carry a neighbour yours don't, and every segment edge would drift.

HRVAnalyzer.analyze(_:windowStart:windowEnd:) filters to the window (line 253) and only then calls analyze(rawRR:), which cleans (line 272). Filter-then-clean, so passing only the bucket's samples with the same bounds yields an identical raw array. Both sides also preserve input order within a segment, which matters because the day's rr is not necessarily ts-sorted. The optimisation is sound and the output is the same — O(n) instead of O(n x segments), which on a 9-hour night is not nothing.

The remaining shape differences are equivalences, not divergences: Swift walks every segment from first to last including empty spans (which yield nil sdnn and are skipped), you skip them by construction; Swift's explicit inBed.isEmpty guard is redundant because both sdnnIndex implementations already return null on empty input.

Everything else I checked

  • Sleep scope identical: same half-open >= start && < end, same segmentSec = 300.
  • Apple Health mirrors iOS exactly — AppleHealthImport.swift:52 writes avgHrv: d.hrvSDNN and avgSdnn: d.hrvSDNN, and yours now does the same. Health Connect correctly stays out of avgSdnn, with the reason written down.
  • Every construction and rebuild site: the three merge/coalesce paths you updated are the ones that mattered (a Kotlin data class silently defaults a missed field to null). On the Swift side both hand-written DailyMetric.with(...) rebuilds carry avgSdnn, and the sources that don't set it — Oura, Shortcut, CSV, Xiaomi, wearable exports — don't set it on either platform.
  • Migration: version and SCHEMA_VERSION bumped together, registered in the list, additive nullable, and the entity appends the field last so it matches ALTER TABLE ADD COLUMN order — the Room rule in CLAUDE.md.
  • Schema oracle: removing daily-avg-sdnn-ios-only is earned rather than a weakening. The entry itself said "Closing it means porting HRVAnalyzer.sdnnIndex to Kotlin plus a Room version bump", which is precisely this PR. Both copies stay byte-identical (same blob hash).
  • Nice touch that ParseProbe now exposes the real rows through one shared dailyMetricRow, so the Apple tests exercise the production mapping instead of a parallel copy of it.

Verification I ran

The branch is based on 083fc751b, two merges behind (#1490 and #1530 landed since), so I tested the merged state:

  • merges onto current main with zero conflicts
  • merged tree: 4,208 tests, 0 failures, 6 skips (main's 4,198 + your 10), --no-build-cache --rerun-tasks — the no-cache part matters here, since cross-branch Room/KSP runs can serve a stale generated DAO and quietly test the old schema
  • SchemaOracleTest 4/4 and DailySdnnMigrationTest 2/2 green in that merged run
  • your own 14 checks pass, including test (WhoopStore) — which is the Swift half of the oracle

Your note about there being no Room runtime migration harness is the right call to make explicitly. An additive nullable column is the safest migration shape there is, and claiming an exercised on-device 31-to-32 upgrade without a harness would have been the wrong thing to write.

One pre-existing thing, not yours

While reading the Apple mapping I noticed the daily step total diverges across platforms on the same import:

  • iOS steps: d.steps.map { Int($0) } — truncates
  • Android steps = d.steps?.let { Math.round(it).toInt() } — rounds half-up

AppleHealthAggregator.steps is a Double accumulated per source (stepsBySource[...] += v), so a fractional daily total is representable, and where it is fractional the two platforms store step counts one apart. Both lines predate this PR — you only moved the Android one into the shared helper — so it is an observation rather than a request. Worth a separate issue if it's reachable in practice.

Ready to merge from my side.

@ryanbr
ryanbr merged commit 49c97ca into ryanbr:main Aug 22, 2026
14 checks passed
ryanbr added a commit that referenced this pull request Aug 22, 2026
…1540)

Follow-up to #1535, found reviewing its Apple import mapping and not introduced
by it. The same Apple Health export stored a different step count on each
platform:

  Apple    steps: d.steps.map { Int($0) }                    truncates
  Android  steps = d.steps?.let { Math.round(it).toInt() }   rounds

A fractional daily total is reachable rather than theoretical: the aggregator
accumulates per source (`stepsBySource[...] += v`) before the winning source's
figure is taken, and nothing constrains a source to whole counts. On such a day
Apple stored a total one lower than Android from the same file.

Rounding is the side to converge on, and the truncation was an oversight rather
than a policy: every other count and rate converted on the SAME rows already
rounds -- avgHr, maxHr and walkingHr on the AppleDaily row, restingHr on the
DailyMetric row -- and Android rounds throughout. `steps` was the only field on
the row that did not, so Apple was inconsistent with itself before it was
inconsistent with Android.

Both sites now route through one `AppleHealthImport.stepsInt`, extracted because
the conversion sat inside a file-IO function with no test seam at all -- the same
shape #1535 used for `dailyMetricRow`, so the test exercises the production
conversion rather than a copy of it.

Swift's `.rounded()` is half-away-from-zero and Kotlin's `Math.round` is half-up.
They differ only for negatives, which a step count cannot be, and the test pins
that agreement rather than assuming it.

No Android change: it already rounds. No backfill -- this corrects future
imports, and re-importing an affected export now stores the value both platforms
agree on. The total's DERIVATION was already identical on both sides (max of
stepsBySource, #589); only the final Double-to-Int step differed. `map { Int($0) }`
now appears nowhere in the Swift tree, so these two sites were the only ones.

Verified: app-build green on both legs at the merged head, Test Strand 1191 tests
0 failures (1188 before, so all three new tests ran); doc lint clean.
ryanbr added a commit that referenced this pull request Aug 25, 2026
Curated notes for docs/releases/v10.6.0.md covering the 83 merged PRs since
v10.5.0, then Tools/appchangelog-gen.py run over them to insert the entry into
both AppChangelog.swift and AppChangelog.kt and localize the card title into all
six Android locales.

Generated rather than hand-written on purpose: the release workflow only runs
appchangelog-gen when it performs the version bump itself, so running it here
means the in-app card can be reviewed and lint-checked before anything is
published instead of arriving as a CI warning nobody reads.

Headline items are the selectable Banister TRIMP Effort scale (#1562, #1563),
the background re-score that can now finish rather than restarting forever
(#1557, #1559), the Oura ring being handed out of daytime-HR mode so its own
sleep suite can run (#1526, #1550), ramp-aware sleep with the night's HR line
(#1551, #1552), live-workout pause/discard plus SDNN on Android (#1533, #1535),
and the WHOOP 4.0 device key no longer reaching a shared strap log (#1610).

Third-party contributors credited by handle per CLAUDE.md, covering both merged
PRs and the reports behind the fixes.

The version itself is deliberately NOT bumped here: the release workflow skips
appchangelog-gen when the source version already equals the target, so
pre-bumping would ship a release whose in-app What's New still advertised the
previous one.

Verified: compileFullDebugKotlin, lintVitalFullRelease -PstagingRelease (the
ExtraTranslation gate that compile and the i18n audit both miss), i18n_audit
--ci, doc_comment_lint, and the 108 Tools tests.
@bhelm
bhelm deleted the codex/issue18-android-avgsdnn branch September 15, 2026 22:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

avgSdnn/sdnnIndex: Swift-only metric whose doc comment claims a nonexistent Kotlin twin

2 participants