Skip to content

fix(charge): stop an old WHOOP import from anchoring the Charge baselines - #2666

Merged
ryanbr merged 3 commits into
ryanbr:mainfrom
DX23876:fix/2525-charge-baseline-v2
Oct 3, 2026
Merged

ryanbr merged 3 commits into
ryanbr:mainfrom
DX23876:fix/2525-charge-baseline-v2

Conversation

@DX23876

@DX23876 DX23876 commented Oct 3, 2026

Copy link
Copy Markdown

Supersedes #2615, rebased onto current main (6ce6573). #2615 went stale with conflicts in the generated parity ledger only; the code is unchanged. Its workflow runs never started: they sit at action required because this is my first PR here, so they need your "Approve and run workflows" click.

What this PR does

A bug fix in which nights the Charge baselines are folded from. It is not a new scoring method and does not claim to make Charge more accurate.

The bug (Refs #2525). analyzeRecent folded the HRV, resting-HR and respiration baselines from the whole imported vendor history plus only the nights of the current scan window (21 by default). The wearer's own older nights dropped out on every pass; the import never did. With the 14-night centre half-life and at most 21 own nights after it, the import keeps 0.5^(21/14) ≈ 35 % of the weight on every pass, for good, pinned to whatever state the wearer was in at the end of their subscription, however long ago that was.

The fix. One pure rule, ChargeBaselines in StrandAnalytics with its Kotlin twin, decides which nights a Charge baseline reads:

  • Only nights inside a 21-calendar-day window ending today count. 21 is deliberately the default scan window, so a wearer with no import folds exactly the nights the 21-day pass always folded.
  • The wearer's own (NOOP-computed) nights are the baseline. Imported nights only seed it while the own nights alone would not yet be trusted (Baselines.minNightsTrust, 14 valid nights, counted after the recalibration epoch), and drop out entirely once they would. The rule keeps no state.
  • While seeding, the per-day precedence is exactly the old mergeNightlyIntoHistory: an imported value wins a shared day, and an own night fills a day the import does not cover or covers without a value.
  • The recalibration epochs and the respiration device-era cut (Baselines mix incompatible HRV scales across a device switch (Oura↔WHOOP), corrupting readiness/trends z-scores at the transition #459) apply as before.

Both engines (IntelligenceEngine.swift / .kt) use it for the four pass-2 baselines. The dashboard readouts that describe the same baselines, the "What shaped it" rows, the "Calibrating N of 4" count and the Charge confidence tier, used to fold repo.days themselves (every imported and every stored own night, and the resting-HR row without the recovery epoch). They now read one resolver (Repository.chargeBaselines / AppViewModel.chargeBaselines) that applies the same rule to the imported and own rows before the dashboard merge blends them, so they read the same rule and the same nights as the headline (two edge cases are listed under "Known gaps").

Who notices what

Wearer Before After
No import, normal 21-day pass last 21 own nights identical (pinned by a test on both platforms)
No import, full-history repair pass every own night ever scored the same 21 days as the normal pass
With a WHOOP import the whole import, ~35 % weight for good the import seeds only until 14 valid own nights, and only inside the 21 days
Import that ended more than 21 days ago, new strap scored from the first night against the import calibrates like any new wearer: a score after 4 valid own nights
No new night for 21 days or more, with or without an import (Today's Charge tile) "Latest sleep · date", the last scored value "Calibrating", see below
Anyone, "What shaped it" / calibration count / confidence tier folded a different history than the headline read the headline's baselines

A visible change on Today

After a gap of 21 days or more, the Charge tile on Today (and the Coupled and Liquid views, and the Android twins) now reads "Calibrating" instead of carrying the last score as "Latest sleep". The calibration count now reads the baseline the next score will be computed against, which is the rule the #393 follow-up (Bug B) set for it: N "tracks the baseline the Charge ring rides and can never over-state it". Before, the count folded every stored night, so it said "not calibrating" while the engine's baseline was already empty: for a wearer without an import, the regular 21-day pass had no nights left after such a gap, and the next score needed four nights either way.

Keeping the count on the whole history instead would bring back the #393 shape in a new place: a wearer who pairs a new strap after an old import would see a months-old imported score carried as "Latest sleep" for four days while no Charge is coming. The last score itself stays in the history. If you prefer the carried display for this case, it is a small follow-up in the state mapper (MetricTileState.resolve on Apple, scoreStateForToday on Android), independent of this fix.

Also in this PR

  • Recovery Test Centre line per pass naming what each baseline was folded from (charge baseline anchor=… window=21d hrv=own/N rhr=seed/N+M …), byte-identical on both platforms.
  • mergeNightlyIntoHistory is removed on both platforms (it would have become test-only); its precedence cases moved into ChargeBaselinesTests / ChargeBaselinesTest.
  • A comment in the respiration era block claimed hist holds every day analyzeRecent has scored; those are written under computedId. Corrected.
  • Tools/doc_comment_lint_baseline.txt: IntelligenceEngine.kt 5 → 4 (removing the helper reattached a detached doc comment).
  • Parity ledger refreshed with --base upstream/main. One out_of_scope_twin disposition for WhoopRepository.importedDailyUnionFlow, whose Swift role twin is Repository.refresh in the excluded app layer.

Deliberately not in this PR

Known gaps, stated rather than implied

  • The dashboard resolver reads the active-plus-canonical id unions the dashboard already reads; the engine reads the import id and its -noop sibling. They are the same ids on a single-strap install; with a re-added strap they can differ, as they could before.
  • The engine cuts respiration at a device-era boundary from this pass's per-day owners; the stored rows the dashboard reads carry no per-night source, so the dashboard's respiration baseline agrees with the engine's for every single-brand history only.
  • TodayView.watchScoreState (Apple Watch nights) still folds repo.days; that path scores against WatchRecovery's baseline and is out of this change's scope.

Type of change

  • Bug fix
  • New feature
  • Refactor / cleanup
  • Documentation
  • CI / tooling

How it was tested

All on the rebased branch (base 6ce657305), on a macOS host with a German system locale.

  • swift test in Packages/StrandAnalytics: 2134 tests, 0 failures (20 new in ChargeBaselinesTests).
  • xcodegen generate + xcodebuild -scheme Strand -destination 'platform=macOS' CODE_SIGNING_ALLOWED=NO test with -testLanguage en -testRegion US: build succeeded; StrandTests 2263 tests, 2 skipped, 1 failure (AppLanguageTests.testExplicitLanguageWritesAndSystemRemovesAppleOverride, caused by that flag injecting an AppleLanguages override; without the flag it passes and the English-copy assertions fail on this German host instead). ChargeBreakdownWiringTests pass.
  • iOS: xcodebuild -scheme NOOPiOS -destination 'generic/platform=iOS Simulator' CODE_SIGNING_ALLOWED=NO build: build succeeded.
  • ./gradlew testFullDebugUnitTest: 6632 tests, 8 failures, none in code this PR touches. 6 are locale only and pass with an English JVM locale (TodayExplainabilityTest ×3, VitalReadingsTableTest ×1, StandardHrSensorFormatTest ×2). The other 2 are the RecoveryDriversTest half-tie cases (-0.4999999999999929 vs -0.5, the libm case test(android): keep the Charge driver half-tie tests independent of platform libm #2588 addresses); they call only RecoveryScorer / RecoveryDrivers, which this PR does not change, and your Linux CI is green on the same base. The new and changed suites pass (ChargeBaselinesTest, RecoveryCalibrationTest, RecoveryDriversUiTest).
  • python3 Tools/doc_comment_lint.py and python3 Tools/i18n_audit.py --ci upstream/main pass.
  • python3 Tools/parity_ledger.py --refresh-derived --base upstream/main: accepted (290 known findings). The parity-governance unit tests: 160 tests, OK.

Cross-platform parity is pinned by an oracle: both suites generate the same deterministic spread of 40 imported/own histories, anchors and epochs (17 of them in the seeding phase) and assert the same literal, which was produced by the Swift implementation. Reverting the seeding precedence to a key-presence check fails it.

Not tested on hardware: nothing here is on the BLE path.

Validation

This PR changes a scored output (Charge), so the checklist from docs/VALIDATION_PROTOCOL.md follows. No improvement is claimed; the change corrects which nights enter the baselines.

Validation
- [ ] R1 Pre-registered prediction: not applicable. No accuracy claim is made, so there is no prediction to pre-register.
- [ ] R2 Self-comparison audit: not applicable, no sleep-stage reference is involved.
- [x] R3 Every behavioural claim below is regenerated by the listed test commands.
- [x] R4 Instrument matches question ("did behaviour change at all?"): replay of old and new rule on identical input, as unit tests:
      - no import: identical history and identical folded baseline on the 21-day pass;
      - repair pass: same 21 days as the tick;
      - import after handoff: shifting every imported value leaves the history unchanged (zero weight);
      - import before handoff: seeds exactly as the old merge did (precedence cases carried over).
- [ ] R5 Temporal held-out split: not measured. There is no reference for Charge to hold out against.
- [x] R6 Mechanism confirmed reachable in code: `hist` is read over "0000-01-01"…"9999-12-31" and merged with the scan window's nights only (IntelligenceEngine.swift / .kt, pass 2).
- [x] R7 The one figure quoted (≈35 % import weight) is INFERRED from the fold's half-life, 0.5^(21/14); the tests pin its consequence (old rule ≈ 54.1 bpm for 58 → 52 bpm), not the percentage.
- [x] R8 Whether the corrected baseline makes Charge "better" is unmeasurable on the data available and is not claimed.

Checklist

  • Swift package tests pass for any package I touched (swift test in Packages/StrandAnalytics)
  • Android unit tests pass if I touched android/ (8 failures outside the touched code: 6 locale, 2 libm half-tie; see above)
  • No new build warnings introduced
  • UI changes use only StrandDesign tokens — no UI was restyled, only the data behind existing rows
  • No hardcoded hex frame bytes; protocol facts live in the schema / decoders
  • Follows the conventions in docs/CONTRIBUTING.md
  • I did not commit generated output (Strand.xcodeproj/) or any secrets/keystores

One concern, two platforms: the root CONTRIBUTING.md asks to keep Android changes separate where practical, while AGENTS.md requires the Kotlin twin of an analytics change in the same PR. This follows AGENTS.md, since splitting would leave the two platforms scoring Charge differently between merges.

Related issues

Refs #2525, #459, #2583. Supersedes #2615.

Refs ryanbr#2525. The Charge baselines were folded from the whole imported
vendor history plus only the nights of the current scan window, so an
import kept about a third of the weight for good (0.5^(21/14) with the
14-night half-life), however long ago it ended.

ChargeBaselines (StrandAnalytics, with its Kotlin twin) states the
corrected rule as one pure function:

- only nights inside a 21-calendar-day window ending on the anchor day
  count; 21 is the default scan window, so a wearer with no import folds
  exactly the nights the 21-day pass always folded;
- imported nights only seed the baseline while the wearer's own nights
  alone would not be trusted (minNightsTrust valid nights after the
  epoch), then drop out entirely;
- while seeding, the per-day precedence is the existing one: an imported
  value wins, an own night fills a day the import leaves open.

A bug fix, not a new scoring method. Both suites assert the same oracle
literal over a deterministic spread of histories, produced by the Swift
implementation. Wired into the engines and the dashboard in the next two
commits.
Refs ryanbr#2525. Pass 2 of analyzeRecent now builds the HRV, resting-HR,
respiration and skin-temperature histories with ChargeBaselines.history
on both platforms, anchored on the pass's local today. The own nights are
the pass's fresh values; the window trims a full-history repair pass to
the same 21 days as the normal tick. Recalibration epochs and the
respiration device-era cut apply as before.

A Recovery Test Centre line per pass names what each baseline was folded
from (own nights, or own nights still seeded by the import).

mergeNightlyIntoHistory is removed on both platforms, since it would be
reached only from tests; its precedence cases now live in the
ChargeBaselines suites. The respiration era comment no longer claims
that hist holds the days this engine scored: those are written under
computedId. Removing the Kotlin helper reattaches a detached doc comment,
so its doc_comment_lint count drops from 5 to 4.
Refs ryanbr#2525. The "What shaped it" rows, the "Calibrating N of 4" count and
the Charge confidence tier folded repo.days themselves: every imported
and every stored own night, and the resting-HR row without the recovery
epoch. That is a different history than the one the Charge headline was
scored against.

They now read one resolver per platform, Repository.chargeBaselines and
AppViewModel.chargeBaselines, which applies ChargeBaselines.resolve to
the imported and own rows before the dashboard merge blends them.
Android gains WhoopRepository.importedDailyUnionFlow, the imported twin
of computedDailyUnionFlow, for that.

Parity ledger refreshed against upstream/main; importedDailyUnionFlow is
recorded as an out_of_scope_twin of Repository.refresh.
@DX23876

DX23876 commented Oct 3, 2026

Copy link
Copy Markdown
Author

@ryanbr heads-up: the workflow runs on this PR are waiting at action required for your "Approve and run workflows" click (first PR from me here, so GitHub holds them). This replaces #2615, which went stale before CI could run. Same three commits, rebased onto current main; the local test results are in the description.

@ryanbr
ryanbr merged commit 8931adb into ryanbr:main Oct 3, 2026
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants