From 224e9c9b1b9e8231e05c2a317dd28d48219c369b Mon Sep 17 00:00:00 2001 From: Alex Wagner Date: Mon, 5 Oct 2026 18:16:51 +0200 Subject: [PATCH 1/5] strain: price every strain on the user's own quiet-waking level All five strain sites subtracted the cost of being awake at the 0.20 HRR population reference: the pipeline headline, series.strain_curve, the engine recompute, session/live scoring and the stored-strain rescale. analytics made that level a required argument precisely so callers would pass the user's own, but dailyQuietWakingHrr was never called. That is a substituted default, and it is wrong in both directions: - Above 0.20, a day with no activity scores 7-12/21. - Below 0.20 (fit users, low RHR), quiet minutes net negative against exercise. A 45-min run at 145 bpm scored 0.00/21 for someone sitting at 0.10 HRR, and the intraday curve climbed through the run and fell back to 0 by bedtime. The level: - Each derived day measures its own level (dailyQuietWakingHrr over >= 360 wake minutes) and persists it as `quiet_hrr`. It feeds later days only, never the day itself. - Strain is scored against the median of the trailing 28 prior measured days, imports excluded. Both derive halves get the same `valuesBefore('quiet_hrr', date)` window, so the pipeline headline, the curve and the engine recompute cannot disagree. - Sessions (save, retime, confirm, on-read re-score) read the same window for the session's own local day (`personalQuietLevelBefore`). The live gauge reads the level fresh for the session's own start day and fills the gap (never moving a level a running session was scored on); a resumed session's restored tally is re-scored as soon as it lands. - Scoring goes through analytics' strainScoreFromSeries / strainCurveFromSeries. Minutes at or above 40 % HRR are never offset by quiet time, and the curve's last point is the headline. - Fewer than 3 prior levels: strain abstains with need_baseline (the UI's "Need N more days"). 3-6 days: scored at lower confidence, "calibrating". The 0.20 constant is no longer referenced in lib/. - The net load above the level is persisted per day as `trimp_net`, the input a CTL/ATL training-load read needs instead of gross TRIMP. Keeping strain from going blank: - One-shot `quiet_hrr_seed_v1` seeds existing installs' trailing history from stored bundles (strain_curve's wake minutes joined to hr_curve), off the UI isolate. It runs first in run(), ahead of every early return, so a restored install with bundles but no 1 Hz rows is seeded too. Without it, a version bump would re-derive only a few days and every existing user's strain would go blank. - A sweep prices every day on one frozen history snapshot, so a first sync of a whole week would score every day with no level. A day whose strain abstained on the quiet level while any earlier day exists (unfinalized raw, or a stored level) gets a durable per-day `quiet_repass:` marker, written before its row. While marked, the day is never finalized, so the raw prune holds it like any unfinished day. Every run()/runDays() re-derives marked days that now have a level, before the prune (held while a timezone change is suspected, like the rescan). The marker clears only on proof: a derive that completed both halves and is either no longer blocked on the level or provably can never reach three levels from the data in hand, or when its raw has aged out or the prune has cut into its night. A failed or timed-out re-score, or the app dying between passes, leaves the day for the next run. Marking never counts; the unreachable proof counts with its reads ordered (derived set, then levels) so a concurrent commit can only make it more cautious. - The rescan signature fingerprints the dated quiet_hrr entries that can feed any rescanned day's window, so a level moving under one day's window re-prices it even when the overall count and median do not move. The newest entry (normally today's, rewritten on every light derive) is left out: it sits only in unfinalized days' windows. - The v63 strain rescale stays on the lump form (only TRIMP and the wake-minute count survive for those days), now at the personal level. It retries rather than marking done while no level exists. Before/after on the existing strain_resting_hr_source fixture (RHR 55, 300 min @62 + 60 min @130): 2.85 -> 8.55. Low-quiet day (RHR 50, 855 min @64 + 45 min @145): 0.00 -> 9.90 at their own level. Requires kAlgoVersion bump (batched) and an analytics repin to the commit adding strainScoreFromSeries / personalQuietWakingHrr / netTrimpAboveQuiet. Until then this builds only against a local analytics override. --- lib/compute/derivation_engine.dart | 401 +++++++++++++- lib/compute/manual_session.dart | 46 +- lib/compute/onehz_pipeline.dart | 117 ++--- lib/compute/quiet_level.dart | 29 + lib/compute/quiet_level_seed.dart | 105 ++++ lib/compute/strain_backfill.dart | 44 +- lib/data/db.dart | 19 + lib/data/local_repository_impl.dart | 12 + lib/state/app_state.dart | 74 ++- test/app_state_regressions_test.dart | 74 +++ test/derive_day_window_test.dart | 6 + test/live_strain_convergence_test.dart | 26 + test/manual_session_test.dart | 57 +- test/manual_workout_repo_test.dart | 6 + test/quiet_hrr_seed_test.dart | 151 ++++++ test/quiet_level_engine_test.dart | 555 ++++++++++++++++++++ test/readiness_baseline_pollution_test.dart | 68 +++ test/strain_quiet_level_test.dart | 203 +++++++ test/strain_rescale_backfill_test.dart | 44 +- test/strain_resting_hr_source_test.dart | 3 + test/tonight_sleep_test.dart | 5 + 21 files changed, 1900 insertions(+), 145 deletions(-) create mode 100644 lib/compute/quiet_level.dart create mode 100644 lib/compute/quiet_level_seed.dart create mode 100644 test/quiet_hrr_seed_test.dart create mode 100644 test/quiet_level_engine_test.dart create mode 100644 test/strain_quiet_level_test.dart diff --git a/lib/compute/derivation_engine.dart b/lib/compute/derivation_engine.dart index 44fa09a24..b84c6fbc3 100644 --- a/lib/compute/derivation_engine.dart +++ b/lib/compute/derivation_engine.dart @@ -27,8 +27,10 @@ import 'dart:io' show Platform; import 'dart:isolate'; import 'dart:math' as math; +import 'quiet_level_seed.dart' show seedQuietHrrHistoryOnce; import 'strain_backfill.dart' show backfillStrainScale; +import 'package:crypto/crypto.dart' show sha1; import 'package:flutter/foundation.dart'; import 'findings.dart'; import 'nap_edits.dart'; @@ -2278,6 +2280,13 @@ Future>> debugSweepBaselineWindows( return [for (final day in orderedDays) history.valuesBefore(key, day)]; } +/// Test seam: the baseline signature [DerivationEngine.rescanRecent] gates on, +/// rebuilt from the current `metric_series`. +@visibleForTesting +Future debugBaselineSignature() async => + (await _BaselineHistoryCache.load()).toArtifactJson()['signature'] + as String; + @visibleForTesting ({List days, String reason}) selectLightDeriveDays({ required Set rawDays, @@ -2375,6 +2384,10 @@ class _BaselineHistoryCache { // derive needs the PRIOR days' values of, which is exactly what this // snapshot is (see [maxBefore]). 'hr_ceiling_bpm', + // Each day's own quiet-waking level (median wake %HRR). Strain is priced + // against the median of the PRIOR days' values — a trait, never the day + // under test. Unitless, so no family-seam mask applies. + 'quiet_hrr', ]; /// DATED baseline samples, ascending by date, one entry per day (metric_series @@ -2524,6 +2537,34 @@ class _BaselineHistoryCache { return [for (var i = from; i < samples.length; i++) samples[i].value]; } + /// Strain's quiet-waking level is a baseline too, and [rescanRecent] has to + /// re-price the recent days when ANY of their windows moves — a count and a + /// median of the whole series do not see that: each day is priced on the + /// median of the days strictly BEFORE it, so a value can change under one + /// day's window while the overall count and median stay put (e.g. yesterday + /// .15 → .25 in [.10,.10,.20,.20,.20,.15,.30] moves today's baseline .175 → + /// .20 and leaves both unchanged). So this fingerprints the DATED entries + /// themselves — every one that can sit in the window of a day the rescan + /// covers: the [_rescanWindowDays] + [_baselineWindowDays] stored days + /// before the newest. + /// + /// THE NEWEST IS LEFT OUT. It is normally today's, rewritten on every light + /// derive as the wake series grows, and the only windows it sits in are of + /// days after it — which are not finalized and are re-derived by the + /// ordinary passes anyway. Hashing it re-ran the whole rescan window on + /// every heavy pass for nothing. + String _quietFingerprint() { + final series = _series['quiet_hrr'] ?? const <_DatedValue>[]; + const span = _rescanWindowDays + _baselineWindowDays; + final end = series.isEmpty ? 0 : series.length - 1; + final from = end <= span ? 0 : end - span; + final dated = [ + for (var i = from; i < end; i++) + '${series[i].date}=${series[i].value.toStringAsFixed(6)}', + ].join(','); + return sha1.convert(utf8.encode(dated)).toString().substring(0, 12); + } + Map toArtifactJson() { double? avg(List xs) { if (xs.isEmpty) return null; @@ -2538,7 +2579,8 @@ class _BaselineHistoryCache { final readiness = values('readiness'); final signature = 'v$kAlgoVersion|n${rhr.length}|rhr${fmt(_median(rhr))}|rmssd${fmt(_median(rmssd))}' - '|temp${fmt(_median(temp))}|resp${fmt(_median(resp))}'; + '|temp${fmt(_median(temp))}|resp${fmt(_median(resp))}' + '|q${_quietFingerprint()}'; return { 'algo_version': kAlgoVersion, 'signature': signature, @@ -2751,6 +2793,23 @@ class DerivationEngine { } catch (_) {} try { + // FIRST, ahead of every early return below: it reads only stored bundles, + // so an install with bundles but no decoded rows (a restore) or with + // every day finalized still gets its quiet levels — the live gauge and + // session scoring need them as much as a derive does. And before the + // history snapshot, so this very sweep prices strain on them. One-shot + // (`compute_freshness`); a failure leaves the key unset and the next pass + // retries — never fatal to the derive. + _diag['stage'] = 'quiet_hrr_seed'; + try { + final seeded = await seedQuietHrrHistoryOnce( + manualRestingHr: profile.restingHrManual?.toDouble(), + ); + if (seeded > 0) _log('[derive] quiet_hrr seed: $seeded day(s)'); + } catch (e) { + _log('[derive] quiet_hrr seed failed (will retry): $e'); + } + _diag['stage'] = 'scope'; final scope = await _deriveScope(heavy: heavy, force: force); _diag ..['scope_days'] = scope.targetDays.length @@ -2776,8 +2835,17 @@ class DerivationEngine { ]; if (todoDays.isEmpty) { _log('derive: all days finalized — nothing to do'); + // A day still owed a strain score is unfinalized, so it is normally in + // the todo set; re-score it here too rather than leave it a pass. + final rescored = + await _quietLevelRepass(profile, dataNowSec, scope.rawDays); + if (rescored > 0) { + await _refreshBaselines(); + await _runCrossDay(profile); + await _runNotifications(); + } await _pruneOldDecoded(scope.rawDays, dataNowSec); - return 0; + return rescored; } _diag['todo_days'] = todoDays.length; _diag['stage'] = 'history'; @@ -2866,6 +2934,10 @@ class DerivationEngine { } await runWithConcurrency(orderedDays, _deriveConcurrency, processDay); + // BEFORE the prune below: a day still owed a strain score is re-scored + // while its raw exists (and the prune holds it if this fails). + _diag['stage'] = 'quiet_repass'; + done += await _quietLevelRepass(profile, dataNowSec, scope.rawDays); // 4. Cross-day rollup + notifications (best-effort). if (done > 0) { @@ -3053,6 +3125,7 @@ class DerivationEngine { } await runWithConcurrency(orderedDays, _deriveConcurrency, processDay); + done += await _quietLevelRepass(profile, dataNowSec); // A SELECTED re-analyze that happens to cover the whole raw history, with // every day resolved, is a full restage by any other name — it re-derived // every day under the current timezone, so it clears the travel hold too. @@ -4073,6 +4146,201 @@ class DerivationEngine { ); } + /// `compute_freshness` key prefix (``) marking a day + /// whose strain abstained for want of a quiet-waking level that prior days + /// could still supply. DURABLE on purpose: the day is held unfinalized and + /// its raw is held from the prune (see [_pruneOldDecoded]) until a pass + /// actually scores it — so a failed second pass, or the app being killed + /// between passes, leaves it for the next run instead of losing it. + static const String kQuietRepassKeyPrefix = 'quiet_repass:'; + + /// Test seam: awaited at the start of [_quietLevelRepass] (with null) and + /// before each day it re-derives. A throw simulates the pass failing — or, + /// at the start, the process dying before it ran. Null in production. + @visibleForTesting + static Future Function(String? day)? debugQuietRepassHook; + + /// Test seam: awaited just before a day's offloaded second half. A throw + /// lands exactly where a second-half timeout does. Null in production. + @visibleForTesting + static Future Function(String day)? debugSecondHalfHook; + + /// Of the days still owed a strain score ([kQuietRepassKeyPrefix]), the ones + /// that have a personal quiet level now — the days [_quietLevelRepass] + /// re-derives. + /// + /// A sweep prices every day on ONE frozen history snapshot. On a fresh + /// install whose first sync brings a week, that snapshot holds no `quiet_hrr` + /// at all, so every day abstains on strain — while each writes its own level + /// for the days after it. A day whose level merely MOVED is not here: that + /// is ordinary baseline drift, refreshed by [rescanRecent] like every other. + @visibleForTesting + static List quietLevelRepassDays( + Iterable owedDays, { + required List Function(String day) after, + }) => [ + for (final day in owedDays) + if (ana.personalQuietWakingHrr(after(day)).present) day, + ]; + + /// Whether [bundle] (day [date], about to be persisted) MAY be owed a strain + /// re-score — deliberately an OVER-approximation, with no counting. + /// + /// Owed when strain abstained ON THE QUIET LEVEL (the `need_baseline` rung, + /// last in the gate order, so no other input was missing) and ANY earlier + /// day could have changed that: one whose raw is not finalized yet (a sweep + /// derives newest-first and concurrently, so it may simply not be done) or + /// one with a stored level (which may have landed after this day's + /// snapshot). Deciding it exactly — "will the prior days reach three?" — + /// meant counting across reads that concurrent day commits interleave with, + /// and a miscount there finalized and pruned a day still owed a score. + /// + /// Race-free as ordered: raw days are never pruned mid-sweep and levels are + /// only ever added, and a day's level lands in the SAME transaction as its + /// row (`putDayResult`). So an earlier day either still reads as unfinalized + /// raw below, or its level is already visible to the read after it. + /// + /// Over-marking costs a held raw day (under the [_maxRawHoldDays] floor) and + /// a cheap check per run — never a number. + Future _quietRepassOwed(String date, Map bundle) async { + if (!quietBlocked(bundle)) return false; + final finalized = await LocalDb.finalizedDayIds(kAlgoVersion); + final rawDays = (await LocalDb.decodedRecTsMaxByDay()).keys; + if (rawDays.any((d) => d.compareTo(date) < 0 && !finalized.contains(d))) { + return true; + } + await debugQuietBetweenReads?.call(); + return (await LocalDb.trailingSeriesValues('quiet_hrr', 1, before: date)) + .isNotEmpty; + } + + /// PROOF that no pass can ever give [date] a quiet level from the data in + /// hand — the one thing besides a scored derive that may clear a marker. + /// Without it a day that can never reach [ana.quietHrrMinDays] (a cold + /// start's days 2–3, or a user whose days are all too short to yield a + /// level) stays owed until its raw ages out, and every later day is marked + /// in turn because the one before it is still unfinalized. + /// + /// Counts, so it orders its reads where a concurrent commit cannot slip + /// through: the DERIVED set first, then the levels. An earlier day that + /// commits after the first read is counted as still pending; one that + /// committed before it has its level (written in the same transaction as + /// its row) visible to the second. Partial rows are not "derived" — they + /// never wrote their series — so they count as pending too. + Future _quietLevelUnreachable(String date) async { + final derived = await LocalDb.dayResultIds(kAlgoVersion); + await debugQuietBetweenReads?.call(); + final prior = await LocalDb.trailingSeriesValues( + 'quiet_hrr', ana.quietHrrWindowDays, before: date); + final valid = prior + .where((q) => q.isFinite && q > 0 && q <= ana.maxQuietHrr) + .length; + if (valid >= ana.quietHrrMinDays) return false; + final pending = [ + for (final d in (await LocalDb.decodedRecTsMaxByDay()).keys) + if (d.compareTo(date) < 0 && !derived.contains(d)) d, + ].length; + return valid + pending < ana.quietHrrMinDays; + } + + /// Test seam onto [_quietLevelUnreachable]. + @visibleForTesting + Future debugQuietLevelUnreachable(String date) => + _quietLevelUnreachable(date); + + /// Strain abstained for want of a quiet-waking level and nothing else. + @visibleForTesting + static bool quietBlocked(Map bundle) { + if ((bundle['scalars'] as Map?)?['strain'] != null) return false; + final note = (bundle['absent_notes'] as Map?)?['strain']; + return note is String && note.startsWith('need_baseline:'); + } + + /// Test seam: awaited between the two reads of [_quietRepassOwed] and of + /// [_quietLevelUnreachable], so a test can commit a concurrent earlier day + /// exactly there. + @visibleForTesting + static Future Function()? debugQuietBetweenReads; + + /// Test seam onto [_quietRepassOwed]. + @visibleForTesting + Future debugQuietRepassOwed( + String date, Map bundle) => + _quietRepassOwed(date, bundle); + + /// Days still owed a strain re-score, from their durable markers. + static Future> _quietRepassOwedDays() async => { + for (final k in await LocalDb.computeFreshnessKeys(kQuietRepassKeyPrefix)) + k.substring(kQuietRepassKeyPrefix.length), + }; + + /// The second pass behind [quietLevelRepassDays]: reload the history once + /// and re-derive the owed days that have a level now, raw still on disk. One + /// pass converges — a day's own `quiet_hrr` reads no history, so the + /// reloaded snapshot holds every level the first pass produced. A day it + /// scores clears its own marker ([_derivePreparedDay]); one it fails keeps + /// it, stays unfinalized with its raw held, and is retried next run. + /// [rawDays] (every day with substrate left) retires markers for days whose + /// raw is gone, and so does a night the prune already cut into (the + /// [_maxRawHoldDays] floor can): re-deriving it would replace a full-night + /// result with a truncated one, exactly as [rescanDayIds] refuses to. Either + /// way the day's raw has aged out and its abstention is final. + /// + /// Held, like [rescanRecent], while a timezone change is suspected: these + /// days are bucketed in the CURRENT zone, so re-deriving them would relabel + /// and overwrite the nights that hold protects. They stay owed (unfinalized, + /// raw held) until "Re-analyze data" clears it. + /// + /// [rawDays] defaults to every day with substrate, read only when a marker + /// exists. Returns how many days it re-derived, so the caller's cross-day + /// refresh runs for them. + Future _quietLevelRepass( + Profile profile, + int dataNowSec, [ + Iterable? rawDays, + ]) async { + var done = 0; + try { + final owed = await _quietRepassOwedDays(); + if (owed.isEmpty) return 0; + if (await _timezoneTravelSuspected()) { + _log('quiet repass: possible timezone change — held'); + return 0; + } + final prunedBeforeSec = await LocalDb.getCursorInt(_prunedBeforeCursor); + final withRaw = { + for (final d in rawDays ?? (await LocalDb.decodedRecTsMaxByDay()).keys) + if (!windowTruncatedByPrune(d, prunedBeforeSec)) d, + }; + for (final d in owed.difference(withRaw)) { + await LocalDb.deleteComputeFreshness('$kQuietRepassKeyPrefix$d'); + } + await debugQuietRepassHook?.call(null); + final after = await _BaselineHistoryCache.load(); + final days = quietLevelRepassDays( + owed.intersection(withRaw), + after: (d) => after.valuesBefore('quiet_hrr', d), + )..sort((a, b) => b.compareTo(a)); // newest first, like the sweep + if (days.isEmpty) return 0; + _log('derive: re-scoring ${days.length} day(s) owed a quiet level'); + await runWithConcurrency(days, _deriveConcurrency, (dayId) async { + try { + await debugQuietRepassHook?.call(dayId); + final prepared = await _prepareTargetDay(dayId); + if (prepared != null && !prepared.daySub.isEmpty) { + await _derivePreparedDay(prepared, profile, dataNowSec, after); + done++; + } + } catch (e) { + _log('quiet repass day $dayId FAILED (kept owed, raw held): $e'); + } + }); + } catch (e) { + _log('quiet repass failed (days kept owed, raw held): $e'); + } + return done; + } + // ── baseline-dirty recent rescan ───────────────────────────────────────────── /// Re-derive the recent (≤ raw-retention) window — INCLUDING finalized days — @@ -4854,6 +5122,7 @@ class DerivationEngine { stepSpans: stepSpans, dynFloorG: dynFloorG, dynHistoryDays: dynHistory.length, + quietHrrHistory: history.valuesBefore('quiet_hrr', day.date), savedSessions: savedSessions, wristOffSpans: wristOffSpans, chargingSpans: chargingSpans, @@ -4876,6 +5145,7 @@ class DerivationEngine { tonightSleepOnsetSec: day.tonightSleepOnsetSec, tonightSleepOffsetSec: day.tonightSleepOffsetSec, ); + await debugSecondHalfHook?.call(day.date); final blocks = await _runDayBlocksCancellable(blocksInput, _perDayTimeout); @@ -5033,6 +5303,34 @@ class DerivationEngine { } } + // OWED A STRAIN SCORE ([kQuietRepassKeyPrefix]). Marked conservatively + // ([_quietRepassOwed]), BEFORE the row, so no interruption can leave an + // unmarked, prunable day still waiting on a score. CLEARED ONLY ON PROOF + // (after the row, below): a derive that completed BOTH halves — no + // timeout, no carried-forward previous row — and is either no longer + // blocked on the quiet level or provably never can be + // ([_quietLevelUnreachable]). A second half that failed leaves `trimp_net`/ + // `quiet_hrr` unwritten even when the pipeline's strain came back, so it + // proves nothing. While marked the day is never locked, by age or by a + // carried-forward row. An import force-finalizes regardless: it has no + // raw to re-score from, and is never marked. + final quietKey = '$kQuietRepassKeyPrefix${day.date}'; + final quietSettled = !forceFinalize && + secondHalfOk && + (!quietBlocked(bundle) || await _quietLevelUnreachable(day.date)); + var quietMarked = false; + if (!forceFinalize) { + if (!quietSettled && + (await LocalDb.computeFreshness(quietKey) != null || + await _quietRepassOwed(day.date, bundle))) { + quietMarked = true; + } + if (quietMarked) { + await LocalDb.putComputeFreshness(quietKey, '{}'); + effectiveFinalized = false; + } + } + final scalars = (bundle['scalars'] as Map?)?.cast() ?? const {}; double? sc(String k) => (scalars[k] as num?)?.toDouble(); @@ -5094,6 +5392,14 @@ class DerivationEngine { // previous version's daytime-RHR strain left behind, not keep it. 'strain': sc('strain'), 'trimp': sc('trimp'), + // This day's own quiet-waking level: the series LATER days' strain is + // priced against (median of the trailing 28, strictly before). Never + // written for an imported day — imports carry no 1 Hz wake series. + 'quiet_hrr': sc('quiet_hrr'), + // Banister TRIMP earned ABOVE the quiet level, exercise unoffset — the + // per-day load a CTL/ATL training-load read takes, where gross `trimp` + // would count being awake as training. + 'trimp_net': sc('trimp_net'), // `strain_effort`, `spo2` and `odi_per_hour` used to be listed here. // Nothing in the tree ever produced them (12 rows, 0 values per key on // a real install), so they were three permanently-null series with a @@ -5162,13 +5468,15 @@ class DerivationEngine { 'hr_ceiling_bpm': sc('hr_ceiling_bpm'), }, ); + // AFTER the row: retired only once the row that proves it is durable. + if (quietSettled) await LocalDb.deleteComputeFreshness(quietKey); // NOTE: the sweep's `history` snapshot is deliberately NOT updated here. // See _BaselineHistoryCache — mutating the shared snapshot mid-sweep is the // duplicate-day pollution bug, and each day already derives its own // date-bounded window from the frozen snapshot. _log( 'derived ${day.date} v$kAlgoVersion ' - '(sleep=${day.sleepOffsetSec > day.sleepOnsetSec}, final=$finalized)', + '(sleep=${day.sleepOffsetSec > day.sleepOnsetSec}, final=$effectiveFinalized)', ); await _maybeFreezeHeadlineReadiness(day, dataNowSec, sc('readiness')); // Learn the step-counter calibration from finalized days only (a partial @@ -5565,6 +5873,10 @@ class DerivationEngine { // not a window (see [maxBefore]), and strictly before today so a day is // never banded on a ceiling its own session set. m['observed_hr_ceiling_bpm'] = history.maxBefore('hr_ceiling_bpm', date); + // The quiet-waking levels strain is priced against — the SAME window the + // second half's recompute gets (`_DayBlocksInput.quietHrrHistory`), so the + // two cannot score the day on different levels. + m['quiet_hrr_history'] = history.valuesBefore('quiet_hrr', date); return m; } @@ -6506,6 +6818,10 @@ class DerivationEngine { /// Onset of tonight's sleep when it began before midnight, 0 = none. See /// `PreparedDerivationDay.tonightSleepOnsetSec`. int tonightSleepOnsetSec = 0, + /// Prior days' `quiet_hrr`, strictly before this one (see + /// [_BaselineHistoryCache.valuesBefore]). Fewer than three valid values + /// and strain abstains. + List quietHrrHistory = const [], }) { final wake = _buildWakeDayFeatures( daySub, @@ -6519,6 +6835,7 @@ class DerivationEngine { restingHr: restingHr, dynFloorG: dynFloorG, stepSpans: stepSpans, + quietHrrHistory: quietHrrHistory, ); // ACTIVE ENERGY WORKOUT-GAP CREDIT. `wake['calories']` above is built // ENTIRELY from `daySub.hr` — the day's own continuous 1 Hz trace — and @@ -6609,6 +6926,13 @@ class DerivationEngine { // a daytime "resting" HR, and `if (strain != null)` kept exactly those. final strain = (wake['strain'] as num?)?.toDouble(); scMap?['strain'] = strain; + // Same rule for the two figures that ride with it: a re-derive that now + // abstains must erase what the last one wrote. `quiet_hrr` is this day's + // own quiet level — it feeds LATER days' strain baselines, never this + // day's — and `trimp_net` is the load earned above it, the per-day input + // a training-load (CTL/ATL) read wants instead of gross TRIMP. + scMap?['quiet_hrr'] = (wake['quiet_hrr'] as num?)?.toDouble(); + scMap?['trimp_net'] = (wake['trimp_net'] as num?)?.toDouble(); final calories = (wake['calories'] as num?)?.toDouble(); if (calories != null) scMap?['calories'] = calories; final steps = (wake['steps'] as num?)?.toDouble(); @@ -7126,6 +7450,7 @@ class DerivationEngine { double? dynFloorG, List> stepSpans = const [], int tonightSleepOnsetSec = 0, + List quietHrrHistory = const [], }) { final activeMin = _activeMinutes(daySub, sleepOnsetSec, sleepOffsetSec); final wear = _wearBlock( @@ -7166,7 +7491,22 @@ class DerivationEngine { // say which strap measured this day. final hrMax = estimatedMaxHr(profile.ageYears, daySub.deviceFamily); final rhrForTrimp = restingHr ?? profile.restingHrManual?.toDouble(); + // THIS user's quiet-waking level, from PRIOR days only — the same list the + // pure pipeline got, through the same analytics call, so both halves price + // the day identically. + final quiet = ana.personalQuietWakingHrr(quietHrrHistory); + // TODAY's own quiet level — persisted for LATER days' baselines only. It + // never scores today (that would subtract today's own living from itself). + // Below `quietHrrTraitMinMinutes` wake minutes the day measured that + // window, not ordinary living, and contributes nothing. + final quietToday = (rhrForTrimp == null || hrMax == null) + ? null + : ana.dailyQuietWakingHrr(perMin, + restingHr: rhrForTrimp, + maxHr: hrMax, + minMinutes: ana.quietHrrTraitMinMinutes); double? strain; + ana.NetTrimp? net; // Why each absent activity figure is absent, in the order the gates below // apply. Absence is never a bare nothing here: the day carries its own // reason PER FIGURE so every caller can say what is missing instead of @@ -7193,6 +7533,8 @@ class DerivationEngine { ? needInputNote('resting_hr') : sex == null ? needInputNote('sex') + : !quiet.present + ? (quiet.note ?? kUnknownAbsenceNote) : null; // `wakeDayEnergy`'s own gates, named. It returns a bare null, so the reason // has to be reconstructed from the same inputs it reads — in its order. @@ -7229,33 +7571,32 @@ class DerivationEngine { // daytime HR when no sleep was detected: ~20 bpm high on the days it // fired, which shrinks the HR reserve and manufactures strain out of // sitting still. No resting HR of either kind now means NO STRAIN. - if (dayHrValid.isNotEmpty && rhrForTrimp != null && sex != null) { - final trimp = ana.banisterTrimp( + if (dayHrValid.isNotEmpty && + rhrForTrimp != null && + sex != null && + quiet.present) { + // Same sex normalisation the calorie path uses. This read `sex == 'f'` + // alone, so a profile stored as 'female' (which the profile screen can + // write) got female calorie coefficients and MALE TRIMP off the same + // field. Banister publishes only two constants, so `nonbinary` has + // nowhere else to go here. + final sexEnum = + _workoutSex(sex) == 'female' ? ana.Sex.female : ana.Sex.male; + // Banister TRIMP ABOVE this user's quiet level over `perMin`, the wake + // window. Exercise minutes (≥ 40 % HRR) are never offset by quiet ones, + // so a quiet afternoon cannot cancel a morning run. + net = ana.netTrimpAboveQuiet( perMin, restingHr: rhrForTrimp, maxHr: hrMax, - // Same sex normalisation the calorie path uses. This read `sex == 'f'` - // alone, so a profile stored as 'female' (which the profile screen can - // write) got female calorie coefficients and MALE TRIMP off the same - // field. Banister publishes only two constants, so `nonbinary` has - // nowhere else to go here. - sex: _workoutSex(sex) == 'female' ? ana.Sex.female : ana.Sex.male, + quietHrr: quiet.value!.hrr, + sex: sexEnum, ); - if (trimp.present && trimp.value != null) { - // `perMin` IS the wake window the TRIMP was accumulated over, so it - // sets the quiet-waking baseline that gets subtracted. Passing the - // observed length (not an assumed 24 h) is what stops a partial-wear - // day from being charged a full day's overhead. - final score = ana.strainScoreMetric( - trimp.value, - wakeMinutes: perMin.length.toDouble(), - // Reference level, not this user's — see onehz_pipeline's - // `strainMetric` for why, and edge#226 for the fix. - quietHrr: ana.quietWakingHrr, - female: _workoutSex(sex) == 'female', - ); - if (score.present) strain = score.value; - } + // The same map `strainScoreFromSeries` applies to the same NetTrimp — + // one pass, and the persisted `trimp_net` and `strain` cannot come + // from two different accumulations. (The calibrating confidence lives + // on the pipeline's envelope; this half writes a bare scalar.) + if (net != null) strain = ana.strainFromNetTrimp(net.net); // Every named input was there and the scorer still abstained. We do not // know why; saying so is the honest floor, and it is what the rule // "never a guessed cause" leaves when there is no cause to name. @@ -7340,6 +7681,8 @@ class DerivationEngine { 'active_min': activeMin, 'movement_min': movementMin, 'strain': strain, + 'quiet_hrr': quietToday, + 'trimp_net': net?.net, // Machine-readable reason `strain` is null (see `strainAbsent`). Null // when a strain WAS produced. 'strain_absent': strain == null ? strainAbsent : null, @@ -8652,6 +8995,7 @@ class DerivationEngine { stepSpans: inp.stepSpans, sessions: inp.savedSessions, tonightSleepOnsetSec: inp.tonightSleepOnsetSec, + quietHrrHistory: inp.quietHrrHistory, ); bundlePatch['daytime_hrv'] = _daytimeHrv(daySub, onset, offset, @@ -9391,6 +9735,10 @@ class _DayBlocksInput { /// How many trailing days backed [dynFloorG] — only for the cold-start note. final int dynHistoryDays; + /// Prior days' `quiet_hrr` (strictly before [date]) — the same window the + /// pure pipeline was handed, so both halves price strain on one level. + final List quietHrrHistory; + final List> savedSessions; /// The user's nap edits for this day, replayed over the detector's output. @@ -9443,6 +9791,7 @@ class _DayBlocksInput { this.stepSpans = const [], required this.dynFloorG, required this.dynHistoryDays, + this.quietHrrHistory = const [], required this.savedSessions, this.napEdits = const [], required this.wristOffSpans, diff --git a/lib/compute/manual_session.dart b/lib/compute/manual_session.dart index 4053aa997..5dabed301 100644 --- a/lib/compute/manual_session.dart +++ b/lib/compute/manual_session.dart @@ -251,33 +251,39 @@ List zoneMinutesFor( /// ceiling is a property of the strap that measured the window as well as the /// athlete's age (`estimatedMaxHr`), and this scorer is device-agnostic. Null /// — no age, or an uncalibrated/unstamped strap — abstains. +/// +/// [quietHrr] is THIS user's quiet-waking level for the day the window sits in +/// (`personalQuietLevelBefore`) — the level that day's own strain was priced +/// on, so a bout and its day are on one baseline. Null abstains: the 0.20 +/// population reference billed being awake as load for some users and erased +/// real workouts for others. [quietSettled] false only lowers confidence +/// inside analytics; the value is the same. double? strainFromPerMinuteHr( List perMinuteHr, { required Profile profile, required double? restingHr, required double? hrMax, + required double? quietHrr, + bool quietSettled = true, }) { final sex = profile.sex?.toLowerCase(); - if (perMinuteHr.isEmpty || hrMax == null || restingHr == null || sex == null) { + if (perMinuteHr.isEmpty || + hrMax == null || + restingHr == null || + sex == null || + quietHrr == null) { return null; } - final trimp = ana.banisterTrimp( + // Banister TRIMP above the quiet level over the window's own minutes, with + // exercise minutes never offset by quiet ones — the same analytics call the + // day headline makes, so "workout strain" and "daily strain" are one method. + final score = ana.strainScoreFromSeries( perMinuteHr, restingHr: restingHr, maxHr: hrMax, + quietHrr: quietHrr, sex: workoutSex(sex) == 'female' ? ana.Sex.female : ana.Sex.male, - ); - if (!trimp.present || trimp.value == null) return null; - // The window's own length is the baseline window: strain is the load earned - // ABOVE quiet waking, and the same sex constant has to price the baseline as - // priced the TRIMP or the subtraction is off by the male/female coefficient. - final score = ana.strainScoreMetric( - trimp.value, - wakeMinutes: perMinuteHr.length.toDouble(), - // Reference level, not this user's — see onehz_pipeline's - // `strainMetric` for why, and edge#226 for the fix. - quietHrr: ana.quietWakingHrr, - female: workoutSex(sex) == 'female', + quietSettled: quietSettled, ); return score.present ? score.value : null; } @@ -289,8 +295,8 @@ double? strainFromPerMinuteHr( /// [restingHr] is the nightly/user RHR; [profile] supplies age/weight/sex. /// /// Uses the SAME published methods as the day-level derivation so a manual -/// session and the day it sits in are on one scale: `ana.banisterTrimp` -> -/// `ana.strainScoreMetric` for strain, `ana.Calories.estimateBoutCalories` +/// session and the day it sits in are on one scale: `ana.strainScoreFromSeries` +/// against [quietHrr] (the session day's level) for strain, `ana.Calories.estimateBoutCalories` /// (Keytel 2005) for kcal. A missing anchor makes the dependent metric null. /// /// [hrMax] is THE ceiling for this window — `estimatedMaxHr(age, family)`, @@ -308,6 +314,8 @@ ManualSessionStats computeManualSessionStats({ required double? hrMax, double? restingHr, ana.HeartRateZoneSet? zoneSet, + double? quietHrr, + bool quietSettled = true, }) { if (hrTs.isEmpty || hrTs.length != hrBpm.length) { return const ManualSessionStats(); @@ -344,7 +352,11 @@ ManualSessionStats computeManualSessionStats({ final sex = profile.sex?.toLowerCase(); final strain = strainFromPerMinuteHr(perMin, - profile: profile, restingHr: restingHr, hrMax: hrMax); + profile: profile, + restingHr: restingHr, + hrMax: hrMax, + quietHrr: quietHrr, + quietSettled: quietSettled); double? calories; if (profile.hasCalorieAnchors && diff --git a/lib/compute/onehz_pipeline.dart b/lib/compute/onehz_pipeline.dart index 628de95c7..2b13ced49 100644 --- a/lib/compute/onehz_pipeline.dart +++ b/lib/compute/onehz_pipeline.dart @@ -293,6 +293,12 @@ class DayBundleInput { /// matches today's raw mean (the old z-vs-z series was a unit mismatch bug). final List skinTempAdcHistory; + /// Trailing per-day quiet-waking levels (`quiet_hrr`, %HRR fraction), + /// strictly before this day, oldest→newest. Strain is priced against their + /// median ([personalQuietWakingHrr]) — never against this day's own level, + /// which would subtract the day's own living from itself. + final List quietHrrHistory; + /// TS-03 — the highest heart rate the band has OBSERVED (held >=15 s with /// corroborating motion, `observed_max_hr.dart`) on any day STRICTLY BEFORE /// this one, or null when there is none. Not a physiological HRmax: if the @@ -350,6 +356,7 @@ class DayBundleInput { this.respHistory = const [], this.rmssdHistory = const [], this.skinTempAdcHistory = const [], + this.quietHrrHistory = const [], this.observedHrCeilingBpm, this.dayConfidence = 0, this.dayFlags = const [], @@ -381,6 +388,7 @@ class DayBundleInput { 'resp_history': respHistory, 'rmssd_history': rmssdHistory, 'skin_temp_adc_history': skinTempAdcHistory, + 'quiet_hrr_history': quietHrrHistory, 'observed_hr_ceiling_bpm': observedHrCeilingBpm, 'day_confidence': dayConfidence, 'day_flags': dayFlags, @@ -431,6 +439,7 @@ class DayBundleInput { respHistory: dbls('resp_history'), rmssdHistory: dbls('rmssd_history'), skinTempAdcHistory: dbls('skin_temp_adc_history'), + quietHrrHistory: dbls('quiet_hrr_history'), observedHrCeilingBpm: (m['observed_hr_ceiling_bpm'] as num?)?.toDouble(), dayConfidence: (m['day_confidence'] as num?)?.toDouble() ?? 0, dayFlags: strs('day_flags'), @@ -823,6 +832,11 @@ Map deriveDayBundle(Map inputJson) { // TRIMP, HR zones, and calories so all three see the same wake series). final wakeHr = _perMinuteWakeSeries(d); final perMin = [for (final p in wakeHr) p.hr]; + // THIS user's quiet-waking level: the median of the trailing days' own + // levels, strictly before today. Absent below three days — strain then + // abstains with the baseline grammar rather than pricing being awake at a + // population constant. + final quiet = personalQuietWakingHrr(d.quietHrrHistory); // ── WHY the activity family is absent, named AT THE GATE THAT CAUSED IT ──── // // These four figures — strain/TRIMP, the zone minutes, the ceiling they were @@ -857,6 +871,8 @@ Map deriveDayBundle(Map inputJson) { ? needInputNote('resting_hr') : sex == null ? needInputNote('sex') + : !quiet.present + ? (quiet.note ?? kUnknownAbsenceNote) : null; final caloriesAbsentNote = perMin.isEmpty ? needInputNote('wake_hr') @@ -950,26 +966,25 @@ Map deriveDayBundle(Map inputJson) { } } - // HEADLINE STRAIN = 0–21 map of the TRIMP earned ABOVE the quiet-waking - // baseline; raw TRIMP kept as a detail. `perMin` is the wake window the TRIMP - // was accumulated over, so it sets the baseline that gets subtracted. - final rawTrimp = trimp.present ? trimp.value : null; - final strainMetric = strainScoreMetric( - rawTrimp, - wakeMinutes: perMin.isEmpty ? null : perMin.length.toDouble(), - // THE REFERENCE LEVEL, NOT THIS USER'S (edge#226 is still open). analytics - // stopped defaulting the quiet-waking level so every caller has to state - // which one it means; `quietWakingHrr` is the constant the anchor table was - // generated at, so passing it reproduces the strain this app ships today - // and nobody's number moves on this commit. The real level is - // `dailyQuietWakingHrr` fed through a rolling personal median — a trait, - // not a day, and the workout scorers need the same one the day uses or a - // bout subtracts its own effort away. That plumbing is edge#226. - // ponytail: population constant, swap for the rolling personal median when - // edge#226 lands — see the same comment at the other four call sites. - quietHrr: quietWakingHrr, - female: workoutSex(sex) == 'female', - ); + // HEADLINE STRAIN = 0–21 map of the TRIMP earned ABOVE this user's + // quiet-waking level, exercise minutes (≥ 40 % HRR) never offset by quiet + // ones; raw TRIMP kept as a detail. Same analytics arithmetic as the curve + // below and the engine's recompute, so the three cannot disagree. + final sexEnum = workoutSex(sex) == 'female' ? Sex.female : Sex.male; + final strainMetric = strainAbsentNote != null || !quiet.present + ? Metric.absent( + tier: Tier.estimate, + inputs_used: const ['hr_1hz', 'profile', 'quiet_waking_hrr_history'], + note: strainAbsentNote ?? kUnknownAbsenceNote, + ) + : strainScoreFromSeries( + perMin, + restingHr: rhrForTrimp, + maxHr: hrMax, + quietHrr: quiet.value!.hrr, + sex: sexEnum, + quietSettled: quiet.value!.settled, + ); // ── curve series for the UI ──────────────────────────────────────────────── final hrCurve = _downsampleHr(d.dayTsSec, d.dayHr); @@ -1002,6 +1017,7 @@ Map deriveDayBundle(Map inputJson) { restingHr: rhrForTrimp, maxHr: hrMax, sex: sex, + quietHrr: quiet.value?.hrr, ); final zoneTimeline = zoneSet == null ? const >[] @@ -1802,50 +1818,25 @@ List> _strainCurve( required double? restingHr, required double? maxHr, required String? sex, + required double? quietHrr, }) { - if (wakeHr.isEmpty || - restingHr == null || - maxHr == null || - maxHr <= restingHr || - sex == null) { - return const []; - } - // Banister's sex constants, via the ONE shared weighting factor. This used to - // inline `exp(b·hrr)` and drop the 0.64/0.86 scale coefficient entirely, so - // the curve accumulated a TRIMP 1.5625× the day's own — the curve and the - // headline were never on the same scale. It matters more now: the headline - // subtracts a baseline priced with `banisterY`, so a curve accumulating - // without it would be netted against an allowance from a different formula. - final female = workoutSex(sex) == 'female'; - final reserve = maxHr - restingHr; - var trimp = 0.0; - var wakeMin = 0.0; - final out = >[]; - for (final p in wakeHr) { - var hrr = (p.hr - restingHr) / reserve; - if (hrr < 0) hrr = 0; - if (hrr > 1) hrr = 1; - trimp += hrr * StrainScorer.banisterY(hrr, female: female); - // The baseline grows with the wake window ALREADY elapsed, so the curve - // stays flat through quiet waking and climbs only on real effort — rather - // than charging a whole day's allowance against the first minute. - wakeMin += 1; - out.add({ - 't': p.tsSec, - 'v': _round( - strainScore( - trimp, - wakeMinutes: wakeMin, - // Reference level, not this user's — see onehz_pipeline's - // `strainMetric` for why, and edge#226 for the fix. - quietHrr: quietWakingHrr, - female: female, - ), - 2, - ), - }); - } - return out; + if (wakeHr.isEmpty || sex == null) return const []; + // The headline's own arithmetic, minute by minute (`strainCurveFromSeries`), + // so the last point IS the headline and exercise already banked never falls + // back out through a quiet afternoon. Null — no level, or bad anchors — + // leaves the curve empty, which the strain screen explains from the note. + final curve = strainCurveFromSeries( + [for (final p in wakeHr) p.hr], + restingHr: restingHr, + maxHr: maxHr, + quietHrr: quietHrr, + sex: workoutSex(sex) == 'female' ? Sex.female : Sex.male, + ); + if (curve == null) return const []; + return [ + for (var i = 0; i < wakeHr.length; i++) + {'t': wakeHr[i].tsSec, 'v': _round(curve[i], 2)}, + ]; } double? _mean(List xs) { diff --git a/lib/compute/quiet_level.dart b/lib/compute/quiet_level.dart new file mode 100644 index 000000000..fb6b69e4b --- /dev/null +++ b/lib/compute/quiet_level.dart @@ -0,0 +1,29 @@ +// THE personal quiet-waking level for scoring anything OUTSIDE a derive sweep +// — a saved/edited session, the on-read re-score, the live gauge, the strain +// rescale. A day's own derive reads the same window from its sweep snapshot +// (`_BaselineHistoryCache.valuesBefore('quiet_hrr', date)`); this is that read +// against the database, so a session and the day it sits in are priced on one +// level. + +import 'package:openstrap_analytics/onehz.dart' as ana; + +import '../data/db.dart'; + +/// The median of the trailing 28 measured `quiet_hrr` days strictly before +/// [dayLabel] (a LOCAL day label). Absent — never a stand-in — below three +/// days, or when the read fails. +Future> personalQuietLevelBefore( + String dayLabel, +) async { + try { + return ana.personalQuietWakingHrr( + await LocalDb.trailingSeriesValues( + 'quiet_hrr', + ana.quietHrrWindowDays, + before: dayLabel, + ), + ); + } catch (_) { + return ana.personalQuietWakingHrr(const []); // abstains, never a stand-in + } +} diff --git a/lib/compute/quiet_level_seed.dart b/lib/compute/quiet_level_seed.dart new file mode 100644 index 000000000..d844f26bc --- /dev/null +++ b/lib/compute/quiet_level_seed.dart @@ -0,0 +1,105 @@ +// ONE-SHOT SEED — `quiet_hrr` history for installs that predate it. +// +// Strain is priced against the median of the trailing days' own quiet-waking +// level (`quiet_hrr`), and abstains below three of them. A derive writes that +// series going forward, but raw 1 Hz substrate is pruned `rawRetentionDays` +// behind the data edge, so a re-derive reaches only the last few days: without +// this every existing user would start at zero prior levels and see strain go +// blank for days. +// +// It does not need raw. Every stored bundle still carries the day's per-minute +// wake HR in two halves: `series.strain_curve` has one point per WAKE minute +// (its `t` values are exactly the wake-minute keys), and `series.hr_curve` is +// the per-minute mean HR for every minute, rounded to whole bpm. Their join is +// the wake series the pipeline scored, to within ±0.5 bpm (≈0.004 HRR), so the +// seeded median is the user's own measurement, not an estimate of it. Days +// whose curve was dropped by the v63 strain rescale yield nothing and are +// skipped; the series then fills naturally. + +import 'dart:isolate'; + +import 'package:openstrap_analytics/onehz.dart' as ana; + +import '../data/db.dart'; +import '../data/series_codec.dart'; + +/// `compute_freshness` key marking the seed as applied. +const String kQuietHrrSeedKey = 'quiet_hrr_seed_v1'; + +/// Pure: this stored day's quiet-waking HRR, or null. [payload] is a decoded +/// bundle (`SeriesCodec.decodePayloadJson`, so curves are `[{t, v}]` lists). +/// [manualRestingHr] stands in only for a missing NOCTURNAL resting HR — the +/// same pair, in the same order, the day's own TRIMP is anchored on. +double? quietHrrFromStoredBundle( + Map payload, { + double? manualRestingHr, +}) { + final scalars = payload['scalars'] is Map + ? payload['scalars'] as Map + : const {}; + final series = payload['series'] is Map + ? payload['series'] as Map + : const {}; + final rhr = (scalars['rhr_nocturnal'] as num?)?.toDouble() ?? manualRestingHr; + final hrMax = (payload['max_hr_used'] as num?)?.toDouble(); + final wake = { + for (final p in (series['strain_curve'] as List? ?? const [])) + if (p is Map && p['t'] is num) (p['t'] as num).toInt(), + }; + final hr = [ + for (final p in (series['hr_curve'] as List? ?? const [])) + if (p is Map && + p['t'] is num && + p['v'] is num && + wake.contains((p['t'] as num).toInt())) + (p['v'] as num).toDouble(), + ]; + return ana.dailyQuietWakingHrr( + hr, + restingHr: rhr, + maxHr: hrMax, + minMinutes: ana.quietHrrTraitMinMinutes, + ); +} + +/// Seed `quiet_hrr` for the trailing window of stored days, once. +/// +/// Skips imported days (another vendor's numbers, no 1 Hz wake series), +/// skipped and partial rows (a partial day must not seed the trait, the same +/// reason `putDayResult` skips its series write), and days that already have a +/// value — a derive's own measurement is never overwritten. Decoding and the +/// median run off the UI isolate. Returns the number of days written. +Future seedQuietHrrHistoryOnce({double? manualRestingHr}) async { + if (await LocalDb.computeFreshness(kQuietHrrSeedKey) != null) return 0; + final rows = await LocalDb.recentDayResults(ana.quietHrrWindowDays + 3); + final imported = await LocalDb.importedDates(); + final have = { + for (final r in await LocalDb.metricSeries('quiet_hrr')) + r['date'] as String, + }; + final todo = <(String, String)>[ + for (final r in rows) + if (r['day_id'] is String && + r['payload_json'] is String && + !imported.contains(r['day_id']) && + !have.contains(r['day_id']) && + (r['skipped'] as num?)?.toInt() != 1 && + (r['partial'] as num?)?.toInt() != 1) + (r['day_id'] as String, r['payload_json'] as String), + ]; + final levels = todo.isEmpty + ? const <(String, double)>[] + : await Isolate.run(() => [ + for (final (date, json) in todo) + if (SeriesCodec.decodePayloadJson(json) case final p?) + if (quietHrrFromStoredBundle(p, + manualRestingHr: manualRestingHr) + case final q?) + (date, q), + ]); + for (final (date, q) in levels) { + await LocalDb.putMetricSeriesValue(date, 'quiet_hrr', q); + } + await LocalDb.putComputeFreshness(kQuietHrrSeedKey, '{"done":true}'); + return levels.length; +} diff --git a/lib/compute/strain_backfill.dart b/lib/compute/strain_backfill.dart index 8afc2a28f..2184a729b 100644 --- a/lib/compute/strain_backfill.dart +++ b/lib/compute/strain_backfill.dart @@ -13,10 +13,11 @@ // logs "no substrate (raw pruned) — kept" and keeps the old row — so a // kAlgoVersion bump alone can only ever fix the last few days. // -// It does not need raw. Strain is a pure function of (TRIMP, wake minutes, -// sex); `metric_series` stores `trimp` and the stored bundle carries -// `series.strain_curve`, which is ONE POINT PER WAKE MINUTE — literally the -// series the pipeline handed the scorer. +// It does not need raw. The lump-form strain is a pure function of (TRIMP, +// wake minutes, sex, the user's quiet-waking level); `metric_series` stores +// `trimp` and `quiet_hrr`, and the stored bundle carries `series.strain_curve`, +// which is ONE POINT PER WAKE MINUTE — literally the series the pipeline +// handed the scorer. // // THE WAKE WINDOW IS READ, NOT RECONSTRUCTED. This used to derive it as // `worn_min − tst_min`, which is a different quantity in both directions: the @@ -35,6 +36,7 @@ import 'package:openstrap_analytics/onehz.dart' as ana; import '../data/day_label.dart'; import '../data/db.dart'; import 'derivation_engine.dart' show kAlgoVersion, rawRetentionDays; +import 'quiet_level.dart'; /// `compute_freshness` key marking the rescale as already applied. Bumped with /// the algo version so a future rescale is a new one-shot rather than a no-op. @@ -62,22 +64,32 @@ class StrainBackfillResult { /// Rebuild one day's headline strain from its stored TRIMP and the wake window /// the pipeline actually priced it over ([wakeMinutes] = `strain_curve.length`). /// -/// Returns null when the day cannot be rescaled — no TRIMP to rescale from, or -/// no wake window to price the baseline over. A day that cannot be rescaled is -/// LEFT ALONE: an un-rescalable day must not silently become 0, which is a -/// number, not an absence. +/// Returns null when the day cannot be rescaled — no TRIMP to rescale from, no +/// wake window to price the baseline over, or no [quietHrr] (this user's +/// quiet-waking level) to price it at. A day that cannot be rescaled is LEFT +/// ALONE: an un-rescalable day must not silently become 0, which is a number, +/// not an absence. +/// +/// THE LUMP FORM, deliberately. Everywhere else strain is scored from the +/// per-minute series with exercise minutes protected +/// (`ana.strainScoreFromSeries`); these days have only their TRIMP and the +/// wake-minute count left, so the stored-TRIMP form is the only one possible. double? rescaledStrain({ required double? trimp, required double? wakeMinutes, required bool female, + required double? quietHrr, }) { - if (trimp == null || wakeMinutes == null || wakeMinutes <= 0) return null; + if (trimp == null || + wakeMinutes == null || + wakeMinutes <= 0 || + quietHrr == null) { + return null; + } return ana.strainScore( trimp, wakeMinutes: wakeMinutes, - // Reference level, not this user's — see onehz_pipeline's - // `strainMetric` for why, and edge#226 for the fix. - quietHrr: ana.quietWakingHrr, + quietHrr: quietHrr, female: female, ); } @@ -102,6 +114,13 @@ Future backfillStrainScale({ return none; } + // This user's quiet level, resolved once. Without one there is nothing + // honest to rescale against, so return WITHOUT marking done: the one-shot + // runs again once three days of `quiet_hrr` exist. + final quiet = await personalQuietLevelBefore(LocalDb.localDayLabelNow()); + final quietHrr = quiet.value?.hrr; + if (quietHrr == null) return none; + final trimpBy = await _byDate('trimp'); // The DATA EDGE is the newest day on disk, matching how the pruner measures @@ -177,6 +196,7 @@ Future backfillStrainScale({ series is Map ? series['strain_curve'] : null, )?.toDouble(), female: female, + quietHrr: quietHrr, ); if (next == null) { skipped++; diff --git a/lib/data/db.dart b/lib/data/db.dart index ed24a5b2c..6dbd63e26 100644 --- a/lib/data/db.dart +++ b/lib/data/db.dart @@ -10258,6 +10258,25 @@ class LocalDb { return rows.isEmpty ? null : rows.first; } + /// Every `compute_freshness` key starting with [prefix] — for per-day + /// markers (``), which are one row each so concurrent + /// day derives never read-modify-write a shared list. + static Future> computeFreshnessKeys(String prefix) async { + final db = await instance; + final rows = await db.query( + 'compute_freshness', + columns: ['key'], + where: 'substr(key, 1, ?) = ?', + whereArgs: [prefix.length, prefix], + ); + return {for (final r in rows) r['key'] as String}; + } + + static Future deleteComputeFreshness(String key) async { + final db = await instance; + await db.delete('compute_freshness', where: 'key = ?', whereArgs: [key]); + } + /// Bookkeeping key for the one-time walk in [reencodeLegacyDayResults]. static const String kReencodeCursorKey = 'series_reencode'; diff --git a/lib/data/local_repository_impl.dart b/lib/data/local_repository_impl.dart index 40b016a79..48153ca58 100644 --- a/lib/data/local_repository_impl.dart +++ b/lib/data/local_repository_impl.dart @@ -23,6 +23,7 @@ import '../compute/hr_max.dart'; import '../compute/manual_session.dart'; import '../compute/onehz_pipeline.dart' show kUnknownAbsenceNote, needInputNote; import '../compute/profile.dart'; +import '../compute/quiet_level.dart'; import 'package:openstrap_protocol/openstrap_protocol.dart' as proto; import 'package:openstrap_analytics/onehz.dart' as ana; @@ -2895,6 +2896,8 @@ class LocalRepositoryImpl extends LocalRepository { // row, every import, every raw replay) — is a refusal, not gen4 by default. final deviceFamily = (existing?['device_family'] as String?) ?? await _windowDeviceFamily(startTs, endTs); + final quiet = await personalQuietLevelBefore( + dayLabelOf(DateTime.fromMillisecondsSinceEpoch(startTs * 1000))); final stats = computeManualSessionStats( hrTs: hrTs, @@ -2907,6 +2910,10 @@ class LocalRepositoryImpl extends LocalRepository { // calorie anchor; the two are named separately because they can now be // different ceilings. zoneSet: _zoneSetFor(deviceFamily, await _zoneAnchors()), + // The quiet level of the session's OWN local day (its start), so the + // bout is priced on the baseline that day's strain was. + quietHrr: quiet.value?.hrr, + quietSettled: quiet.value?.settled ?? true, ); final row = buildManualSessionRow( @@ -3015,6 +3022,9 @@ class LocalRepositoryImpl extends LocalRepository { _profileMaxHr(row['device_family'] as String?)?.toDouble(); final restingHrForSession = await _recentRestingHr() ?? profile.restingHrManual?.toDouble(); + // Same rule as the save path: the session's own local day's level. + final quiet = await personalQuietLevelBefore( + dayLabelOf(DateTime.fromMillisecondsSinceEpoch(startTs * 1000))); final stats = computeManualSessionStats( hrTs: [for (final e in hrRows) (e['rec_ts'] as num).toInt()], hrBpm: hrBpm, @@ -3023,6 +3033,8 @@ class LocalRepositoryImpl extends LocalRepository { restingHr: restingHrForSession, zoneSet: _zoneSetFor( row['device_family'] as String?, await _zoneAnchors()), + quietHrr: quiet.value?.hrr, + quietSettled: quiet.value?.settled ?? true, ); // The peak is smoothed inside `computeManualSessionStats` now — one // definition for the manual save, this re-score and the workout list diff --git a/lib/state/app_state.dart b/lib/state/app_state.dart index a2ad006af..3d1008eb2 100644 --- a/lib/state/app_state.dart +++ b/lib/state/app_state.dart @@ -49,6 +49,7 @@ import '../compute/manual_session.dart' show strainFromPerMinuteHr, supersededSuggestionIds; import '../compute/hr_max.dart'; import '../compute/profile.dart'; +import '../compute/quiet_level.dart'; import '../data/day_label.dart'; import '../data/journal_fields.dart' show JournalMetricValue, kJournalFieldsByKey; @@ -1662,6 +1663,11 @@ class AppState extends ChangeNotifier { Future debugReconcileOrphanedLiveWorkout() => _reconcileOrphanedLiveWorkout(); + /// Run the cross-day anchor read (nightly RHR, quiet-waking level) that + /// `_initSteps` awaits before the reconcile above. Tests only. + @visibleForTesting + Future debugRefreshNightlyRhr() => _refreshNightlyRhr(); + /// Feed one live accel frame through the live-pedometer path exactly as /// [_onLiveFrame] does, with the ingest wall-clock supplied by the caller. /// Tests only — lets a test replay a session's frames deterministically. @@ -6337,9 +6343,35 @@ class AppState extends ChangeNotifier { double? _observedCeilingBpm; List _rhr28 = const []; + /// FILL A GAP on [w] with the personal quiet-waking level for ITS OWN local + /// start day — the level the saved session is re-scored on, so the gauge + /// and the stored number agree, even for a session resumed after midnight. + /// Read fresh every time, never cached: a level read earlier can predate the + /// derive that wrote yesterday's `quiet_hrr`, or belong to another day. Same + /// rule as the resting HR: never move a level a running session was already + /// scored on. Absent below three measured days; the gauge then shows "—". + Future _fillLiveQuietLevel(LiveWorkoutState w) async { + if (w.quietHrr != null) return; + final lvl = + (await personalQuietLevelBefore(dayLabelOf(w.startTime))).value; + if (lvl == null || w.quietHrr != null || !identical(activeWorkout, w)) { + return; + } + w.quietHrr = lvl.hrr; + w.quietSettled = lvl.settled; + // Now, not at the next sample: a resumed session's restored series was + // scored before the level landed. + w._scoreStrain(); + notifyListeners(); + } + Future _refreshNightlyRhr() async { try { _observedCeilingBpm = (await LocalDb.observedHrCeiling())?.bpm; + // BEFORE the resting-HR early return below: a user with no nightly + // series still has a quiet level once three days are measured. + final live = activeWorkout; + if (live != null) await _fillLiveQuietLevel(live); _rhr28 = await LocalDb.trailingSeriesValues('rhr', 28); final vals = await LocalDb.trailingSeriesValues('rhr', 7); if (vals.isEmpty) return; @@ -6453,6 +6485,8 @@ class AppState extends ChangeNotifier { manualZoneLowerBpm: manualZoneBoundsFromProfile(user), ), restingHr: _liveRestingHr, + // No quiet level yet: the refresh above reads it fresh for this + // session's day and fills the gap within the first samples. ); // The workout is now an owner (HR; plus IMU for a foreground gait type — // the live step count rides the 100 Hz stream). AFTER the assignment: the @@ -6680,6 +6714,9 @@ class AppState extends ChangeNotifier { manualZoneLowerBpm: manualZoneBoundsFromProfile(user), ), restingHr: _liveRestingHr, + // The quiet level is filled by the refresh below, from the + // session's OWN start day — after a relaunch past midnight that + // is not today. ); // Restore the per-second tallies a PREVIOUS process snapshotted // before it was killed — without this, strain/calories/zone @@ -7550,6 +7587,16 @@ class LiveWorkoutState { /// fraction of a second after it started. double? restingHr; + /// This user's quiet-waking level the strain score is priced above + /// (`personalQuietLevelBefore`). Mutable for the same reason as + /// [restingHr]: [AppState._refreshNightlyRhr] fills it when the read lands, + /// and only fills a gap. Null keeps strain absent. + double? quietHrr; + + /// Whether [quietHrr] rests on a settled week of days; only the confidence + /// moves with it, never the number. + bool quietSettled; + /// Per-minute mean HR, the unit Banister TRIMP weights. Live HR arrives at /// 1 Hz, so it is folded into the current minute here rather than kept as /// thousands of raw samples. @@ -7624,6 +7671,8 @@ class LiveWorkoutState { this.hrMax, this.zoneSet, this.restingHr, + this.quietHrr, + this.quietSettled = true, }) : _hrPeak = RollingMaxHr(age: age), idleWatch = WorkoutIdleWatch(startedAt: startTime); @@ -7683,17 +7732,25 @@ class LiveWorkoutState { // ONE strain method across the app (see strainFromPerMinuteHr). Null when // an anchor is missing — the gauge shows "—" rather than a number built on // an invented HRmax or resting HR. + _scoreStrain(); + + // Calories re-score off the same series, through the same estimator the + // substrate re-score uses, so both live figures on the gauge mean the same + // thing the finished session will. + _scoreCalories(); + } + + /// Re-score [strain] from the per-minute series — see the note in + /// [accrueHr]. Null as well while the quiet level has not landed. + void _scoreStrain() { strain = strainFromPerMinuteHr( perMinuteHr(), profile: profile, restingHr: restingHr, hrMax: hrMax, + quietHrr: quietHrr, + quietSettled: quietSettled, ); - - // Calories re-score off the same series, through the same estimator the - // substrate re-score uses, so both live figures on the gauge mean the same - // thing the finished session will. - _scoreCalories(); } /// Restore per-second tallies persisted mid-session by a PREVIOUS process, @@ -7735,12 +7792,7 @@ class LiveWorkoutState { ..clear() ..addAll(secondsByBpm); maxHrSeen = maxHrSeenIn; - strain = strainFromPerMinuteHr( - perMinuteHr(), - profile: profile, - restingHr: restingHr, - hrMax: hrMax, - ); + _scoreStrain(); _scoreCalories(); } } diff --git a/test/app_state_regressions_test.dart b/test/app_state_regressions_test.dart index 9f1ee89de..399802083 100644 --- a/test/app_state_regressions_test.dart +++ b/test/app_state_regressions_test.dart @@ -901,7 +901,14 @@ void main() { 'resting_hr': 55, }; + // A settled week of quiet-waking levels. Live strain is priced on the + // user's own level, which the reconcile's anchor refresh fills in after + // the restore — awaited here, so the restored series is re-scored on it. + for (var d = 1; d <= 7; d++) { + await LocalDb.putMetricSeriesValue('2020-01-0$d', 'quiet_hrr', 0.20); + } await app.debugReconcileOrphanedLiveWorkout(); + await app.debugRefreshNightlyRhr(); final w = app.activeWorkout; expect(w?.workoutId, id); @@ -989,6 +996,73 @@ void main() { expect(await LocalDb.session(id), isNull); }); }); + + // ── the live gauge prices a session on ITS OWN day's quiet level ───────── + group('live quiet level is read for the session\'s own day', () { + // Seven prior days at 0.20: every day after 2020-01-07 has a settled + // level; 2020-01-03 has only two days behind it. + Future seedLevels() async { + for (var d = 1; d <= 7; d++) { + await LocalDb.putMetricSeriesValue('2020-01-0$d', 'quiet_hrr', 0.20); + } + } + + test('a new session is filled with a level read fresh for its day', + () async { + await seedLevels(); + final app = AppState.forTesting(); + addTearDown(app.dispose); + app.startWorkout(workoutId: 'fresh-level', type: 'run'); + final w = app.activeWorkout!; + expect(w.quietHrr, isNull, + reason: 'nothing cached — the refresh reads it for this day'); + await app.debugRefreshNightlyRhr(); + expect(w.quietHrr, 0.20); + await app.stopWorkout(); + }); + + test('a level a running session was scored on never moves', () async { + await seedLevels(); + final app = AppState.forTesting(); + addTearDown(app.dispose); + app.startWorkout(workoutId: 'pinned-level', type: 'run'); + final w = app.activeWorkout!..quietHrr = 0.33; + await app.debugRefreshNightlyRhr(); + expect(w.quietHrr, closeTo(0.33, 1e-12)); + await app.stopWorkout(); + }); + + test('a resumed session is priced on its START day, not today', () async { + await seedLevels(); + final app = AppState.forTesting(); + addTearDown(app.dispose); + // Started on a day with only two prior levels: its own day abstains, + // even though today has a settled level cached. + final early = LiveWorkoutState( + startTime: DateTime(2020, 1, 3, 23, 50), + targetKcal: 300, + workoutId: 'resumed-early', + type: 'run', + ); + app.activeWorkout = early; + await app.debugRefreshNightlyRhr(); + expect(early.quietHrr, isNull); + + // Started on a day with five prior levels: present, but calibrating — + // not today's settled week. + final later = LiveWorkoutState( + startTime: DateTime(2020, 1, 6, 23, 50), + targetKcal: 300, + workoutId: 'resumed-later', + type: 'run', + ); + app.activeWorkout = later; + await app.debugRefreshNightlyRhr(); + expect(later.quietHrr, 0.20); + expect(later.quietSettled, isFalse); + app.activeWorkout = null; + }); + }); } class _ConnectedEngine extends BleEngine { diff --git a/test/derive_day_window_test.dart b/test/derive_day_window_test.dart index be12e1c2e..a98a35d88 100644 --- a/test/derive_day_window_test.dart +++ b/test/derive_day_window_test.dart @@ -269,6 +269,12 @@ void main() { }); test('a real profile still produces strain and calories', () async { + // Strain is priced on the user's own prior quiet-waking levels and + // abstains below three of them — a settled week, so only the profile + // decides here. + for (var d = 1; d <= 7; d++) { + await LocalDb.putMetricSeriesValue('2026-04-0$d', 'quiet_hrr', 0.20); + } final got = await deriveWith( const Profile( ageYears: 34, diff --git a/test/live_strain_convergence_test.dart b/test/live_strain_convergence_test.dart index d5fac06e0..83f9837ad 100644 --- a/test/live_strain_convergence_test.dart +++ b/test/live_strain_convergence_test.dart @@ -43,6 +43,7 @@ void main() { profile: _profile, hrMax: 187.0, restingHr: 55, + quietHrr: 0.20, ); test('starts absent, not zero', () { @@ -96,6 +97,31 @@ void main() { expect(w.strain, isNull); }); + test('abstains until a quiet level lands, then fills the gap', () { + final w = LiveWorkoutState( + startTime: DateTime(2026, 1, 1, 9), + targetKcal: 300, + type: 'run', + age: 30, + profile: _profile, + hrMax: 187.0, + restingHr: 55, + ); + for (var i = 0; i < 120; i++) { + w.elapsed = Duration(seconds: i); + w.accrueHr(150); + } + // No level read yet — never the 0.20 population reference. + expect(w.strain, isNull); + // `_refreshNightlyRhr` fills the gap when the read lands; the next + // sample re-scores through it. + w.quietHrr = 0.20; + w.elapsed = const Duration(seconds: 120); + w.accrueHr(150); + expect(w.strain, isNotNull); + expect(w.strain, greaterThan(0)); + }); + test('off-skin zeros neither pollute a minute nor reset the score', () { final w = make(); for (var i = 0; i < 600; i++) { diff --git a/test/manual_session_test.dart b/test/manual_session_test.dart index cff456933..a1f07fd82 100644 --- a/test/manual_session_test.dart +++ b/test/manual_session_test.dart @@ -253,6 +253,7 @@ void main() { profile: _profile, hrMax: _hrMax, restingHr: 55, + quietHrr: 0.20, ); expect(s.isUnscored, isTrue); expect(s.hrSampleCount, 0); @@ -271,6 +272,7 @@ void main() { profile: _profile, hrMax: _hrMax, restingHr: null, + quietHrr: 0.20, ); // Banister carries RHR as a term in the formula — absent it, abstain. expect(s.strain, isNull); @@ -295,6 +297,7 @@ void main() { profile: const Profile(weightKg: 75, heightCm: 180, sex: 'm'), hrMax: null, restingHr: 55, + quietHrr: 0.20, ); expect(s.strain, isNull); expect(s.calories, isNull); @@ -312,6 +315,7 @@ void main() { profile: const Profile(ageYears: 30, heightCm: 180, sex: 'm'), hrMax: _hrMax, restingHr: 55, + quietHrr: 0.20, ); expect(s.calories, isNull); expect(s.strain, isNotNull); @@ -325,6 +329,7 @@ void main() { profile: _profile, hrMax: _hrMax, restingHr: 55, + quietHrr: 0.20, ); expect(s.isUnscored, isFalse); expect(s.avgHr, 150); @@ -343,6 +348,7 @@ void main() { profile: _profile, hrMax: _hrMax, restingHr: 55, + quietHrr: 0.20, ); // THE WHOLE POINT of the feature: correcting 25 min → 60 min must // actually move the number the athlete came here to fix. @@ -360,6 +366,7 @@ void main() { profile: _profile, hrMax: _hrMax, restingHr: 55, + quietHrr: 0.20, ); expect(s.avgHr, 150); expect(s.hrSampleCount, 60); @@ -375,6 +382,7 @@ void main() { profile: _profile, hrMax: _hrMax, restingHr: 55, + quietHrr: 0.20, ); expect(s.calories, wornOnly.calories); expect(s.strain, wornOnly.strain); @@ -391,7 +399,7 @@ void main() { test('a hard hour scores on-scale, nowhere near the old accrual', () { final perMin = List.filled(60, 150); final s = strainFromPerMinuteHr(perMin, - profile: _profile, restingHr: 55, hrMax: _hrMax)!; + profile: _profile, restingHr: 55, hrMax: _hrMax, quietHrr: 0.20)!; // The canonical figure for this effort. expect(s, closeTo(11.62, 0.05)); @@ -409,7 +417,7 @@ void main() { test('never exceeds 21, even for an absurdly long maximal effort', () { // 12 h pinned at 190 bpm — the old accrual would read into the hundreds. final s = strainFromPerMinuteHr(List.filled(720, 190), - profile: _profile, restingHr: 55, hrMax: _hrMax)!; + profile: _profile, restingHr: 55, hrMax: _hrMax, quietHrr: 0.20)!; expect(s, lessThanOrEqualTo(21.0)); }); @@ -418,7 +426,7 @@ void main() { List.filled(minutes, bpm), profile: _profile, restingHr: 55, - hrMax: _hrMax)!; + hrMax: _hrMax, quietHrr: 0.20)!; expect(at(60, 150), greaterThan(at(30, 150))); expect(at(60, 165), greaterThan(at(60, 150))); }); @@ -428,24 +436,55 @@ void main() { // No resting HR. expect( strainFromPerMinuteHr(perMin, - profile: _profile, restingHr: null, hrMax: _hrMax), + profile: _profile, restingHr: null, hrMax: _hrMax, quietHrr: 0.20), isNull); // No ceiling — no age, OR a strap with no calibrated HRmax (TS-03a: an // unknown/unstamped device family REFUSES rather than borrowing gen4's). expect( strainFromPerMinuteHr(perMin, - profile: _profile, restingHr: 55, hrMax: null), + profile: _profile, restingHr: 55, hrMax: null, quietHrr: 0.20), isNull); // No sex → no Banister weighting constant. expect( strainFromPerMinuteHr(perMin, profile: const Profile(ageYears: 30), restingHr: 55, - hrMax: _hrMax), + hrMax: _hrMax, quietHrr: 0.20), isNull); // No HR at all. expect( strainFromPerMinuteHr(const [], - profile: _profile, restingHr: 55, hrMax: _hrMax), + profile: _profile, restingHr: 55, hrMax: _hrMax, quietHrr: 0.20), + isNull); + }); + + test('abstains without this user\'s quiet-waking level', () { + // The 0.20 population reference used to stand in here. A session is + // priced on the quiet level of the day it sits in, or not at all. + final w = _flat(0, 45, 150); + final none = computeManualSessionStats( + hrTs: w.ts, + hrBpm: w.bpm, + profile: _profile, + hrMax: _hrMax, + restingHr: 55, + quietHrr: null, + ); + expect(none.strain, isNull); + // Everything else about the session is still scored. + expect(none.avgHr, 150); + expect(none.calories, isNotNull); + final priced = computeManualSessionStats( + hrTs: w.ts, + hrBpm: w.bpm, + profile: _profile, + hrMax: _hrMax, + restingHr: 55, + quietHrr: 0.20, + ); + expect(priced.strain, isNotNull); + expect( + strainFromPerMinuteHr(List.filled(60, 150), + profile: _profile, restingHr: 55, hrMax: _hrMax, quietHrr: null), isNull); }); @@ -458,9 +497,10 @@ void main() { profile: _profile, hrMax: _hrMax, restingHr: 55, + quietHrr: 0.20, ).strain; final direct = strainFromPerMinuteHr(hrPerMinute(w.ts, w.bpm), - profile: _profile, restingHr: 55, hrMax: _hrMax); + profile: _profile, restingHr: 55, hrMax: _hrMax, quietHrr: 0.20); expect(viaStats, direct); }); }); @@ -472,6 +512,7 @@ void main() { profile: _profile, hrMax: _hrMax, restingHr: 55, + quietHrr: 0.20, ); test('a new entry is done/manual with a start-keyed id', () { diff --git a/test/manual_workout_repo_test.dart b/test/manual_workout_repo_test.dart index 00f4b84fa..56b74b352 100644 --- a/test/manual_workout_repo_test.dart +++ b/test/manual_workout_repo_test.dart @@ -79,6 +79,12 @@ void main() { LocalDb.dbName = 'openstrap_manual_workout_test.db'; final dir = await databaseFactory.getDatabasesPath(); await databaseFactory.deleteDatabase(p.join(dir, LocalDb.dbName)); + // A settled week of quiet-waking levels before every window below: a + // session is priced on the quiet level of the day it sits in, and strain + // abstains without one. + for (var d = 1; d <= 7; d++) { + await LocalDb.putMetricSeriesValue('2023-01-0$d', 'quiet_hrr', 0.20); + } }); tearDownAll(() async { diff --git a/test/quiet_hrr_seed_test.dart b/test/quiet_hrr_seed_test.dart new file mode 100644 index 000000000..a995a9a05 --- /dev/null +++ b/test/quiet_hrr_seed_test.dart @@ -0,0 +1,151 @@ +// One-shot `quiet_hrr` seed for installs that predate the series. +// +// A re-derive only reaches the last few days (raw is pruned), so without a +// seed every existing user would start with zero prior quiet levels and see +// strain go blank. Every stored bundle still carries the per-minute wake HR as +// `strain_curve` (one point per wake minute) joined to `hr_curve`. + +import 'dart:convert'; + +import 'package:flutter_test/flutter_test.dart'; +import 'package:path/path.dart' as p; +import 'package:sqflite_common_ffi/sqflite_ffi.dart'; + +import 'package:openstrap_edge/compute/quiet_level.dart'; +import 'package:openstrap_edge/compute/quiet_level_seed.dart'; +import 'package:openstrap_edge/compute/strain_backfill.dart'; +import 'package:openstrap_edge/data/db.dart'; +import 'package:openstrap_edge/data/series_codec.dart'; + +/// 400 wake minutes at 62 bpm, then 400 sleep minutes at 50 that are in +/// `hr_curve` but not in `strain_curve` — so they must not enter the median. +Map _payload({ + bool withCurve = true, + double? rhrNocturnal = 55, +}) => + { + 'scalars': {'rhr_nocturnal': ?rhrNocturnal}, + 'max_hr_used': 183.5, + 'series': { + 'hr_curve': [ + for (var i = 0; i < 400; i++) {'t': i * 60, 'v': 62}, + for (var i = 400; i < 800; i++) {'t': i * 60, 'v': 50}, + ], + if (withCurve) + 'strain_curve': [ + for (var i = 0; i < 400; i++) {'t': i * 60, 'v': 0.0}, + ], + }, + }; + +void main() { + group('quietHrrFromStoredBundle — pure', () { + test('the wake minutes\' median HRR', () { + // 62 bpm against RHR 55 / HRmax 183.5 = 7/128.5. + expect(quietHrrFromStoredBundle(_payload()), closeTo(0.0545, 0.0005)); + }); + + test('no strain curve, no wake series — null', () { + expect(quietHrrFromStoredBundle(_payload(withCurve: false)), isNull); + }); + + test('no nocturnal RHR — the user-entered one anchors it', () { + final noRhr = _payload(rhrNocturnal: null); + expect(quietHrrFromStoredBundle(noRhr), isNull); + expect(quietHrrFromStoredBundle(noRhr, manualRestingHr: 55), + closeTo(0.0545, 0.0005)); + }); + + test('columnar curves decode to the same value', () { + final stored = + SeriesCodec.encodePayloadJson(jsonEncode(_payload())); + expect(stored, isNot(contains('"strain_curve":[{')), + reason: 'the codec actually stored a columnar curve'); + final decoded = SeriesCodec.decodePayloadJson(stored)!; + expect(quietHrrFromStoredBundle(decoded), closeTo(0.0545, 0.0005)); + }); + }); + + group('seedQuietHrrHistoryOnce — an existing install', () { + setUpAll(() async { + sqfliteFfiInit(); + databaseFactory = databaseFactoryFfi; + LocalDb.dbName = 'openstrap_quiet_hrr_seed_test.db'; + final dir = await databaseFactory.getDatabasesPath(); + await databaseFactory.deleteDatabase(p.join(dir, LocalDb.dbName)); + }); + + tearDownAll(() async { + await LocalDb.close(); + final dir = await databaseFactory.getDatabasesPath(); + await databaseFactory.deleteDatabase(p.join(dir, LocalDb.dbName)); + }); + + Future day( + String date, { + bool partial = false, + bool skipped = false, + bool imported = false, + }) => + LocalDb.putDayResult( + dayId: date, + algoVersion: 97, + payloadJson: SeriesCodec.encodePayloadJson(jsonEncode({ + 'date': date, + if (imported) 'imported': true, + ..._payload(), + })), + windowJson: '{}', + finalized: true, + partial: partial, + skipped: skipped, + source: imported ? 'whoop_export' : 'band', + series: {'strain': 5.0}, + ); + + Future> quietRows() async => { + for (final r in await LocalDb.metricSeries('quiet_hrr')) + r['date'] as String: (r['value'] as num).toDouble(), + }; + + test('seeds once, even with the strain rescale long since done', () async { + // The population this exists for: an install whose v63 strain rescale + // already ran, so `backfillStrainScale` returns at its freshness check. + await LocalDb.putComputeFreshness(kStrainRescaleKey, '{"done":true}'); + for (final d in ['2026-09-01', '2026-09-02', '2026-09-03', '2026-09-04']) { + await day(d); + } + await day('2026-09-05', partial: true); + await day('2026-09-06', skipped: true); + await day('2026-09-07', imported: true); + // A derive already measured this one — the seed must not overwrite it. + await day('2026-09-08'); + await LocalDb.putMetricSeriesValue('2026-09-08', 'quiet_hrr', 0.3); + + expect((await backfillStrainScale(female: false)).didWork, isFalse); + expect(await seedQuietHrrHistoryOnce(), 4); + + final rows = await quietRows(); + expect(rows.keys.toSet(), + {'2026-09-01', '2026-09-02', '2026-09-03', '2026-09-04', '2026-09-08'}); + for (final d in ['2026-09-01', '2026-09-02', '2026-09-03', '2026-09-04']) { + expect(rows[d], closeTo(0.0545, 0.0005), reason: d); + } + expect(rows['2026-09-08'], 0.3); + expect(await LocalDb.computeFreshness(kQuietHrrSeedKey), isNotNull); + + // …so the next day's strain has a level to be priced on. + final level = await personalQuietLevelBefore('2026-09-09'); + expect(level.present, isTrue); + expect(level.value!.days, 5); + }); + + test('a second run is a no-op', () async { + final before = await quietRows(); + // Even a newly stored day is left for its own derive to measure. + await day('2026-09-09'); + expect(await seedQuietHrrHistoryOnce(), 0); + expect(await quietRows(), before); + }); + }); +} diff --git a/test/quiet_level_engine_test.dart b/test/quiet_level_engine_test.dart new file mode 100644 index 000000000..923b330c1 --- /dev/null +++ b/test/quiet_level_engine_test.dart @@ -0,0 +1,555 @@ +// The quiet-waking level through the REAL engine: decoded 1 Hz rows → +// DerivationEngine.run/runDays → day_result + metric_series. +// +// Strain is priced on the median of the PRIOR days' own `quiet_hrr`, read from +// one frozen snapshot per sweep. These pin what that means on disk: a first +// sync of a whole week must not leave the week blank, the persisted rows are +// replaced (not kept) when a re-derive abstains, a day needs 360 wake minutes +// to contribute a level, and the one-shot history seed runs even when there +// is nothing to derive. + +import 'dart:async'; +import 'dart:convert'; + +import 'package:flutter_test/flutter_test.dart'; +import 'package:path/path.dart' as p; +import 'package:sqflite_common_ffi/sqflite_ffi.dart'; + +import 'package:openstrap_edge/compute/derivation_engine.dart'; +import 'package:openstrap_edge/compute/profile.dart'; +import 'package:openstrap_edge/compute/quiet_level_seed.dart'; +import 'package:openstrap_edge/data/db.dart'; +import 'package:openstrap_edge/data/series_codec.dart'; + +const _profile = Profile( + ageYears: 35, + sex: 'm', + weightKg: 75, + heightCm: 178, + restingHrManual: 55, +); + +int _localSec(String day, int hour) { + final d = DateTime.parse(day); + return DateTime(d.year, d.month, d.day, hour).millisecondsSinceEpoch ~/ 1000; +} + +/// [wakeMin] minutes of daytime 1 Hz rows from 09:00 local: 45 min at 130 bpm, +/// the rest at ~62, with enough wrist motion to read as awake. +Future _insertDay(String day, {int wakeMin = 400}) async { + final db = await LocalDb.instance; + final start = _localSec(day, 9); + final batch = db.batch(); + for (var i = 0; i < wakeMin * 60; i++) { + final ts = start + i; + batch.insert('decoded_onehz', { + 'device_id': '', + 'ts_ms': ts * 1000, + 'rec_ts': ts, + 'counter': ts, + 'hr': i < 45 * 60 ? 130 : 62 + (i % 3) - 1, + 'ax': 0.05 * ((i % 60) / 60.0), + 'ay': 0.05 * ((i % 30) / 30.0), + 'az': 0.98, + 'device_family': 'gen4', + }, conflictAlgorithm: ConflictAlgorithm.replace); + } + await batch.commit(noResult: true); +} + +Future _clear() async { + final db = await LocalDb.instance; + for (final t in const [ + 'decoded_onehz', + 'day_result', + 'metric_series', + 'metric_series_version', + 'compute_freshness', + ]) { + await db.delete(t); + } + // The highest cutoff the raw prune has applied — carried across tests it + // would mark later fixtures' nights as already cut. + await db.delete('sync_cursor', + where: 'name = ?', whereArgs: ['decoded_pruned_before']); +} + +/// The stored `metric_series` row itself — `metricSeries` drops NULL values, +/// and a NULL row is exactly what an abstaining re-derive must leave. +Future>> _row(String day, String key) async { + final db = await LocalDb.instance; + return db.query('metric_series', + where: 'date = ? AND key = ?', whereArgs: [day, key]); +} + +/// Owed days right after a sweep: every day that can still reach three +/// levels ([mustOwe]) and, at most, the cold-start days before them. Those can +/// stay marked one pass longer — proving a day unreachable counts across two +/// reads, and an earlier day committing in between is counted on both sides, +/// the safe direction — and their next complete derive clears them. +Future _expectOwedAfterSweep( + List days, Iterable mustOwe) async { + final owed = await _owed(); + expect(owed, containsAll(mustOwe)); + expect(owed.difference(mustOwe.toSet()), + everyElement(isIn(days.sublist(1, 3)))); +} + +/// Days still owed a strain re-score (their durable markers). +Future> _owed() async => { + for (final k in await LocalDb.computeFreshnessKeys( + DerivationEngine.kQuietRepassKeyPrefix)) + k.substring(DerivationEngine.kQuietRepassKeyPrefix.length), +}; + +/// Decoded 1 Hz rows left on [day] (local calendar day). +Future _decodedRows(String day) async { + final db = await LocalDb.instance; + final d = DateTime.parse(day); + final start = _localSec(day, 0); + // Next local midnight from the calendar, not +86400 (DST). + final end = DateTime(d.year, d.month, d.day + 1).millisecondsSinceEpoch ~/ 1000; + final rows = await db.rawQuery( + 'SELECT COUNT(*) AS n FROM decoded_onehz WHERE rec_ts >= ? AND rec_ts < ?', + [start, end]); + return (rows.single['n'] as num).toInt(); +} + +Future _value(String day, String key) async { + final rows = await _row(day, key); + return rows.isEmpty ? null : (rows.single['value'] as num?)?.toDouble(); +} + +void main() { + TestWidgetsFlutterBinding.ensureInitialized(); + + setUpAll(() async { + sqfliteFfiInit(); + databaseFactory = databaseFactoryFfi; + LocalDb.dbName = 'openstrap_quiet_level_engine_test.db'; + final dir = await databaseFactory.getDatabasesPath(); + await databaseFactory.deleteDatabase(p.join(dir, LocalDb.dbName)); + }); + + tearDownAll(() async { + await LocalDb.close(); + final dir = await databaseFactory.getDatabasesPath(); + await databaseFactory.deleteDatabase(p.join(dir, LocalDb.dbName)); + }); + + test('the seed runs even with an empty decoded ledger', () async { + await _clear(); + // A restored install: bundles on disk, no 1 Hz rows, so run() returns at + // "no decoded data" — after the seed, not before it. + for (final day in const ['2026-02-01', '2026-02-02', '2026-02-03']) { + await LocalDb.putDayResult( + dayId: day, + algoVersion: kAlgoVersion, + payloadJson: SeriesCodec.encodePayloadJson(jsonEncode({ + 'date': day, + 'max_hr_used': 183.5, + 'scalars': {'rhr_nocturnal': 55}, + 'series': { + 'hr_curve': [ + for (var i = 0; i < 400; i++) {'t': i * 60, 'v': 62}, + ], + 'strain_curve': [ + for (var i = 0; i < 400; i++) {'t': i * 60, 'v': 0.0}, + ], + }, + })), + windowJson: '{}', + finalized: true, + source: 'band', + ); + } + expect(await DerivationEngine().run(_profile, heavy: true), 0); + expect(await LocalDb.computeFreshness(kQuietHrrSeedKey), isNotNull); + expect(await _value('2026-02-03', 'quiet_hrr'), closeTo(0.0545, 0.0005)); + }); + + // The tests below run real derives over 1 Hz rows — well past the default + // 30 s once the suite is loaded, hence their explicit budgets. + test('a first sync of a whole week does not leave it blank', () async { + await _clear(); + const days = [ + '2026-03-01', '2026-03-02', '2026-03-03', '2026-03-04', // + '2026-03-05', '2026-03-06', '2026-03-07', + ]; + for (final d in days) { + await _insertDay(d); + } + // ONE run, one frozen snapshot with no `quiet_hrr` in it at all. + await DerivationEngine().run(_profile, heavy: true); + + for (final d in days) { + expect(await _value(d, 'quiet_hrr'), isNotNull, reason: d); + } + // Days 1–3 have fewer than three prior levels: an honest cold start. + for (final d in days.take(3)) { + expect(await _value(d, 'strain'), isNull, reason: d); + } + // Day 4 on had three or more by the end of the sweep, and was re-scored + // on them before its raw could be pruned — finalized or not, never blank. + for (final d in days.skip(3)) { + expect(await _value(d, 'strain'), isNotNull, reason: d); + expect(await _value(d, 'trimp_net'), isNotNull, reason: d); + final row = await LocalDb.dayResult(d); + final scalars = + (SeriesCodec.decodePayloadJson(row!['payload_json'])!['scalars'] + as Map); + expect(scalars['strain'], isNotNull, reason: '$d bundle'); + } + // Every day that could be scored was, and cleared its marker. Days 2–3 + // can never reach three levels; the next complete derive proves it and + // releases them. + await _expectOwedAfterSweep(days, const []); + await DerivationEngine().run(_profile, heavy: true); + expect(await _owed(), isEmpty); + }, timeout: const Timeout(Duration(minutes: 5))); + + // A failed second pass, or the app dying before it ran, must not cost the + // days it owes: they stay unfinalized, their raw is held from the prune, and + // the NEXT run — a fresh engine, a light pass — scores them. + for (final (label, failAt) in const [ + ('a failed second pass', 'day'), + ('the process dying between the passes', 'start'), + ]) { + test('$label leaves the owed days for the next run', () async { + await _clear(); + const days = [ + '2026-06-01', '2026-06-02', '2026-06-03', '2026-06-04', // + '2026-06-05', '2026-06-06', '2026-06-07', + ]; + for (final d in days) { + await _insertDay(d); + } + DerivationEngine.debugQuietRepassHook = (day) async { + if ((day == null) == (failAt == 'start')) throw StateError('injected'); + }; + addTearDown(() => DerivationEngine.debugQuietRepassHook = null); + await DerivationEngine().run(_profile, heavy: true); + DerivationEngine.debugQuietRepassHook = null; + + final finalized = await LocalDb.finalizedDayIds(kAlgoVersion); + await _expectOwedAfterSweep(days, days.skip(3)); + for (final d in days.skip(3)) { + expect(await _value(d, 'strain'), isNull, reason: d); + expect(finalized, isNot(contains(d)), reason: '$d is not locked'); + } + // The prune ran and held every owed day's raw — day 4's included, + // which is past the 3-day retention on its own. + expect(await _decodedRows(days[3]), 400 * 60, reason: 'held whole'); + + await DerivationEngine().run(_profile); + for (final d in days.skip(3)) { + expect(await _value(d, 'strain'), isNotNull, reason: d); + } + expect(await _owed(), everyElement(isIn(days.sublist(1, 3)))); + await DerivationEngine().run(_profile, heavy: true); + expect(await _owed(), isEmpty); + }, timeout: const Timeout(Duration(minutes: 5))); + } + + test('persisted rows, replaced with NULL when the inputs go', () async { + await _clear(); + for (var d = 1; d <= 7; d++) { + await LocalDb.putMetricSeriesValue('2026-04-0$d', 'quiet_hrr', 0.20); + } + // Seeded already, so the run below does not try to backfill anything. + await LocalDb.putComputeFreshness(kQuietHrrSeedKey, '{"done":true}'); + const day = '2026-04-10'; + await _insertDay(day); + await DerivationEngine().runDays(_profile, {day}); + expect(await _value(day, 'quiet_hrr'), closeTo(0.0545, 0.002)); + expect(await _value(day, 'trimp_net'), greaterThan(0)); + expect(await _value(day, 'strain'), greaterThan(0)); + + // No resting HR of any kind now: no reserve, so no level, no net load, + // no strain — and the rows a previous derive wrote must not survive it. + await DerivationEngine().runDays( + const Profile(ageYears: 35, sex: 'm', weightKg: 75, heightCm: 178), + {day}); + for (final key in const ['quiet_hrr', 'trimp_net', 'strain']) { + final rows = await _row(day, key); + expect(rows, hasLength(1), reason: '$key row is written'); + expect(rows.single['value'], isNull, reason: '$key is erased'); + } + }, timeout: const Timeout(Duration(minutes: 5))); + + test('360 wake minutes contribute a level, 359 do not', () async { + await _clear(); + for (var d = 1; d <= 7; d++) { + await LocalDb.putMetricSeriesValue('2026-05-0$d', 'quiet_hrr', 0.20); + } + await LocalDb.putComputeFreshness(kQuietHrrSeedKey, '{"done":true}'); + await _insertDay('2026-05-10', wakeMin: 359); + await _insertDay('2026-05-11', wakeMin: 360); + await DerivationEngine().runDays(_profile, {'2026-05-10', '2026-05-11'}); + + // Guard: the wake series really is that long (no minute carved off). + for (final (day, n) in const [('2026-05-10', 359), ('2026-05-11', 360)]) { + final row = await LocalDb.dayResult(day); + final series = SeriesCodec.decodePayloadJson(row!['payload_json'])![ + 'series'] as Map; + expect((series['strain_curve'] as List).length, n, reason: day); + } + expect(await _value('2026-05-10', 'quiet_hrr'), isNull); + // …and still scored, on the prior days' level. + expect(await _value('2026-05-10', 'strain'), greaterThan(0)); + expect(await _value('2026-05-11', 'quiet_hrr'), closeTo(0.0545, 0.002)); + }, timeout: const Timeout(Duration(minutes: 5))); + + test('a second-half timeout on the re-score keeps the day owed', () async { + // The re-score's pipeline half DOES produce strain once the level exists; + // its second half (the only producer of `trimp_net`) times out, and the + // first pass's same-version row is carried forward. Strain present is not + // proof the day was scored: the marker stays, the day stays unlocked with + // its raw, and a later COMPLETE derive writes the net load. + // + // The first pass's second half times out too, so the row carried forward + // is that pass's own headline-only row — the recovery the engine can + // perform today. (Carrying a COMPLETE row's curves into a fresh pipeline + // bundle currently throws before anything is written, which keeps the + // marker by accident; it is not what this test is about.) + await _clear(); + const days = [ + '2026-07-01', '2026-07-02', '2026-07-03', '2026-07-04', // + '2026-07-05', '2026-07-06', '2026-07-07', + ]; + for (final d in days) { + await _insertDay(d); + } + final target = days[3]; + var calls = 0; + DerivationEngine.debugSecondHalfHook = (day) async { + // Both the first pass and the re-score time out on the second half. + if (day == target && ++calls <= 2) { + throw TimeoutException('injected second-half timeout'); + } + }; + addTearDown(() => DerivationEngine.debugSecondHalfHook = null); + await DerivationEngine().run(_profile, heavy: true); + DerivationEngine.debugSecondHalfHook = null; + expect(calls, 2, reason: 'the re-score reached the second half'); + + expect(await _value(target, 'strain'), isNull, + reason: 'a headline-only row never writes its series'); + final row = await LocalDb.dayResult(target); + expect( + (SeriesCodec.decodePayloadJson(row!['payload_json'])!['scalars'] + as Map)['strain'], + isNotNull, + reason: 'the re-score\'s pipeline half did score it'); + expect(await _owed(), contains(target)); + expect(await LocalDb.finalizedDayIds(kAlgoVersion), isNot(contains(target))); + expect(await _value(target, 'trimp_net'), isNull); + expect(await _decodedRows(target), 400 * 60, reason: 'held whole'); + + // Restart: a fresh engine's light pass completes it. + await DerivationEngine().run(_profile); + expect(await _value(target, 'trimp_net'), isNotNull); + expect(await _value(target, 'strain'), isNotNull); + expect(await _owed(), isNot(contains(target))); + }, timeout: const Timeout(Duration(minutes: 5))); + + // The race the old counting rule lost: it read the stored levels, then the + // derived days, and a third earlier day committing its level AND its row + // between those reads was counted by neither. The rule now asks only + // whether ANY earlier day exists — as unfinalized raw, or as a stored + // level — and a day's level lands in the same transaction as its row, so + // every point of that interleaving answers "owed". + test('owed holds at every point of a concurrent earlier commit', () async { + await _clear(); + final engine = DerivationEngine(); + const days = ['2026-08-01', '2026-08-02', '2026-08-03', '2026-08-04']; + final blocked = { + 'scalars': {'strain': null}, + 'absent_notes': {'strain': 'need_baseline:have=2,need=3'}, + }; + Future raw(String d) async { + final db = await LocalDb.instance; + final t = _localSec(d, 9); + await db.insert('decoded_onehz', { + 'device_id': '', + 'ts_ms': t * 1000, + 'rec_ts': t, + 'counter': t, + 'hr': 62, + 'ax': 0.0, + 'ay': 0.0, + 'az': 1.0, + 'device_family': 'gen4', + }); + } + + Future commit(String d) => LocalDb.putDayResult( + dayId: d, + algoVersion: kAlgoVersion, + payloadJson: '{"date":"$d"}', + windowJson: '{}', + finalized: true, + source: 'band', + series: {'quiet_hrr': 0.2}, + ); + + for (final d in days) { + await raw(d); + } + // Nothing earlier committed yet / two of three / all three committed. + expect(await engine.debugQuietRepassOwed(days[3], blocked), isTrue); + await commit(days[0]); + await commit(days[1]); + expect(await engine.debugQuietRepassOwed(days[3], blocked), isTrue); + await commit(days[2]); + expect(await engine.debugQuietRepassOwed(days[3], blocked), isTrue); + // The interleaving itself: every earlier day finalized, so the raw read + // finds nothing pending — then an earlier day commits its level and its + // row before the level read. It is seen. + await _clear(); + await raw(days[2]); + await LocalDb.putDayResult( + dayId: days[2], + algoVersion: kAlgoVersion, + payloadJson: '{"date":"${days[2]}"}', + windowJson: '{}', + finalized: true, + source: 'band', + ); + DerivationEngine.debugQuietBetweenReads = () => commit(days[2]); + addTearDown(() => DerivationEngine.debugQuietBetweenReads = null); + expect(await engine.debugQuietRepassOwed(days[3], blocked), isTrue); + DerivationEngine.debugQuietBetweenReads = null; + + // Even with every earlier raw day gone, a stored level still counts. + final db = await LocalDb.instance; + await db.delete('decoded_onehz'); + expect(await engine.debugQuietRepassOwed(days[3], blocked), isTrue); + + // Never owed: the first day (nothing before it), or a strain that is + // present or absent for another reason. + await raw(days[0]); + expect(await engine.debugQuietRepassOwed('2026-07-31', blocked), isFalse); + expect( + await engine.debugQuietRepassOwed(days[3], { + 'scalars': {'strain': 8.0}, + }), + isFalse); + expect( + await engine.debugQuietRepassOwed(days[3], { + 'scalars': {'strain': null}, + 'absent_notes': {'strain': 'need_input:resting_hr'}, + }), + isFalse); + }); + + test('an owed day whose night the prune already cut is retired, not re-scored', + () async { + // Only the 14-day hold floor can cut into an owed day's night. Re-deriving + // it then would replace a full-night result with a truncated one — the + // case the rescan refuses — so its abstention becomes final instead. + await _clear(); + const days = [ + '2026-09-01', '2026-09-02', '2026-09-03', '2026-09-04', // + '2026-09-05', '2026-09-06', '2026-09-07', + ]; + for (final d in days) { + await _insertDay(d); + } + // Leave days 4–7 owed: no second pass ran. + DerivationEngine.debugQuietRepassHook = (day) async { + if (day == null) throw StateError('injected'); + }; + addTearDown(() => DerivationEngine.debugQuietRepassHook = null); + await DerivationEngine().run(_profile, heavy: true); + DerivationEngine.debugQuietRepassHook = null; + await _expectOwedAfterSweep(days, days.skip(3)); + + // As if the floor had cut at day 5's midnight: days 2–5's nights (from + // the previous noon) are truncated, days 6–7's are whole. + await LocalDb.setCursor( + 'decoded_pruned_before', '${_localSec(days[4], 0)}'); + await DerivationEngine().run(_profile); + expect(await _owed(), isEmpty); + for (final d in days.sublist(1, 5)) { + expect(await _value(d, 'strain'), isNull, reason: '$d not re-scored'); + } + for (final d in days.skip(5)) { + expect(await _value(d, 'strain'), isNotNull, reason: d); + } + }, timeout: const Timeout(Duration(minutes: 5))); + + // Clearing a marker on "can never reach three" COUNTS, so its reads must be + // ordered against a concurrent commit: the derived set first, then the + // levels. Read the other way round, an earlier day committing its level and + // its row in between is seen by neither, and a day still owed a score is + // "proven" unreachable and finalized blank. + test('unreachable is never proven across a concurrent earlier commit', + () async { + await _clear(); + final engine = DerivationEngine(); + const days = ['2026-10-01', '2026-10-02', '2026-10-03', '2026-10-04']; + Future raw(String d) async { + final db = await LocalDb.instance; + final t = _localSec(d, 9); + await db.insert('decoded_onehz', { + 'device_id': '', + 'ts_ms': t * 1000, + 'rec_ts': t, + 'counter': t, + 'hr': 62, + 'ax': 0.0, + 'ay': 0.0, + 'az': 1.0, + 'device_family': 'gen4', + }); + } + + Future commit(String d) => LocalDb.putDayResult( + dayId: d, + algoVersion: kAlgoVersion, + payloadJson: '{"date":"$d","scalars":{"rhr":55}}', + windowJson: '{}', + finalized: true, + source: 'band', + rhr: 55, + series: {'quiet_hrr': 0.2}, + ); + + for (final d in days) { + await raw(d); + } + await commit(days[0]); + await commit(days[1]); + // Two levels in hand and a third day still deriving: reachable. + expect(await engine.debugQuietLevelUnreachable(days[3]), isFalse); + // …and still reachable when that third day commits between the reads. + DerivationEngine.debugQuietBetweenReads = () => commit(days[2]); + addTearDown(() => DerivationEngine.debugQuietBetweenReads = null); + expect(await engine.debugQuietLevelUnreachable(days[3]), isFalse); + DerivationEngine.debugQuietBetweenReads = null; + + // Proven only when nothing in hand can lift it: day 2 has one level + // before it and no earlier day still pending. + expect(await engine.debugQuietLevelUnreachable(days[1]), isTrue); + // Day 3: two levels before it, nothing earlier still pending. + expect(await engine.debugQuietLevelUnreachable(days[2]), isTrue); + }); + + test('a cold start\'s days 2–3 are released: they can never be scored', + () async { + await _clear(); + const days = ['2026-11-01', '2026-11-02', '2026-11-03']; + for (final d in days) { + await _insertDay(d); + } + await DerivationEngine().run(_profile, heavy: true); + await _expectOwedAfterSweep(days, const []); + await DerivationEngine().run(_profile, heavy: true); + expect(await _owed(), isEmpty); + for (final d in days) { + expect(await _value(d, 'strain'), isNull, reason: d); + expect(await _value(d, 'quiet_hrr'), isNotNull, reason: d); + } + }, timeout: const Timeout(Duration(minutes: 5))); +} diff --git a/test/readiness_baseline_pollution_test.dart b/test/readiness_baseline_pollution_test.dart index e8f332301..1917c429a 100644 --- a/test/readiness_baseline_pollution_test.dart +++ b/test/readiness_baseline_pollution_test.dart @@ -264,4 +264,72 @@ void main() { expect(window.last, 40.0 + 9); expect(window.every((v) => v < 40.0 + 10), isTrue); }); + + test('quiet_hrr rides the sweep: prior measured days only', () async { + // Strain is priced on the median of these. The day itself (its own level + // feeds LATER days, never itself) and an imported day are both out. + await LocalDb.putMetricSeriesValue('2026-12-01', 'quiet_hrr', 0.11); + await LocalDb.putMetricSeriesValue('2026-12-02', 'quiet_hrr', 0.12); + await LocalDb.putDayResult( + dayId: '2026-12-03', + algoVersion: 1, + payloadJson: '{"date":"2026-12-03","imported":true,"source":"whoop_export"}', + windowJson: '{}', + finalized: true, + source: 'whoop_export', + series: {'quiet_hrr': 0.13}, + ); + await LocalDb.putMetricSeriesValue('2026-12-04', 'quiet_hrr', 0.14); + final window = + (await debugSweepBaselineWindows('quiet_hrr', ['2026-12-04'])).single; + expect(window, [0.11, 0.12]); + // The non-sweep read (sessions, live gauge) sees the same window. + expect( + await LocalDb.trailingSeriesValues('quiet_hrr', 28, before: '2026-12-04'), + [0.11, 0.12]); + }); + + test('the rescan signature sees a quiet level move under ONE day\'s window', + () async { + // Chronological [.10, .10, .20, .20, .20, .15, .30]: changing the sixth + // day .15 → .25 keeps the count (7) and the overall median (.20) — all a + // count-and-median signature saw — but the seventh day's strictly-prior + // window goes from median .175 to .20, so that day's strain moves and the + // rescan must not skip it. + final db = await LocalDb.instance; + await db.delete('metric_series', where: 'key = ?', whereArgs: ['quiet_hrr']); + const dates = [ + '2027-01-01', '2027-01-02', '2027-01-03', '2027-01-04', // + '2027-01-05', '2027-01-06', '2027-01-07', + ]; + const values = [0.10, 0.10, 0.20, 0.20, 0.20, 0.15, 0.30]; + for (var i = 0; i < dates.length; i++) { + await LocalDb.putMetricSeriesValue(dates[i], 'quiet_hrr', values[i]); + } + final before = await debugBaselineSignature(); + expect(await debugBaselineSignature(), before, + reason: 'stable when nothing moved — a rescan costs nothing then'); + final whole = await debugBaselineWindow('quiet_hrr'); + final todayBefore = + (await debugSweepBaselineWindows('quiet_hrr', ['2027-01-07'])).single; + + await LocalDb.putMetricSeriesValue('2027-01-06', 'quiet_hrr', 0.25); + final wholeAfter = await debugBaselineWindow('quiet_hrr'); + final todayAfter = + (await debugSweepBaselineWindows('quiet_hrr', ['2027-01-07'])).single; + // What the old signature term read did not move… + expect(wholeAfter.length, whole.length); + expect(ana.median(wholeAfter), ana.median(whole)); + expect(ana.median(whole), closeTo(0.20, 1e-9)); + // …while the window the day is actually priced on did. + expect(ana.median(todayBefore), closeTo(0.175, 1e-9)); + expect(ana.median(todayAfter), closeTo(0.20, 1e-9)); + expect(await debugBaselineSignature(), isNot(before)); + + // The NEWEST level (normally today's, rewritten as its wake series grows) + // sits in no earlier day's window, so moving it must not trigger a rescan. + final stable = await debugBaselineSignature(); + await LocalDb.putMetricSeriesValue('2027-01-07', 'quiet_hrr', 0.12); + expect(await debugBaselineSignature(), stable); + }); } diff --git a/test/strain_quiet_level_test.dart b/test/strain_quiet_level_test.dart new file mode 100644 index 000000000..5a6bfbb3f --- /dev/null +++ b/test/strain_quiet_level_test.dart @@ -0,0 +1,203 @@ +// Strain is priced against THIS user's quiet-waking level, and quiet time never +// debits exercise. +// +// Every strain site used to subtract a population 0.20 HRR as the cost of +// being awake. A user who sits below it had real workouts cancelled (a 45-min +// run at 145 bpm scored 0.00 for someone at 0.10 HRR); a user above it scored +// 7–12/21 for doing nothing. The day now scores against the median of its own +// trailing `quiet_hrr` days (strictly before it), persists its own level for +// later days, and abstains with the baseline grammar until three exist. + +import 'package:flutter_test/flutter_test.dart'; + +import 'package:openstrap_edge/compute/derivation_engine.dart'; +import 'package:openstrap_edge/compute/onehz_pipeline.dart'; +import 'package:openstrap_edge/compute/profile.dart'; +import 'package:openstrap_edge/compute/substrate.dart'; + +// A fixed epoch second on a minute boundary, so N minutes of samples are N +// per-minute buckets. +const int _t0 = 1786699980; + +/// [bouts] as (minutes, bpm), one sample per second, all of it waking. +Substrate _sub(List<(int, int)> bouts) { + final ts = [], hr = []; + final ax = [], ay = [], az = []; + var i = 0; + for (final (mins, bpm) in bouts) { + for (var s = 0; s < mins * 60; s++, i++) { + ts.add(_t0 + i); + hr.add(bpm); + ax.add(0.01 * ((i % 7) - 3)); + ay.add(0.01 * ((i % 5) - 2)); + az.add(1.0); + } + } + final n = ts.length; + return Substrate( + tsSec: ts, + hr: hr, + rrTsMs: const [], + rrMs: const [], + ax: ax, + ay: ay, + az: az, + spo2Red: List.filled(n, 0), + spo2Ir: List.filled(n, 0), + skinTemp: List.filled(n, 0), + skinContact: List.filled(n, 0), + deviceFamily: 'gen4', + ); +} + +/// The `strain_resting_hr_source_test` day: 300 min at 62 bpm, 60 at 130. +Substrate _daySub() => _sub(const [(300, 62), (60, 130)]); + +/// RHR 50 / HRmax 187 (age 30): 45 min at 145, then 855 quiet minutes at 64. +Substrate _lowQuietSub() => _sub(const [(45, 145), (855, 64)]); + +const _profile = Profile(ageYears: 35, sex: 'm', weightKg: 75, heightCm: 178); +const _profile30 = Profile(ageYears: 30, sex: 'm', weightKg: 75, heightCm: 178); + +({Map scalars, Map bundle}) _run( + Substrate sub, { + required List quietHrrHistory, + Profile profile = _profile, + double restingHr = 55, +}) { + final scalars = {}; + final bundle = {}; + DerivationEngine.applyDayActivity( + bundle: bundle, + scalars: scalars, + daySub: sub, + profile: profile, + sleepOnsetSec: 0, + sleepOffsetSec: 0, + dayStartSec: sub.tsSec.first, + dayCalendarEndSec: sub.tsSec.last + 1, + dataNowSec: sub.tsSec.last + 1, + restingHr: restingHr, + quietHrrHistory: quietHrrHistory, + ); + return (scalars: scalars, bundle: bundle); +} + +/// The same day through the pure pipeline. No sleep, so its resting HR is the +/// profile's — set to the anchor the engine half is handed. +Map _pipeline( + Substrate sub, { + required List quietHrrHistory, + int age = 35, + int restingHr = 55, +}) => + deriveDayBundle(DayBundleInput( + date: '2026-08-15', + dayTsSec: sub.tsSec, + dayHr: sub.hr, + sleepTsSec: const [], + sleepHr: const [], + sleepRrTsMs: const [], + sleepRrMs: const [], + sleepSkinTemp: const [], + sleepJson: const {}, + hypnoStages: const [], + sleepOnsetSec: 0, + sleepOffsetSec: 0, + profile: { + 'age': age, + 'sex': 'm', + 'weight_kg': 75, + 'height_cm': 178, + 'resting_hr': restingHr, + }, + deviceFamily: 'gen4', + quietHrrHistory: quietHrrHistory, + ).toJson()); + +void main() { + group('the day is priced on its own trailing quiet level', () { + test('no prior levels: strain abstains with the baseline grammar', () { + final r = _run(_daySub(), quietHrrHistory: const []); + expect(r.scalars.containsKey('strain'), isTrue, + reason: 'written even when null, so a stale value is erased'); + expect(r.scalars['strain'], isNull); + expect((r.bundle['absent_notes'] as Map)['strain'], + 'need_baseline:have=0,need=3'); + }); + + test('seven prior levels: strain, the day\'s own level and net load', () { + final r = _run(_daySub(), quietHrrHistory: List.filled(7, 0.20)); + // The shipped lump form scored this fixture 2.85: 300 quiet minutes at + // 0.0545 HRR cancelled most of the hour at 130 bpm. + expect(r.scalars['strain'] as num, closeTo(8.55, 0.05)); + // 62 bpm against RHR 55 / HRmax 183.5 = 7/128.5. Persisted for LATER + // days; it never prices this one. + expect(r.scalars['quiet_hrr'] as num, closeTo(0.0545, 0.0005)); + expect(r.scalars['trimp_net'] as num, closeTo(57.46, 0.1)); + }); + + test('three prior levels score the same value, as "calibrating"', () { + final three = _run(_daySub(), quietHrrHistory: List.filled(3, 0.20)); + final seven = _run(_daySub(), quietHrrHistory: List.filled(7, 0.20)); + expect(three.scalars['strain'], seven.scalars['strain']); + + Map env(int n) => ((_pipeline(_daySub(), + quietHrrHistory: List.filled(n, 0.20))['clinical'] + as Map)['strain'] as Map).cast(); + expect(env(3)['confidence'], 0.45); + expect(env(3)['note'] as String, startsWith('calibrating')); + expect(env(7)['confidence'], 0.6); + }); + + test('re-deriving the same day is idempotent', () { + final runs = [ + for (var i = 0; i < 3; i++) + _run(_daySub(), quietHrrHistory: List.filled(7, 0.20)).scalars, + ]; + for (final key in ['strain', 'trimp_net', 'quiet_hrr']) { + expect(runs[1][key], runs[0][key], reason: key); + expect(runs[2][key], runs[0][key], reason: key); + } + }); + + test('a low-quiet user\'s 45-min run is not erased', () { + final own = _run(_lowQuietSub(), + quietHrrHistory: List.filled(7, 0.10), + profile: _profile30, + restingHr: 50); + final ref = _run(_lowQuietSub(), + quietHrrHistory: List.filled(7, 0.20), + profile: _profile30, + restingHr: 50); + expect(own.scalars['strain'] as num, closeTo(9.90, 0.05)); + expect(ref.scalars['strain'] as num, closeTo(9.38, 0.05)); + }); + + test('the curve ends on the engine\'s headline', () { + final engine = _run(_lowQuietSub(), + quietHrrHistory: List.filled(7, 0.10), + profile: _profile30, + restingHr: 50); + final bundle = _pipeline(_lowQuietSub(), + quietHrrHistory: List.filled(7, 0.10), age: 30, restingHr: 50); + final curve = ((bundle['series'] as Map)['strain_curve'] as List) + .cast(); + expect(curve, hasLength(900)); + expect((curve.last['v'] as num).toDouble(), + closeTo(engine.scalars['strain'] as num, 0.01)); + // Banked exercise never falls back out through the quiet afternoon. + final atEndOfRun = (curve[44]['v'] as num).toDouble(); + for (final p in curve.skip(44)) { + expect((p['v'] as num).toDouble(), greaterThanOrEqualTo(atEndOfRun)); + } + }); + + test('the pipeline abstains with the same reason', () { + final bundle = _pipeline(_daySub(), quietHrrHistory: const [0.2, 0.2]); + expect((bundle['absent_notes'] as Map)['strain'], + 'need_baseline:have=2,need=3'); + expect((bundle['series'] as Map)['strain_curve'], isEmpty); + }); + }); +} diff --git a/test/strain_rescale_backfill_test.dart b/test/strain_rescale_backfill_test.dart index cc9681aa5..ef7c8fd55 100644 --- a/test/strain_rescale_backfill_test.dart +++ b/test/strain_rescale_backfill_test.dart @@ -95,6 +95,7 @@ void main() { trimp: 177.80394321843846, wakeMinutes: 611, female: false, + quietHrr: 0.20, ); expect(s, isNotNull); expect(s!, closeTo(9.03, 0.05)); @@ -104,7 +105,7 @@ void main() { // Real bundle 2026-07-10: 23 steps, 135 wake minutes. expect( rescaledStrain(trimp: 23.416868457643158, wakeMinutes: 135, - female: false), + female: false, quietHrr: 0.20), 0.0, ); }); @@ -114,35 +115,45 @@ void main() { // −15..+22 min. It is small but it is not nothing: 22 minutes of extra // quiet-waking allowance is a real slice of a light day's headline. final measured = rescaledStrain( - trimp: 177.80394321843846, wakeMinutes: 611, female: false)!; + trimp: 177.80394321843846, wakeMinutes: 611, female: false, quietHrr: 0.20)!; final guessed = rescaledStrain( - trimp: 177.80394321843846, wakeMinutes: 633, female: false)!; + trimp: 177.80394321843846, wakeMinutes: 633, female: false, quietHrr: 0.20)!; expect(guessed, lessThan(measured)); }); test('abstains rather than guessing when an input is missing', () { // No TRIMP → nothing to rescale from. Must leave the day alone, not zero it. expect( - rescaledStrain(trimp: null, wakeMinutes: 611, female: false), + rescaledStrain(trimp: null, wakeMinutes: 611, female: false, quietHrr: 0.20), isNull, ); // No curve in the bundle → no window to price the baseline over. The old // code reached for `worn_min − tst_min` here; abstaining is the fix. expect( - rescaledStrain(trimp: 177.8, wakeMinutes: null, female: false), + rescaledStrain(trimp: 177.8, wakeMinutes: null, female: false, quietHrr: 0.20), isNull, ); expect( - rescaledStrain(trimp: 177.8, wakeMinutes: 0, female: false), + rescaledStrain(trimp: 177.8, wakeMinutes: 0, female: false, quietHrr: 0.20), + isNull, + ); + }); + + test('abstains without this user\'s quiet-waking level', () { + // The 0.20 population reference used to be passed here. No level, no + // rescale — the day keeps its old value rather than a stand-in one. + expect( + rescaledStrain( + trimp: 177.8, wakeMinutes: 611, female: false, quietHrr: null), isNull, ); }); test('sex changes the baseline, matching how the TRIMP was scored', () { final male = rescaledStrain( - trimp: 300, wakeMinutes: 900, female: false)!; + trimp: 300, wakeMinutes: 900, female: false, quietHrr: 0.20)!; final female = rescaledStrain( - trimp: 300, wakeMinutes: 900, female: true)!; + trimp: 300, wakeMinutes: 900, female: true, quietHrr: 0.20)!; // The female quiet-waking allowance is larger (0.86·e^0.334 vs // 0.64·e^0.384), so the same TRIMP nets less strain. expect(female, lessThan(male)); @@ -150,6 +161,23 @@ void main() { }); group('backfillStrainScale — the stored history', () { + test('waits, UNMARKED, until a quiet-waking level exists', () async { + await seedDay('2026-07-09', + trimp: 177.80394321843846, strain: 12.790964777435558, + wornMin: 827, tstMin: 216, wakeMin: 611); + final r = await backfillStrainScale(female: false); + expect(r.didWork, isFalse); + // Not marked done: the one-shot must run once a level exists. + expect(await LocalDb.computeFreshness(kStrainRescaleKey), isNull); + expect(await seriesValue('strain', '2026-07-09'), + closeTo(12.790964777435558, 1e-9)); + // A settled week at the reference level, so the rescaled values below + // stay comparable with the pure tests. + for (var d = 1; d <= 7; d++) { + await LocalDb.putMetricSeriesValue('2026-05-0$d', 'quiet_hrr', 0.20); + } + }); + test('rescales a raw-pruned historical day in series AND bundle', () async { // worn − tst would be 611 here too, but the curve is what counts and it // is seeded at 611 to keep the pinned 9.03 comparable with the pure test. diff --git a/test/strain_resting_hr_source_test.dart b/test/strain_resting_hr_source_test.dart index 8e2b74986..e6e47845b 100644 --- a/test/strain_resting_hr_source_test.dart +++ b/test/strain_resting_hr_source_test.dart @@ -76,6 +76,9 @@ Map _runDayActivity({ dayCalendarEndSec: sub.tsSec.last + 1, dataNowSec: sub.tsSec.last + 1, restingHr: restingHr, + // A settled week of prior quiet levels, so the resting-HR gate is the only + // thing deciding presence here (strain_quiet_level_test covers the level). + quietHrrHistory: List.filled(7, 0.20), ); return scalars; } diff --git a/test/tonight_sleep_test.dart b/test/tonight_sleep_test.dart index 3c278f540..68900dc69 100644 --- a/test/tonight_sleep_test.dart +++ b/test/tonight_sleep_test.dart @@ -60,6 +60,11 @@ double _strain({int tonight = 0}) { dataNowSec: sub.tsSec.last + 1, restingHr: 50, tonightSleepOnsetSec: tonight, + // Strain is priced on the user's prior quiet-waking levels and abstains + // without them. A level below this day's waking minutes (62 bpm ≈ 0.09 + // HRR), so they net positive and minutes asleep (≈ 0.015 HRR) would net + // that back down if they were still counted as waking. + quietHrrHistory: List.filled(7, 0.05), ); return (scalars['strain'] as num).toDouble(); } From d78f63685d81d65529cbc09f73cfa4230ce474ce Mon Sep 17 00:00:00 2001 From: Alex Wagner Date: Thu, 8 Oct 2026 14:54:11 +0200 Subject: [PATCH 2/5] strain: seed quiet levels on the live derive's ceiling The one-time quiet_hrr seed priced each stored day on that bundle's max_hr_used, while the live derive prices it on estimatedMaxHr at the current profile age. After a birthday or a profile edit the two are on different scales, and the trailing median mixes them. The seed now uses the same ceiling as the live derive; without an age it seeds nothing and stays armed, as the live derive also abstains. The live gauge's quiet-level fill gets its own guard, so a failure there cannot skip the resting-HR refresh. The strain rescale's comment now states why one current level prices every rescaled day. --- lib/compute/derivation_engine.dart | 1 + lib/compute/quiet_level_seed.dart | 20 +++++++++++++--- lib/compute/strain_backfill.dart | 5 +++- lib/state/app_state.dart | 10 ++++++-- test/quiet_hrr_seed_test.dart | 38 +++++++++++++++++++++++------- 5 files changed, 60 insertions(+), 14 deletions(-) diff --git a/lib/compute/derivation_engine.dart b/lib/compute/derivation_engine.dart index b84c6fbc3..194eebbc6 100644 --- a/lib/compute/derivation_engine.dart +++ b/lib/compute/derivation_engine.dart @@ -2803,6 +2803,7 @@ class DerivationEngine { _diag['stage'] = 'quiet_hrr_seed'; try { final seeded = await seedQuietHrrHistoryOnce( + ageYears: profile.ageYears, manualRestingHr: profile.restingHrManual?.toDouble(), ); if (seeded > 0) _log('[derive] quiet_hrr seed: $seeded day(s)'); diff --git a/lib/compute/quiet_level_seed.dart b/lib/compute/quiet_level_seed.dart index d844f26bc..935e872cd 100644 --- a/lib/compute/quiet_level_seed.dart +++ b/lib/compute/quiet_level_seed.dart @@ -22,6 +22,7 @@ import 'package:openstrap_analytics/onehz.dart' as ana; import '../data/db.dart'; import '../data/series_codec.dart'; +import 'hr_max.dart'; /// `compute_freshness` key marking the seed as applied. const String kQuietHrrSeedKey = 'quiet_hrr_seed_v1'; @@ -30,8 +31,14 @@ const String kQuietHrrSeedKey = 'quiet_hrr_seed_v1'; /// bundle (`SeriesCodec.decodePayloadJson`, so curves are `[{t, v}]` lists). /// [manualRestingHr] stands in only for a missing NOCTURNAL resting HR — the /// same pair, in the same order, the day's own TRIMP is anchored on. +/// +/// The ceiling is [estimatedMaxHr] at the CURRENT profile [ageYears], exactly +/// what the live derive prices `quiet_hrr` on — not the bundle's stored +/// `max_hr_used`, which a later birthday or profile edit leaves on another +/// scale. Null without an age, as the live derive is. double? quietHrrFromStoredBundle( Map payload, { + required num? ageYears, double? manualRestingHr, }) { final scalars = payload['scalars'] is Map @@ -41,7 +48,7 @@ double? quietHrrFromStoredBundle( ? payload['series'] as Map : const {}; final rhr = (scalars['rhr_nocturnal'] as num?)?.toDouble() ?? manualRestingHr; - final hrMax = (payload['max_hr_used'] as num?)?.toDouble(); + final hrMax = estimatedMaxHr(ageYears, payload['device_family'] as String?); final wake = { for (final p in (series['strain_curve'] as List? ?? const [])) if (p is Map && p['t'] is num) (p['t'] as num).toInt(), @@ -69,8 +76,15 @@ double? quietHrrFromStoredBundle( /// reason `putDayResult` skips its series write), and days that already have a /// value — a derive's own measurement is never overwritten. Decoding and the /// median run off the UI isolate. Returns the number of days written. -Future seedQuietHrrHistoryOnce({double? manualRestingHr}) async { +/// +/// Without [ageYears] there is no ceiling to price on: nothing is seeded and +/// the seed stays armed for a pass that has one. +Future seedQuietHrrHistoryOnce({ + required num? ageYears, + double? manualRestingHr, +}) async { if (await LocalDb.computeFreshness(kQuietHrrSeedKey) != null) return 0; + if (ageYears == null) return 0; final rows = await LocalDb.recentDayResults(ana.quietHrrWindowDays + 3); final imported = await LocalDb.importedDates(); final have = { @@ -93,7 +107,7 @@ Future seedQuietHrrHistoryOnce({double? manualRestingHr}) async { for (final (date, json) in todo) if (SeriesCodec.decodePayloadJson(json) case final p?) if (quietHrrFromStoredBundle(p, - manualRestingHr: manualRestingHr) + ageYears: ageYears, manualRestingHr: manualRestingHr) case final q?) (date, q), ]); diff --git a/lib/compute/strain_backfill.dart b/lib/compute/strain_backfill.dart index 2184a729b..c3b3e5b08 100644 --- a/lib/compute/strain_backfill.dart +++ b/lib/compute/strain_backfill.dart @@ -116,7 +116,10 @@ Future backfillStrainScale({ // This user's quiet level, resolved once. Without one there is nothing // honest to rescale against, so return WITHOUT marking done: the one-shot - // runs again once three days of `quiet_hrr` exist. + // runs again once three days of `quiet_hrr` exist. ONE level for every + // rescaled day, deliberately: these days predate any per-day level, so a + // trailing median per day would mostly be a median of nothing. A live + // derive prices each day on its own prior days; the difference is small. final quiet = await personalQuietLevelBefore(LocalDb.localDayLabelNow()); final quietHrr = quiet.value?.hrr; if (quietHrr == null) return none; diff --git a/lib/state/app_state.dart b/lib/state/app_state.dart index 3d1008eb2..b98fb75db 100644 --- a/lib/state/app_state.dart +++ b/lib/state/app_state.dart @@ -6371,8 +6371,14 @@ class AppState extends ChangeNotifier { // BEFORE the resting-HR early return below: a user with no nightly // series still has a quiet level once three days are measured. final live = activeWorkout; - if (live != null) await _fillLiveQuietLevel(live); - _rhr28 = await LocalDb.trailingSeriesValues('rhr', 28); + if (live != null) { + // Its own guard: a failed fill leaves the gauge at "—" but must not + // skip the resting-HR refresh below. + try { + await _fillLiveQuietLevel(live); + } catch (_) {} + } + _rhr28 =await LocalDb.trailingSeriesValues('rhr', 28); final vals = await LocalDb.trailingSeriesValues('rhr', 7); if (vals.isEmpty) return; _nightlyRhr = vals.last; diff --git a/test/quiet_hrr_seed_test.dart b/test/quiet_hrr_seed_test.dart index a995a9a05..fdb8e8239 100644 --- a/test/quiet_hrr_seed_test.dart +++ b/test/quiet_hrr_seed_test.dart @@ -41,18 +41,31 @@ Map _payload({ void main() { group('quietHrrFromStoredBundle — pure', () { test('the wake minutes\' median HRR', () { - // 62 bpm against RHR 55 / HRmax 183.5 = 7/128.5. - expect(quietHrrFromStoredBundle(_payload()), closeTo(0.0545, 0.0005)); + // 62 bpm against RHR 55 / HRmax 183.5 (age 35) = 7/128.5. + expect(quietHrrFromStoredBundle(_payload(), ageYears: 35), closeTo(0.0545, 0.0005)); + }); + + test('the ceiling comes from the CURRENT age, as the live derive\'s does', + () { + // The stored bundle was scored on another ceiling (a later birthday, an + // edited profile). Seeded and live levels must share one scale: age 40 + // is HRmax 180, so 7/125 — not 7/95 off the stored 150. + final stale = {..._payload(), 'max_hr_used': 150.0}; + expect(quietHrrFromStoredBundle(stale, ageYears: 40), + closeTo(0.056, 0.0005)); + expect(quietHrrFromStoredBundle(stale, ageYears: null), isNull, + reason: 'no age, no ceiling — the live derive abstains too'); }); test('no strain curve, no wake series — null', () { - expect(quietHrrFromStoredBundle(_payload(withCurve: false)), isNull); + expect(quietHrrFromStoredBundle(_payload(withCurve: false), ageYears: 35), isNull); }); test('no nocturnal RHR — the user-entered one anchors it', () { final noRhr = _payload(rhrNocturnal: null); - expect(quietHrrFromStoredBundle(noRhr), isNull); - expect(quietHrrFromStoredBundle(noRhr, manualRestingHr: 55), + expect(quietHrrFromStoredBundle(noRhr, ageYears: 35), isNull); + expect( + quietHrrFromStoredBundle(noRhr, ageYears: 35, manualRestingHr: 55), closeTo(0.0545, 0.0005)); }); @@ -62,7 +75,8 @@ void main() { expect(stored, isNot(contains('"strain_curve":[{')), reason: 'the codec actually stored a columnar curve'); final decoded = SeriesCodec.decodePayloadJson(stored)!; - expect(quietHrrFromStoredBundle(decoded), closeTo(0.0545, 0.0005)); + expect(quietHrrFromStoredBundle(decoded, ageYears: 35), + closeTo(0.0545, 0.0005)); }); }); @@ -108,6 +122,14 @@ void main() { r['date'] as String: (r['value'] as num).toDouble(), }; + test('without an age it seeds nothing and stays armed', () async { + await day('2026-08-20'); + expect(await seedQuietHrrHistoryOnce(ageYears: null), 0); + expect(await LocalDb.computeFreshness(kQuietHrrSeedKey), isNull, + reason: 'it runs once an age is set'); + await (await LocalDb.instance).delete('day_result'); + }); + test('seeds once, even with the strain rescale long since done', () async { // The population this exists for: an install whose v63 strain rescale // already ran, so `backfillStrainScale` returns at its freshness check. @@ -123,7 +145,7 @@ void main() { await LocalDb.putMetricSeriesValue('2026-09-08', 'quiet_hrr', 0.3); expect((await backfillStrainScale(female: false)).didWork, isFalse); - expect(await seedQuietHrrHistoryOnce(), 4); + expect(await seedQuietHrrHistoryOnce(ageYears: 35), 4); final rows = await quietRows(); expect(rows.keys.toSet(), @@ -144,7 +166,7 @@ void main() { final before = await quietRows(); // Even a newly stored day is left for its own derive to measure. await day('2026-09-09'); - expect(await seedQuietHrrHistoryOnce(), 0); + expect(await seedQuietHrrHistoryOnce(ageYears: 35), 0); expect(await quietRows(), before); }); }); From 3a9437174919463ec10e55cb9c9939c360a68c5b Mon Sep 17 00:00:00 2001 From: Alex Wagner Date: Thu, 8 Oct 2026 16:37:10 +0200 Subject: [PATCH 3/5] strain: the quiet-level seed picks measured days first and moves the rescan gate The seed read only the newest window of result rows and dropped imports, skipped and partial rows afterwards, so a long import newer than the measured days filled the window, nothing was seeded, and the seed was marked done. It now selects measured days first (payload-free), applies the window to those, and reads each bundle on its own. A seed that wrote levels left the rolling artifact's signature as it was, so on a pass with nothing to derive rescanRecent saw no change and never re-scored the days the new levels price. The artifact is now refreshed after any nonzero seed, ahead of every early return, under its own guard. --- lib/compute/derivation_engine.dart | 15 +++++++++++- lib/compute/quiet_level_seed.dart | 20 +++++++++++----- test/quiet_hrr_seed_test.dart | 21 +++++++++++++++++ test/quiet_level_engine_test.dart | 38 ++++++++++++++++++++++++++++++ 4 files changed, 87 insertions(+), 7 deletions(-) diff --git a/lib/compute/derivation_engine.dart b/lib/compute/derivation_engine.dart index 194eebbc6..9377a45b8 100644 --- a/lib/compute/derivation_engine.dart +++ b/lib/compute/derivation_engine.dart @@ -2801,8 +2801,9 @@ class DerivationEngine { // (`compute_freshness`); a failure leaves the key unset and the next pass // retries — never fatal to the derive. _diag['stage'] = 'quiet_hrr_seed'; + var seeded = 0; try { - final seeded = await seedQuietHrrHistoryOnce( + seeded = await seedQuietHrrHistoryOnce( ageYears: profile.ageYears, manualRestingHr: profile.restingHrManual?.toDouble(), ); @@ -2810,6 +2811,18 @@ class DerivationEngine { } catch (e) { _log('[derive] quiet_hrr seed failed (will retry): $e'); } + // New levels move the baseline signature `rescanRecent` gates on. + // Refreshed here, ahead of every early return: on a pass with nothing + // to derive nothing else would, and the days the levels now price would + // never be re-scored. Its own guard — the seed is already marked done, + // so a failure here is not a seed failure; the next refresh catches up. + if (seeded > 0) { + try { + await _refreshBaselines(); + } catch (e) { + _log('[derive] baseline refresh after the quiet_hrr seed failed: $e'); + } + } _diag['stage'] = 'scope'; final scope = await _deriveScope(heavy: heavy, force: force); _diag diff --git a/lib/compute/quiet_level_seed.dart b/lib/compute/quiet_level_seed.dart index 935e872cd..1584131c7 100644 --- a/lib/compute/quiet_level_seed.dart +++ b/lib/compute/quiet_level_seed.dart @@ -85,21 +85,29 @@ Future seedQuietHrrHistoryOnce({ }) async { if (await LocalDb.computeFreshness(kQuietHrrSeedKey) != null) return 0; if (ageYears == null) return 0; - final rows = await LocalDb.recentDayResults(ana.quietHrrWindowDays + 3); final imported = await LocalDb.importedDates(); final have = { for (final r in await LocalDb.metricSeries('quiet_hrr')) r['date'] as String, }; - final todo = <(String, String)>[ - for (final r in rows) + // The window is over MEASURED days, chosen before it is applied: a long + // import (or a run of skipped/partial rows) newer than them must not fill + // the window and leave the levels that are recoverable unseeded. Every + // served day, payload-free (LIMIT -1 is SQLite's "no limit"); bundles are + // then read one day at a time. + final measured = [ + for (final r in await LocalDb.recentDayResultsMeta(-1)) if (r['day_id'] is String && - r['payload_json'] is String && !imported.contains(r['day_id']) && - !have.contains(r['day_id']) && (r['skipped'] as num?)?.toInt() != 1 && (r['partial'] as num?)?.toInt() != 1) - (r['day_id'] as String, r['payload_json'] as String), + r['day_id'] as String, + ].take(ana.quietHrrWindowDays + 3); + final todo = <(String, String)>[ + for (final day in measured) + if (!have.contains(day)) + if ((await LocalDb.dayResult(day))?['payload_json'] case final String j) + (day, j), ]; final levels = todo.isEmpty ? const <(String, double)>[] diff --git a/test/quiet_hrr_seed_test.dart b/test/quiet_hrr_seed_test.dart index fdb8e8239..3427e4e9d 100644 --- a/test/quiet_hrr_seed_test.dart +++ b/test/quiet_hrr_seed_test.dart @@ -169,5 +169,26 @@ void main() { expect(await seedQuietHrrHistoryOnce(ageYears: 35), 0); expect(await quietRows(), before); }); + + test('imports filling the recent rows do not crowd out older measured days', + () async { + final db = await LocalDb.instance; + for (final t in ['day_result', 'metric_series', 'compute_freshness']) { + await db.delete(t); + } + const measured = ['2026-06-01', '2026-06-02', '2026-06-03', '2026-06-04']; + for (final d in measured) { + await day(d); + } + // A vendor import newer than all of them, longer than the whole window. + for (var i = 0; i < 40; i++) { + await day( + '2026-07-${(i % 30 + 1).toString().padLeft(2, '0')}' + .replaceFirst('07', i < 30 ? '07' : '08'), + imported: true); + } + expect(await seedQuietHrrHistoryOnce(ageYears: 35), 4); + expect((await quietRows()).keys.toSet(), measured.toSet()); + }); }); } diff --git a/test/quiet_level_engine_test.dart b/test/quiet_level_engine_test.dart index 923b330c1..156c35e64 100644 --- a/test/quiet_level_engine_test.dart +++ b/test/quiet_level_engine_test.dart @@ -168,6 +168,44 @@ void main() { expect(await _value('2026-02-03', 'quiet_hrr'), closeTo(0.0545, 0.0005)); }); + test('a seed that writes levels refreshes the rescan signature', () async { + await _clear(); + // Every day finalized, nothing for a derive to do: the seed is the only + // thing that moves the baseline, so it must move the gate too, or + // rescanRecent never re-scores the days the new levels now price. + for (final day in const ['2026-02-01', '2026-02-02', '2026-02-03']) { + await LocalDb.putDayResult( + dayId: day, + algoVersion: kAlgoVersion, + payloadJson: SeriesCodec.encodePayloadJson(jsonEncode({ + 'date': day, + 'scalars': {'rhr_nocturnal': 55}, + 'series': { + 'hr_curve': [ + for (var i = 0; i < 400; i++) {'t': i * 60, 'v': 62}, + ], + 'strain_curve': [ + for (var i = 0; i < 400; i++) {'t': i * 60, 'v': 0.0}, + ], + }, + })), + windowJson: '{}', + finalized: true, + source: 'band', + ); + } + final before = await debugBaselineSignature(); + await LocalDb.putBaseline( + 'rolling_artifact', jsonEncode({'signature': before})); + await DerivationEngine().run(_profile, heavy: true); + final after = await debugBaselineSignature(); + expect(after, isNot(before), reason: 'the seed moved the quiet levels'); + final stored = jsonDecode( + (await LocalDb.baseline('rolling_artifact'))!['payload_json'] + as String) as Map; + expect(stored['signature'], after); + }); + // The tests below run real derives over 1 Hz rows — well past the default // 30 s once the suite is loaded, hence their explicit budgets. test('a first sync of a whole week does not leave it blank', () async { From 38e0378078f88d3d7aeedbfe250113d4dab3c94c Mon Sep 17 00:00:00 2001 From: Mohammad Abdul Sahil <127765312+abdulsaheel@users.noreply.github.com> Date: Fri, 9 Oct 2026 01:16:04 +0530 Subject: [PATCH 4/5] strain: hash the newest quiet level once a later day sits on it The rescan signature left the newest quiet_hrr entry out on the assumption that only unfinalized days come after it. A day too short to write its own level (under 360 wake minutes) can follow it and be finalized, priced on that newest level; moving the level then never re-priced it. The newest entry is now left out only while no baseline series has a later day, which keeps today's rewrites from triggering a rescan. Also restores the missing space in `_rhr28 = await`. --- lib/compute/derivation_engine.dart | 19 +++++++++++++------ lib/state/app_state.dart | 2 +- test/readiness_baseline_pollution_test.dart | 8 ++++++++ 3 files changed, 22 insertions(+), 7 deletions(-) diff --git a/lib/compute/derivation_engine.dart b/lib/compute/derivation_engine.dart index 9377a45b8..9278fd691 100644 --- a/lib/compute/derivation_engine.dart +++ b/lib/compute/derivation_engine.dart @@ -2548,15 +2548,22 @@ class _BaselineHistoryCache { /// covers: the [_rescanWindowDays] + [_baselineWindowDays] stored days /// before the newest. /// - /// THE NEWEST IS LEFT OUT. It is normally today's, rewritten on every light - /// derive as the wake series grows, and the only windows it sits in are of - /// days after it — which are not finalized and are re-derived by the - /// ordinary passes anyway. Hashing it re-ran the whole rescan window on - /// every heavy pass for nothing. + /// THE NEWEST IS LEFT OUT while no stored day comes after it. It is then + /// normally today's, rewritten on every light derive as the wake series + /// grows, and no day yet sits on a window holding it. Hashing it re-ran the + /// whole rescan window on every heavy pass for nothing. Once ANY baseline + /// series has a later day (one too short to write its own level, say, which + /// may already be finalized), that day was priced on it, so it is hashed. String _quietFingerprint() { final series = _series['quiet_hrr'] ?? const <_DatedValue>[]; const span = _rescanWindowDays + _baselineWindowDays; - final end = series.isEmpty ? 0 : series.length - 1; + final newest = series.isEmpty ? null : series.last.date; + final laterDay = newest != null && + _series.entries.any((e) => + e.key != 'quiet_hrr' && + e.value.isNotEmpty && + e.value.last.date.compareTo(newest) > 0); + final end = series.isEmpty ? 0 : series.length - (laterDay ? 0 : 1); final from = end <= span ? 0 : end - span; final dated = [ for (var i = from; i < end; i++) diff --git a/lib/state/app_state.dart b/lib/state/app_state.dart index b98fb75db..756f29252 100644 --- a/lib/state/app_state.dart +++ b/lib/state/app_state.dart @@ -6378,7 +6378,7 @@ class AppState extends ChangeNotifier { await _fillLiveQuietLevel(live); } catch (_) {} } - _rhr28 =await LocalDb.trailingSeriesValues('rhr', 28); + _rhr28 = await LocalDb.trailingSeriesValues('rhr', 28); final vals = await LocalDb.trailingSeriesValues('rhr', 7); if (vals.isEmpty) return; _nightlyRhr = vals.last; diff --git a/test/readiness_baseline_pollution_test.dart b/test/readiness_baseline_pollution_test.dart index 1917c429a..719c31f49 100644 --- a/test/readiness_baseline_pollution_test.dart +++ b/test/readiness_baseline_pollution_test.dart @@ -331,5 +331,13 @@ void main() { final stable = await debugBaselineSignature(); await LocalDb.putMetricSeriesValue('2027-01-07', 'quiet_hrr', 0.12); expect(await debugBaselineSignature(), stable); + + // …until a LATER day exists that wrote no level of its own (too few wake + // minutes) and may already be finalized: it was priced on the newest + // level, so moving that level must re-price it. + await LocalDb.putMetricSeriesValue('2027-01-08', 'rhr', 52); + final withLater = await debugBaselineSignature(); + await LocalDb.putMetricSeriesValue('2027-01-07', 'quiet_hrr', 0.14); + expect(await debugBaselineSignature(), isNot(withLater)); }); } From 0d8daf569870d90fc91377c992e2868fd441862d Mon Sep 17 00:00:00 2001 From: Mohammad Abdul Sahil <127765312+abdulsaheel@users.noreply.github.com> Date: Fri, 9 Oct 2026 01:37:35 +0530 Subject: [PATCH 5/5] two-device golden: strain's provenance label for the personal quiet level The golden's day has no quiet-level history, so strain abstains, as it did before. Only the absent metric's inputs_used changes, from the lump form's (trimp, wake_minutes, quiet_waking_hrr) to the series form's (hr_1hz, profile, quiet_waking_hrr_history). No number moves. Rides the kAlgoVersion bump this change already requires. --- test/fixtures/two_device_day_expected.json | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/test/fixtures/two_device_day_expected.json b/test/fixtures/two_device_day_expected.json index 7f2b969df..1487d3ae3 100644 --- a/test/fixtures/two_device_day_expected.json +++ b/test/fixtures/two_device_day_expected.json @@ -173,9 +173,9 @@ "confidence": 0.0, "tier": "ESTIMATE", "inputs_used": [ - "trimp", - "wake_minutes", - "quiet_waking_hrr" + "hr_1hz", + "profile", + "quiet_waking_hrr_history" ], "note": "need_input:name=wake_hr" },