Skip to content

sleep: halve the RSA respiration weight, measured against PSG with R-R (DREAMT) - #2613

Open
UtkuDenizAltiok wants to merge 2 commits into
ryanbr:mainfrom
UtkuDenizAltiok:dreamt-psg
Open

UtkuDenizAltiok wants to merge 2 commits into
ryanbr:mainfrom
UtkuDenizAltiok:dreamt-psg

Conversation

@UtkuDenizAltiok

@UtkuDenizAltiok UtkuDenizAltiok commented Sep 30, 2026 •

Copy link
Copy Markdown

What this PR does

NOOP's sleep stager reads how regular your breathing is from the heartbeat (R-R) and uses it to tell deep sleep (regular breathing) from REM (irregular). That signal had never been checked against a sleep lab with heartbeats present: Tools/SleepPSG's dataset has none. This PR adds a second dataset that has them, DREAMT, and on it the signal turns out to be real but counted about twice too strongly: it pushes light sleep into both deep and REM. So its weight goes from 0.6 to 0.3.

Two commits:

  1. sleeppsg: score the shipped stager against DREAMT, with R-R live. A reader for DREAMT's 64 Hz E4 files, and a section 8 that runs the shipped SleepStagerV2 with and without R-R. It is stratified by apnoea severity and by how many beats the E4 kept. It also checks whether the feature separates the PSG stages, and compares shapes and weights of the term subject by subject. The sleep-accel report is byte-identical before and after (same SHA-256 on the full output).
  2. sleep: halve the RSA respiration weight. respWeight 0.6 → 0.3 on Swift and Kotlin, pinned on both.

Why. The recipe adds respWeight · z to deep and subtracts it from REM, where z is the night-normalised breathing regularity; light is left alone. On DREAMT (PhysioNet v2.2.0, 100 adults, 80 190 PSG-scored epochs):

  • The feature is informative, in the assumed direction. Mean z by PSG stage: deep +0.29, light +0.06, wake −0.09, REM −0.25. AUC is 0.658 for deep vs REM and 0.597 for light vs REM. Per subject, NREM vs REM has a median AUC of 0.559 (above 0.5 for 50 of 71).
  • At 0.6 it is too strong. Light epochs spread over z like every other stage, so ±0.6·z moved them into deep on one side and REM on the other. At 0.6, R-R live vs withheld: per-subject κ lower for 55 and higher for 38; REM 16.2 → 19.9 % of the night (truth 10.5); deep 10.7 → 12.8 % (truth 3.4).

Per subject, against 0.6:

RSA term mean κ κ better / worse mean |deep bias| pp mean |REM bias| pp REM % of sleep (truth 14.0)
0.6 (before) 0.204 — 10.60 11.16 21.4
0.45 0.207 58 / 30 10.27 10.47 20.0
0.3 (this PR) 0.216 61 / 28 9.76 9.74 18.4
0.2 0.217 58 / 35 9.51 9.57 17.6
off 0.216 55 / 38 9.18 9.77 17.4
regular breathing only, max(z, 0) at 0.6 0.190 38 / 54 13.23 7.78 14.2
REM side only at 0.6 0.218 59 / 30 9.16 11.13 21.3

The same ordering holds with AHI under 5 (n = 26: 0.3 better than 0.6 for 16, worse for 7) and where the E4 kept at least 80 % of the beats (n = 41: 31 / 10).

Why 0.3 and not less. Against 0.3, the 0.2 and "off" rows are within 0.001 in mean κ (per subject 51 / 31 and 50 / 40). DREAMT's deep truth is 3.4 % of the night, so any push toward deep scores badly on this cohort; below 0.3 it mostly rewards that. The weight the separation itself implies does not depend on base rates. For two equal-variance classes a unit of z is worth d′ = √2·Φ⁻¹(AUC) log-odds: 0.58 for deep vs REM, where the term's slope is 2 × respWeight, and 0.35 for light vs REM, where the slope is respWeight. That gives 0.29–0.35. So 0.3 halves the term rather than deleting a feature that does carry information. "Regular breathing only" fixes the REM share but floods deep, and "REM side only" keeps the REM over-call; neither is taken.

What a user sees. On nights with R-R, a little less REM and deep and a little more light. On one WHOOP 5.0 wearer's six nights: REM 31.9 → 29.3 % of sleep, deep 21.6 → 21.2 %, light 46.5 → 49.5 %. Detection is unchanged and wake barely moves (7.8 → 7.9 % of the window). Nights without R-R are unchanged.

Type of change

  • Bug fix (sleep staging against PSG)

How it was tested

  • DREAMT reader (DreamtTests, 6 tests, synthetic files, runs in CI): beats recovered from the forward-filled IBI column (consecutive, one or two hidden repeats, a gap), the grid aligned on the first scored epoch, the per-second collapse, the participant table. Each test was seen to fail with its logic broken, then restored byte-identical.
  • Port validation passes with 0.3 on both sides (12 377 / 12 377 epochs). The new respShape knob defaults to the shipped shape.
  • Pins: testRespWeightIsPinned / respWeightIsPinned. The frozen golden moves one boundary by three epochs (10740–16290 → 10740–16200 REM), regenerated on both platforms. With 0.6 restored, the pin and the golden fail.
  • testDetectSleepThreadsV2FlagIntoNormalNight asserted that its flat synthetic night shows deep under V2. Nothing in that night calls for deep: flat HR, identical breathing in every epoch. 354 of its 360 epochs score 0.812972. The three at each edge, whose beat windows reach past the stream, differ only in the fourth decimal (up to 0.8148). With a night spread of 0.00016, the per-night z-score turns that into z of +2.7 to +11.6, which the term at 0.6 painted deep. The test now asserts what it states it proves: the flag gives V2's hypnogram, which is not V1's, and it still shows REM. Same change in the Kotlin twin.
  • Quantisation check. The E4 reports intervals in 1/64 s steps. On the six WHOOP nights above, rounding R-R to 1/64 s moved the six-night mean deep and REM by under 1 pp of sleep (single nights by up to 4 pp), against about 4.5 pp for switching the term off. So section 8 measures the term, not the E4's resolution.
  • Full local run on 29f43bb2 (on 7f396e98; the final ca008aba changes only two test comments, re-checked with SleepStagerV2Tests 18/18 and the doc lint): WhoopStore 631 · StrandAnalytics 2103 · StrandImport 327 · doc lint · i18n · parity ledger · parity ratchet · parity governance 127 · macOS StrandTests 2,254 (only the two locale-dependent TodayCarryOverTests) · iOS build. Tools/SleepPSG: swift test 39/39, and the full sleep-accel report is byte-identical to main's before and after both commits.
  • Android CI on the fork (assembleFullDebug + testFullDebugUnitTest): green on ca008aba (run 36712653602).

DREAMT is restricted-access (PhysioNet Restricted Health Data License 1.5.0): each user signs the agreement and downloads it; it is never committed and the tool prints aggregates only. It is a sleep-clinic cohort (#348, #437). Nothing here fits a prior or base rate to it: the change is the strength of one feature, and the table is read per subject and per stratum.

Checklist

  • No new build warnings introduced
  • Follows the conventions in docs/CONTRIBUTING.md
  • I did not commit generated output (Strand.xcodeproj/) or any secrets/keystores

Related issues

Refs #348, #437, #987, #2576

UtkuDenizAltiok and others added 2 commits September 30, 2026 13:44
sleep-accel carries no beat-to-beat intervals, so the recipe's RSA
respiration term has never run against PSG truth. DREAMT (PhysioNet v2.2.0,
100 adults, an Empatica E4 beside PSG) has the E4's inter-beat intervals.

--dataset now also takes a DREAMT folder. The reader collapses the 64 Hz
file per second (gravity mean, HR), turns the forward-filled IBI column back
into beats by arithmetic (a repeated interval hides a change; the gap between
changes says how many), and aligns the 30 s grid on the first scored epoch,
which sits 24 s past a multiple of 30. PSGSubject gains rr and ahi; for
sleep-accel rr stays empty and the full report is byte-identical.

Section 8 (rsa, DREAMT only) runs the shipped stager with and without R-R by
AHI and by beats kept, checks whether the feature separates the PSG stages,
and compares shapes and weights of the term subject by subject. The port
gains a respShape knob for that; shipped stays .symmetric (port validation
12377/12377).

DREAMT is restricted-access: never committed, and the tool prints cohort
aggregates only. It is a sleep-clinic cohort (ryanbr#348, ryanbr#437), so nothing is
fitted to its base rates.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
SleepStagerV2 adds respWeight·z to the deep emission and subtracts it from
REM, where z is the night-normalised regularity of breathing read from the
R-R series. Until now it had never met PSG truth with R-R present.

On DREAMT (PhysioNet v2.2.0, n = 100, Tools/SleepPSG section 8) the feature
separates the stages the way the term assumes (AUC deep vs REM 0.658, light
vs REM 0.597), but light epochs spread over z like every other stage, so at
0.6 the term moved them into deep or REM. Per subject, against 0.6:
0.3 raises kappa for 61 and lowers it for 28 (mean 0.204 -> 0.216), |deep
bias| 10.60 -> 9.76 pp, |REM bias| 11.16 -> 9.74 pp, REM 21.4 -> 18.4 % of
sleep (truth 14.0). AHI < 5: 16 / 7; E4 kept >= 80 % of beats: 31 / 10. The
separation itself implies 0.29-0.35 (d' from the AUCs). sleep-accel has no
R-R and is unchanged.

Swift and Kotlin move together and pin the weight. The frozen golden moves
one boundary by three epochs (16290 -> 16200 s). The V2-flag test asserted
that a flat synthetic night shows deep; that deep came from one truncated
edge epoch whose score the z-score turns into +8, so the test now asserts
what it meant: V2's hypnogram, not V1's. RecipeConfig.shipped follows.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
UtkuDenizAltiok added a commit to UtkuDenizAltiok/noop that referenced this pull request Sep 30, 2026
UtkuDenizAltiok added a commit to UtkuDenizAltiok/noop that referenced this pull request Oct 1, 2026
…, DREAMT harness, a8d25c1 shipped; REM and memory findings; cleanup PRs in progress
@UtkuDenizAltiok

Copy link
Copy Markdown
Author

Hi @ryanbr, could you review this sleep change when you have time? Sleep stages are a big part of why I use NOOP with my WHOOP, so I wanted this checked against sleep-lab recordings. Giving the breathing feature less weight improved agreement for more people than it worsened in the DREAMT comparison, while keeping the feature. The limitations are in the description and all checks are passing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant