Repository navigation
Sleep staging V3: a stager fitted to polysomnography, default, with V2/V1 selectable - #21
Merged
Merged
Conversation
Every published wrist stager that reaches a four-class kappa of 0.5 fits its per-epoch emissions to human-scored PSG. NOOP's V1 and V2 set every coefficient by hand. Tools/SleepTrain fits one on open data only and reports it. - prepare_*.py turn Wearanize+ OA (88 nights, Empatica E4, CC BY 4.0), PhysioNet sleep-accel (31 nights, Apple Watch, ODC-By) and DREAMT (sleep-clinic patients, restricted licence) into one night format: whole-second gravity, integer bpm and whole-second-stamped integer-ms beat intervals, which is what the app hands a stager. - sleeptrain/features.py is the feature definition in plain numpy with every sum taken left to right, so a Swift port can reproduce it. Everything is relative to the night: motion over the night's own jerk floor, posture change over its median change, heart rate as within-night z-scores and ranks, 5-minute HRV scaled to the night's percentiles, and elapsed time as minutes plus three decays so a linear head can learn the sleep-onset descent. - train.py uses every night twice (whole scored span, and cropped to the window SleepStager.bandSleepWindow would allow), fits a base head (motion and heart rate) and an HRV head, counts the stage transitions, reports 10-fold grouped cross-validation and scores DREAMT as an untouched test set, then writes the generated Swift model. - make_oracle.py writes the fixture that pins the Swift port. Datasets are never committed. DREAMT's licence limits it to research, so it only tests the model and never enters a fit. The fit is deterministic: two runs write byte-identical coefficients.
SleepStagerV3 is a drop-in for SleepStagerV2.stageSession: same signature, same [StageSegment] tiling, memoized the same way. It ports Tools/SleepTrain's feature definition line for line and runs the two generated heads. When the night has beat intervals on at least half its epochs both heads run and their log-posteriors are averaged with equal weight; otherwise the base head (motion and heart rate) stages it. A Viterbi pass over the counted stage transitions decodes the result. With the band's sleep window it stages only the epochs inside it, and everything else is wake. Cross-validated agreement with PSG (four-class kappa over every scored epoch, wake outside the window), V2 -> V3: - Wearanize+, band-style window: 0.363 -> 0.547 (second scorer 0.742) - Wearanize+, whole span: 0.301 -> 0.533 - sleep-accel, band-style window: 0.434 -> 0.504 - DREAMT sleep-clinic patients, never trained on: 0.259 -> 0.313 On Wearanize+ every stage share is within 1 pp of PSG (V2: light +13.5, deep -5.4, REM -7.0 pp), and no night starts deep within 5 min of onset. First REM comes 47 min late, as V2's does (48). The equal blend is untuned. The HRV head alone scored 0.554 and 0.320 but called REM high outside its training cohort (+7.9 pp on DREAMT, +5.9 blended). SleepStagerV3OracleTests rebuilds three SplitMix64 synthetic nights and requires every feature and both heads' posteriors within 1e-9 of the Python reference, and every label and segment equal. A night stages in about 0.09 s in a release build (V2: 0.3 s).
`sleeppsg --section v3 --nights DIR ...` replays the shipped V2 and V3 over prepared night-format cohorts. Each night is staged inside a band-style window emulated from its PSG hypnogram, and over its whole scored span. The report gives kappa, per-stage F1, stage-share bias, per-night minute bias with limits of agreement, and first deep / REM latency. On DREAMT, which V3 never trained on, it reproduces train.py's Python figures exactly (0.313 in the window, 0.256 over the span). That is the end-to-end check that the Swift port stages what the reference does. On the training cohorts V3's rows are in-sample, and the report says so.
SleepStagerVersion (.v1 / .v2 / .v3) replaces the useSleepStagerV2 Bool through detectSleep, AnalyticsEngine and the sleep-motion trace line, and dispatches to its recipe. V2 and V3 stage inside the band's sleep window; V1 still relies on the latency trim. The HR-only path keeps V2, as before, because V3 was trained with motion present. PuffinExperiment.sleepStager is the one resolver that the engine, the self-heal restage, the REM-funnel diagnostic line and Settings all read. An explicit choice wins. Otherwise a legacy explicit V2-off stays V1, and everyone else gets V3. The key is not in the backup whitelist, so backups are unchanged. The day-cache config signature carries the choice, so switching restages the scoring window. Settings > Experimental > Sleep staging becomes a V3 / V2 / V1 segmented picker. It reads through that resolver, so an upgraded user whose V2 switch was off sees V1 there, as staged. Five new strings in nine locales; the old toggle's two strings are removed. Tests: SleepStagerSettingTests pins the upgrade rules. detectSleep with .v3 keeps the same detected window and stages it with V3. Every version dispatches to its own recipe. V3 detects no sleep without gravity, like V1 and V2.
Adds the V3 section: features, the two heads and their blend, the decoder, the band window, the training data, and the cross-validated agreement next to V2 and a second human scorer, with its known limits (first REM late, deep over-counted on sleep-clinic nights, and no WHOOP strap in any PSG set). The honest-hedging note no longer says "not PSG-validated". Three lines that named V2 the default are corrected.
…n uses `Repository.restageFromRaw` (edit, add-nap and the post-sync self-heal) called the recipe alone: no band sleep window and no band latency trim. `detectSleep` stages inside the window and trims outside it. Re-staging a detected night over its own bounds therefore staged the lying-awake lead-in as sleep again. On the owner's six banked 5/MG nights, V2's first sleep moved up to 2 h 04 min earlier and TST rose by up to 60 min; V3's first sleep and TST each moved by about 20 min. `SleepStager.stageWindow` is now the one staging funnel. It applies the recipe inside the window (V2/V3), then the trim and the wake-veto. `detectSleep` stages every accepted night through it, and the re-stage reads the band's state and calls it too. The window is resolved once per night and handed to both the recipe and the trim, so the band stream is no longer scanned twice.
…ature records `analyzeRecent` read the staging choice (and the motion-aware wake toggle) once for the day-cache config signature and again inside the day loop. A picker change made while a pass was running could therefore stage the later days of that pass with the new recipe and cache them under a signature that names the old one. The loop now uses the single pass-level read.
…sleeppsg does `train.py`'s summary averaged every night's kappa, so a night with one class on both sides (pe = 1) made the "mean night kappa" column NaN. `sleeppsg --section v3` leaves such a night out, so the two reports that ANALYTICS.md says reproduce each other could print different figures for the same cohort. `kappa` now returns NaN explicitly where it is undefined, as `Confusion.kappa` does, and the summary skips it. No published figure changes: no cohort night has an undefined kappa.
`dayCacheConfigSig` builds 17 values, but the separate `dayCacheConfigFields` list held 16 names: the Healthspan zone set joined the signature without a name. `changedConfigField` refuses to name anything when the counts differ, so every cache drop since has been logged as "unknown", including a switch of the new sleep-stager picker. The pinned test built its signature from the 16-name list, not from the real construction, so it could not notice. The signature is now built from (name, value) pairs, and the names travel to `changedConfigField` with it, so adding or reordering a field cannot leave its name behind. The Kotlin-twin list test goes: this fork ships no Android target.
…the night's recipe "screen staged by V3" printed the current picker value as the recipe that staged the diagnosed night. A stored night keeps no record of its recipe. It may have been staged before the picker moved, and a night the merge takes from an import was never staged here. The line now says the screen shows the stored hypnogram and names the current setting, and it always says the totals can differ, including when the setting is V1.
…and Got up The night editor's default labels became Bedtime / Got up when the Asleep/Woke row moved to the hypnogram's bounds. The nap row opens the same sheet with those defaults, so editing a 14:00 nap showed Bedtime and Got up. A nap edit now uses the nap wording the Add-a-nap sheet and the nap row's pencil already use: "Edit nap times", "Nap started", "Nap ended". All three strings are already translated.
…h it lies in `epochInSleepWindow` judges each 30 s recipe epoch (wall-clock grid) by the one session-grid instant it contains. The session's last recipe epoch is cut short at `end`, and when the session starts δ seconds off the wall clock and ends within δ of a wall-clock boundary, that epoch holds no session-grid instant before `end`. Judged by the instant past `end`, it fell outside a window that runs to the end. V2 and V3 then closed a night the band slept through on 1-29 s of wake, which the trim never samples and so never removed. With the window clamped at `end`, which is what the trim needs, leaving `to` unclamped does not reach that instant either. The epoch now falls back to the session's last grid instant. `end` is threaded through V2's and V3's `stageEpochs` and mirrored in `sleeptrain/model.py`'s `in_window`. The committed V3 oracle regenerates byte-identical: its one windowed case closes 15 min before its end. On the owner's six banked 5/MG nights it applies to two. One gains its last 9 s of sleep. On the other, V3 stages one more epoch. V3 normalises its features over the staged span and decodes it as a whole, so that night's borderline tail re-decodes: its last sleep moves 44 min earlier and TST falls by 6 min. V2's nights move by the sliver alone.
… not 0 A V3 feature missing for the whole staged span was imputed as 0. That is neutral for a z-scored column but an extreme for a percentile rank (typical 0.5) or a ratio. A span with gravity but no heart rate yet, which the re-stage's density gate allows, read as the lowest, flattest heart rate of the night: hr_pct and the three hrsd ranks at z = -1.74, hr_over_low at -1.03. With the heart rate removed, the owner's six banked 5/MG nights staged 32-48 % deep, against 8-18 % with it. A whole-night gap now takes the head's training mean for that column, so it standardises to 0 and moves no logit (those nights come out 84-95 % light). The fit and every PSG figure are unchanged. No record the heads were trained on (Wearanize+ 176, sleep-accel 62) or tested on (DREAMT 20) has such a gap, and `sleeppsg --section v3` scores all three cohorts identically either way. `sleeptrain/features.py`'s `impute` takes the same fallback from `Head.posteriors`. In the regenerated oracle, only the no-beats case's HRV-head posteriors move; that head does not stage the case, and every label is the same.
…say which ran `SleepStager.hrOnlySessions`, the engine's fallback for a day that banked no gravity, always staged with V2. The Rest trace's `stager=` still named the picker's choice (V3 by default), and the picker had no effect on those nights. The choice now reaches them: `hrOnlySessions(stager:)` stages with `stager.hrOnlyRecipe`, which is the choice itself for V2 and V3. V1 builds its epoch grid from gravity and returns one flat "light" block without it, so V1 maps to V2. `analyzeDay` names that recipe in the trace for an HR-only night. This makes V3 reachable with no motion, so it now reads such a span properly. No jerk anywhere in the span (no gravity, or none in two consecutive seconds) leaves the motion features missing, and no measurable block-to-block posture change leaves the posture features missing. With the previous fix, a missing feature takes the head's training mean. Read as present, zero activity and the time-since-move cap made this the stillest night V3 ever saw, and deep came out 19-30 points too high. With motion stripped from the PSG nights, four-class kappa inside the band window: cohort V2 (was) V3 Wearanize+ 0.303 0.477 sleep-accel 0.363 0.446 DREAMT 0.224 0.275 (the only out-of-sample cohort) V3's first REM lands 40-74 min late on these nights. With gravity present, every figure is unchanged: every cohort night carries gravity. `sleeptrain/features.py` mirrors the rule. The oracle gains a fourth, gravity-free night that pins it in both languages; the first three cases are unchanged.
…indow, measured With a band sleep window, V2's REM and deep onset guards run from the band's onset, but its cycle prior (deep early, REM rising toward morning) still reads `c`, the fraction of the whole session. On a long lying-awake lead-in, part of the early deep prior is spent before sleep begins: on the owner's six banked 5/MG nights, `c` at the band's onset was 0.03-0.19, a deep prior of 0.79-1.13 instead of 1.20. Measuring `c` over the window instead scored no better against PSG in the window. Four-class kappa went 0.363 -> 0.360 on Wearanize+, 0.434 -> 0.427 on sleep-accel and 0.259 -> 0.261 on DREAMT, and first REM came later on all three. So the recipe is unchanged, and the claims around it now say exactly what moves with the window: the detection comment said "the recipe starts from the band's onset". `cyclePrior` and ANALYTICS.md record the measurement.
V3 re-implemented V2's `stageSession` veneer: the stable timestamp sort (V2's `stableOrder` closure, V3's `sortedByTs`), the half-open clip (`clipToWindow` / `clip`), the memo key (`V2Key` / `V3Key`, field for field) and the segment tiling. A future fix to the clip bounds or the key would have had to find both copies. `StagerInput` now holds the sort, the clip, one `Key` built from the clipped streams and the sleep window, and `tile`. Both stagers call it and keep their own caches and read windows (V2's pads, V3's epoch span). Output is byte-identical: V2's frozen golden and shuffled-input tests, the V3 oracle and `sleeppsg`'s port validation all pass unchanged. V2's extra "nothing clipped" early return was unreachable once the first bounds check had failed, so it went.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #20 (base
fix/sleep-band-onset-staging); merge that first.Why
V1 and V2 set every staging coefficient by hand. Every published wrist stager that reaches a four-class kappa of 0.5 fits its per-epoch emissions to human-scored polysomnography instead (Fitbit, Oura, Philips, Fonseca et al. 2023, Walch et al. 2019). This PR does that with open data only, makes the result the default, and keeps V2 and V1 one tap away.
What
Tools/SleepTrain(Python, never shipped): dataset preparation into one night format (whole-second gravity, integer bpm, whole-second-stamped integer-ms beat intervals, which is what the app hands a stager), the feature definition, 10-fold grouped cross-validation, the fit, and the generator for the Swift model file and the oracle fixture. Datasets are never committed.SleepStagerV3, a drop-in forSleepStagerV2.stageSession:sleeppsg --section v3replays the shipped V2 and V3 over the prepared cohorts.SleepStagerVersionreplaces the V2 Bool.PuffinExperiment.sleepStageris the one resolver that the engine, the restage, the diagnostic line and Settings all read. An explicit choice wins, a legacy explicit V2-off stays V1, and everyone else gets V3.docs/ANALYTICS.md: new V3 section.Agreement with PSG
Four-class Cohen's kappa over every scored epoch, with wake outside the staged span. Training cohorts are 10-fold cross-validated by subject. DREAMT never enters a fit.
On six banked WHOOP 5/MG nights (no PSG exists), sleep/wake agreement with the strap's own band averages kappa 0.66 for V3 and 0.61 for V2.
Choices the numbers made
bandSleepWindowwould allow. So both of the app's cases are in-distribution.exp(-t/10),/30,/90). Without them the linear head started deep within 5 min of onset on 37 of 88 Wearanize+ nights (PSG: none).Verification
SleepStagerV3OracleTestsrebuilds three SplitMix64 synthetic nights and requires every feature and both heads' posteriors within 1e-9 of the Python reference, with every label and segment equal.sleeppsg --section v3reproduces the Python DREAMT figures exactly, which is the end-to-end parity check on real nights.swift test:SleepStagerSettingTestspins the upgrade rules.doc_comment_lintandi18n_audit --ci mainare clean. Every commit builds on its own.Limits
Data and licences