SleepStagerV2.cyclePrior: the REM step is a fraction of session length, so the REM-latency guard varies 7-85 min across one user's nights

@ryanbr

SleepStagerV2.cyclePrior carries two hardcoded REM terms that aren't fit to anything and aren't documented beyond a one-line comment. I ran an ablation to find out what each one actually does, and the answer surprised me enough that I think it's worth writing down. One of them has a real design bug; the other is doing useful work and should not simply be deleted.

Filing as an issue rather than a PR because the right fix is a judgement call, and because I only have one user's data.

The code

Packages/StrandAnalytics/Sources/StrandAnalytics/SleepStagerV2.swift:385-389:

/// Soft sleep-cycle prior added to the log-emission: deep concentrated early (decays, never hard-wiped);
/// REM suppressed in the first ~12 % (REM latency) then rising toward morning.
static func cyclePrior(_ c: Double) -> [String: Double] {
    ["deep": 1.2 * max(0.0, 1.0 - c / 0.55),
     "rem": 1.0 * c - (c < 0.12 ? 3.0 : 0.0),
     "light": 0.0, "awake": 0.0]
}

Kotlin twin byte-identical at android/app/src/main/java/com/noop/analytics/SleepStagerV2.kt:422-425. Added to the log-emission at SleepStagerV2.swift:467 before Viterbi.

c is Epoch.clock, "time-of-night fraction in [0, 1]" (:194), computed Double(e + 15 - start) / span (:313) — a fraction of the session, not elapsed minutes.

The REM entry is two separate things: a linear ramp 1.0 * c, and a step of −3.0 while c < 0.12. In log-odds, −3.0 is a factor of e⁻³ ≈ 0.05 on the REM emission.

What each term actually does

Ablation over 21 unedited nights (116.2 h) from one WHOOP 5 user — "unedited" so the hypnograms are pure V2 output with no hand-corrections mixed in. REM share = REM epochs / all epochs in that hour-into-session.

hour	as shipped	ramp only (step removed)	whole REM prior removed
0–1	6.8%	15.2%	10.8%
1–2	20.7%	20.7%	10.2%
2–3	27.4%	27.4%	13.8%
3–4	28.4%	28.4%	8.9%
4–5	25.2%	25.2%	5.7%
5–6	33.2%	33.2%	6.6%
6–7	36.3%	36.3%	6.5%
7–8	38.0%	38.0%	6.1%
8–9	73.8%	73.8%	17.2%
9–10	77.8%	77.8%	7.3%
10–11	96.2%	96.2%	10.0%
11–12	94.0%	94.0%	15.0%
total REM	29.8%	31.3%	9.7%
median first-REM latency	82.0 min	10.0 min	11.5 min

Two clean separations, and neither is what I expected going in:

The 1.0 * c ramp produces the entire rising shape. Remove it and REM share is flat — 6–17% with no trend — and total REM collapses from 29.8% to 9.7%. Whatever else is true, this term is load-bearing; without it the stager badly under-calls REM overall.

The −3.0 step does exactly one thing: it sets first-REM latency. It changes hour 0 and nothing else — hours 1–11 are epoch-for-epoch identical with and without it. On its own it moves median first-REM latency from 10.0 min to 82.0 min.

Also worth correcting, since I'd assumed otherwise before measuring: it is not a hard mask. REM still occurs at 6.8% during hour 0 as shipped. −3.0 log-odds is strong suppression that sufficient evidence can overcome, not a veto. Any framing of this as "0% REM in the first cycle" (including framings of mine) is wrong.

The actual bug: the guard is a fraction, but REM latency is an absolute quantity

c < 0.12 is 12% of that session's length. So the width of the suppression window scales with how long you slept:

session	guard width
4 h	28.8 min
6 h	43.2 min
8 h	57.6 min
10 h	72.0 min

On this one user's own 21 nights (spans 1.02 h – 11.74 h, median 5.09 h) the guard ranged from 7.4 min to 84.5 min — an 11× spread, median 36.6 min.

That's backwards from the physiology. First REM onset is an absolute latency of roughly 70–100 minutes in healthy adults; it doesn't stretch because you slept in. And the direction of the error compounds: a short night is disproportionately likely to be a sleep-deprived night, and sleep deprivation shortens REM latency (REM rebound) — so exactly when true REM latency is shortest, this guard is also narrowest, which is right by accident, but for a nap or a fragmented 4 h night the 12% fraction has no physiological meaning at all. The 1.02 h session in this dataset got a 7.4-minute REM guard.

There's a second-order concern I want to flag as inference, not measurement: REM latency also shortens with age, and a minority of the general population shows sleep-onset REM periods. A fixed suppression window is hardest on exactly those users, and they're not a group we can identify from the data to exclude. I have no data on this and one user's nights can't speak to it — I raise it only because it argues against tuning the constant and for changing its shape.

What I'd suggest (and what I'd avoid)

Don't just delete the step. Median first-REM latency without it is 10 minutes, which would mean a sleep-onset REM period essentially every night. That's clearly worse than what ships. The step is solving a real problem.

Do make it a graded function of absolute elapsed time, rather than a step at a fixed fraction of session length. Something like a smooth ramp over roughly the first 60–100 minutes of elapsed sleep gives the same suppression where the physiology supports it, degrades gracefully instead of cliff-edging, and stops varying 11× with session length. This is also the direction the wearable-staging literature has gone — Walch et al. (SLEEP, 2019) reported meaningful accuracy gains from feeding circadian/clock information as a continuous input feature rather than a hard rule. I'm citing that as a direction of travel, not as a result I've reproduced.

Whatever the shape, the constants deserve a provenance note. 1.2, 0.55, 1.0, 0.12, 3.0 currently have no recorded derivation. Even "hand-picked, never fit" written in the comment would be more useful than silence — right now a reader can't tell these from tuned values, which is what led me to run the ablation in the first place.

Caveats on the above

n = 21 nights, one user, one strap family (WHOOP 5), one age. The per-hour percentages are descriptive of this dataset and should not be read as population values. The structural findings — which term causes which effect, the step touching only hour 0, the 11× guard-width spread — are properties of the code and hold regardless of whose nights you run.
The late-night rows rest on very little data (hour 10–11 is 240 epochs = 2 h total across all nights). The shape is clear; those specific percentages are noisy.
The ablation was run by patching cyclePrior in a throwaway worktree and replaying with Tools/SleepBench (proposed in Sleep: correct which stager ships (V2, not V1), and add SleepBench so the claim is measurable #925); nothing here changes shipped code.
No hardware or firmware was probed; this is all code-reading plus replay of already-recorded nights.

Happy to prototype the graded-elapsed-time version behind a flag and score it with SleepBench if that's a direction worth exploring — but given #348/#437, I'd rather agree on the shape and on how we'd measure success before sending a tuning change. @ryanbr @tanarchytan — flagging you both since #348 and the #437 revert are the closest prior art on changing V2's staging constants.

@ryanbr

Two additions to the above — one sharpens the diagnosis, the other constrains how any fix should be evaluated. Both change my view of what the real problem is, so I'd rather post them than leave the issue as filed.

1. The term isn't just untuned — it's dimensionally wrong, at both ends

I framed the bug above as "12% is a fraction, and REM latency is absolute." That's right but it understates it, because both ends of the measurement are unanchored:

The scale is a fraction of session length. Already measured above: 7.4–84.5 min across 21 nights, an 11× spread. A prior about a ~70–100 min physiological latency should not be 11× wider on a long night than a short one.

The origin is the detected sleep start, not a verified onset. c = Double(e + 15 - start) / span (SleepStagerV2.swift:313) measures from whatever detectSleep decided the session began. That's not a free parameter — it inherits detection error. And detection error here is documented and large: #271 ("detected onset lands too early") reports a 06:15 sleep detected as starting ~01:58 — a ~4 hour displacement of the origin.

So the suppression window can be the wrong length and start from the wrong place. A 12% window anchored four hours early doesn't suppress the first REM period at all; it suppresses a stretch of pre-onset wakefulness, and then leaves the genuine first cycle fully unsuppressed.

Scoping that honestly: #271 is a WHOOP 4.0 issue — sparse motion making "lying still" hard to separate from "asleep." My 21 nights are WHOOP 5, where I have no evidence of that failure mode, so I am not claiming the origin is unreliable in the data I measured. The structural point is what matters: this term is defined relative to a detection output with a known, unfixed failure mode on at least one strap family, and nothing in cyclePrior degrades gracefully when that origin is wrong. A prior anchored in minutes since a verified onset would at least fail visibly instead of silently.

That's the cleanest statement of the problem: the term is not merely fit to nothing — it is parameterised in the wrong units, measured from an unreliable origin. The −3.0 magnitude and the 0.12 threshold are second-order next to that.

2. κ is not sufficient evidence for a fix here, and this repo already has the scar

This is the part I'd most want agreed before anyone (me included) sends a patch.

#348 was, by conventional ML criteria, a clean piece of work: fitted on DREAMT, κ improved on the held-out benchmarks. #437 reverted it 48 hours later, and its diagnosis is worth quoting exactly:

The tune's no-regression guardrail was kappa (AAUWSS +0.03, Walch +0.006), and kappa doesn't guard stage-fraction calibration — a systematic wake-fraction bias sails through it.

The field symptom was a healthy night re-scoring from awake 6% → 23% (30 min → 1h 47m), efficiency 94% → 77% — while κ said the change was an improvement. Root cause was cohort bias: DREAMT is a clinical OSA cohort with high true WASO, and the tune absorbed that base rate into the population prior.

This matters directly for cyclePrior, because a cliff is precisely the kind of term that moves stage fractions without moving κ much. The −3.0 step relocates REM within the night rather than changing how much of the night is classified correctly overall. My own ablation shows the mechanism cleanly: removing the step changed total REM by only 1.5 points (29.8% → 31.3%) while moving median first-REM latency by 72 minutes (82.0 → 10.0). A κ-based guardrail would barely register that. A clinician looking at the hypnogram would see a completely different night.

So any proposal here — mine included — should report, before and after:

per-night stage fractions (REM %, deep %, light %, wake %), not just pooled κ
first-REM latency distribution, since that's the quantity this term actually controls
the healthy-sleeper case specifically, since that's the cohort feat(sleep): re-tune SleepStagerV2 deep boundary on DREAMT (n=100 gold) #348 broke and the one benchmark cohorts under-represent

And it should say out loud that κ was insufficient last time. Tools/SleepBench (#925) already reports per-stage sensitivity/specificity and per-night variance; per-night stage fractions and first-REM latency are not in it yet, and I'll add them before proposing any change to this term — the measurement should land before the tuning, which is the lesson I take from #348/#437.

I've deliberately not opened a PR for the fix. Given that history, agreeing the shape (graded vs. threshold, absolute vs. fractional) and the success criteria first seems much more likely to produce something that survives contact with real nights.

@ryanbr — the #437 write-up is what made this argument available; the "kappa doesn't guard stage-fraction calibration" line is doing a lot of work here. @tanarchytan — none of the above is a criticism of #348's method; it's an argument that the guardrail we all had at the time couldn't have caught what went wrong, and that this term has the same signature.

Opened #934 with the fix. Summary of what it does and — more importantly — what it does not claim.

The shape is what this thread argued for: K · clamp01(1 − m / M0) with K = 3.0, M0 = 60 min, m measured in minutes from a detected sleep onset rather than from the window start. K is the incumbent step's own magnitude, kept deliberately; M0 is new. The 1.0 * c ramp and the whole deep term are untouched — both are correctly fractions of the session, and ablating the ramp collapses REM to 1.4 % of night, so it stays exactly as it is.

It is a robustness fix, not an accuracy win, and the PR says so up front. On PhysioNet sleep-accel (n = 31 subjects, LOSO) the −3.0 step is measurably inert: κ 0.349 → 0.349 and median first-REM latency 142.0 → 141.0 min when removed, and all 210 grid cells ({cliff, graded} × {fraction, minutes} × K ∈ 1…8 × threshold) tie. Anyone reviewing it should reject an accuracy claim, because there isn't one.

What is settled is the units question. True first-REM latency is uncorrelated with session duration (Pearson r = −0.058, Spearman −0.076, n = 30 subjects with a scorable first REM period), and re-expressing it as a fraction makes it more variable, not less (CV 0.617 as a fraction vs 0.537 in minutes). That cohort spans only 2.4× in session length, so it understates the problem relative to the 11.5× spread in the 36 sessions I reported here.

On the measurement commitment I made above — I said per-night stage fractions and first-REM latency should land in Tools/SleepBench before any tuning. Being straight about it: I built and ran that measurement, and #934 reports the full before/after (per-stage fraction bias vs a human reference, a healthy-sleeper stratum, and the first-REM latency distribution), but I ran it from a separate harness rather than landing the additions inside Tools/SleepBench itself, to keep #934 to one concern. Happy to send the SleepBench additions as their own small PR if you'd rather have the reusable measurement in the tree first — say the word and I'll do that before #934 is reviewed.

One correction to my own framing worth recording. I chose the stage-locked (stagelock cursor) rows as the before/after reference rather than SleepBench's "byte-exact replay of the current V2" exclusion, because that exclusion is version-dependent by construction — a night can leave it simply because the recipe changed, which silently swaps the denominator underneath a before/after. It caught me on the first pass: 11 reference nights before the change, 12 after, and the κ delta computed off those two sets was meaningless. Anyone scoring a V2 change should pin the reference set to something the recipe cannot move.

The two defects that came out of this and are not fixed by #934, flagged there as known and separately tracked: median first-REM latency runs ~54 min late against PSG (142 vs 88.5 min truth — and removing the step gives 141, so this term is not the cause), and deep is under-called 2–3× (4.7–6.2 % predicted vs 14.8 % truth).

Didn't wait for the word — that commitment is now met in the tree: #935.

Tools/SleepBench gains per-stage fraction calibration (signed bias in pp, per night and aggregated, plus MAE), a healthy-sleeper stratum (in-bed ≥ 5 h) reported separately from the aggregate, and the first-REM latency distribution per night and as a median. κ, accuracy, sensitivity/specificity and the wake-minute and band-state sections are untouched — it adds, it doesn't replace. Read-only properties are unchanged (DB.swift not touched; still SQLITE_OPEN_READONLY | immutable=1, DB path still a required argument).

Two things worth recording here rather than only in the PR.

The in-tree harness reproduces #934's reported figures exactly. Same 36-session database, same 15 stage-locked references (9,163 epochs), replayed against #935's base and base + #934:

	base	base + #934
V2 4-class κ	0.691	0.689
stage bias — wake / light / deep / REM, pp	−7.26 / −7.54 / +6.65 / +8.15	−7.13 / −6.10 / +6.65 / +6.57
first-REM latency, median / minimum	72.2 / 10.0 min	83.2 / 43.0 min
healthy-stratum wake fraction	9.40%	9.43%

Independently rewritten from the scratchpad harness, so the agreement is a check on the numbers #934 reports, not a restatement of them. κ moved −0.002 there and would have called the change nothing.

The reference-set point from my last comment is now enforced in code, with the reasoning next to it. #935's calibration sections score the stagelock rows rather than SleepBench's byte-exact-V2 exclusion, and say why in a doc comment on the filter: the exclusion is version-dependent, so a night can enter or leave it because the recipe changed, swapping the denominator underneath a before/after. Both counts are printed so the difference is visible instead of implied. That's the trap that caught me on the first pass (11 reference nights before, 12 after), and it now can't catch the next person silently.

13 new unit tests, no database needed (swift test in Tools/SleepBench) — they pin the sign convention, that an unused stage reads 0% rather than a missing key, that latency is measured from onset and not window start, and that a night with no REM is nil rather than zero.

One caveat stated in the PR: Tools/SleepBench isn't in swift-packages.yml's matrix, so these tests don't run in CI yet. Adding it changes the working-directory shape for all eight existing package jobs, which is a separate concern — happy to follow up.

#935 is independent of #934 and doesn't touch SleepStagerV2, so it can land in either order; the measurement landing first is the sequence I argued for above.

Measure the REM-latency guard in minutes since onset, not as a fracti…

Three corrections to numbers I posted in this thread. All of them came from a harness that no longer exists; it has been rebuilt in the tree with tests as Tools/SleepPSG (#991), and it does not reproduce all of what I reported.

1. The physiology-beats-clock margin is +0.074, not the +0.329 I have been quoting.

LOSO logistic regression — one whole subject held out per fold, decision threshold tuned on the training folds and applied unchanged to the held-out subject:

model	pooled REM F1	mean per-subject
clock only	0.431	0.408
physiology only	0.479	0.444
both	0.522	0.482

Adding physiology to the clock is +0.074 mean per-subject (+0.091 pooled); physiology alone over clock alone is smaller still, +0.036 per-subject (+0.048 pooled). The direction holds — physiology − clock is positive in 22/31 subjects, both − clock in 26/31 — so "REM detection reads physiology, not the clock" stands. The magnitude does not. The old +0.329 was inflated by a weak baseline: the clock model was scored at a fixed 0.5 threshold against a ~22 % minority class, which under-predicts REM and hands the comparison a win. A clock model with a train-tuned threshold is much harder to beat, and that is the honest baseline.

2. The REM/night-length coupling does not reproduce here — but this is a weak test and I am not calling it a refutation.

corr(scored night length, REM % of sleep) on sleep-accel: predicted +0.037, where PSG truth itself carries +0.398. Taken at face value that is the opposite of the +0.579 I measured across 36 of my own nights. It should not be taken at face value: sleep-accel nights cluster tightly around 476 min (range 208–490), against the 166–654 min spread the +0.579 came from. This cohort has almost no duration leverage to detect the coupling with. So the +0.579 is untested by this instrument, not refuted, and the fixed-60-minute-penalty mechanism argued for in this thread remains the best available explanation — and remains unmeasured on a cohort with real spread.

3. The two defects I flagged here as "known and separately tracked" rest on that harness's prediction side, which does not reproduce.

I wrote above that median first-REM latency runs ~54 min late against PSG (142 vs 88.5 truth) and that deep is under-called 2–3× (4.7–6.2 % predicted vs 14.8 % truth). The truth side reproduces exactly — 88.5 min truth first-REM, deep 14.76 %, 26,773 epochs. The prediction side does not:

predicted median first-REM is 91.5 min — roughly 3 min late, not 54
predicted deep sits about 5 pp above truth: an over-call, where I reported an under-call of 2–3×

The deep number is the sharper problem, because the old harness's own reported predicted deep was 19.25 % and the rebuild measures 19.24 % — they agree to 0.01 pp. So the "4.7–6.2 %" in my comment above does not match the harness I said I took it from, and I cannot reconstruct where it came from. Until someone can, both defects should be treated as unverified, including where #934 (merged) leans on them.

The obvious explanation for the REM residual — that this issue's own change to the guard causes it — was tested with a dedicated pre-#930 variant that runs the old c < 0.12 ? 3.0 : 0.0 fraction-domain step, and it is falsified: κ 0.356 → 0.356, REM F1 0.569 → 0.569, REM 27.02 % → 26.97 %. On 8-hour lab nights the guard barely binds either way. That also re-confirms, on a second instrument, the inertness I reported here when #934 was the candidate — I said κ 0.349 → 0.349, the rebuild says 0.356 → 0.356: different absolute κ, same null. The residual is left open in the tool's README rather than tuned away.

Same limits as everywhere: n = 31 lab subjects, Apple Watch accelerometry collapsed to a per-second mean rather than a WHOOP gravity decode, and no R-R in sleep-accel — respRegularity is nil on every epoch, so the RSA/respiration term contributes exactly 0.0 to every number above.

Picking this up after the retraction, because it changes what this issue is for.

The code ask here has landed. cyclePrior now takes minutesSinceOnset and calls remLatencyGuard() — 3.0 x clamp01(1 - m/60), graded, absolute, measured from detected onset — via #934 (61380198), with the provenance note this issue asked for sitting above it. All three asks are met: the step was kept rather than deleted, its shape is now a function of elapsed time, and the constants have a recorded derivation.

More importantly: #934's justification survives your own retraction, and I think that is worth saying explicitly — because "a merged PR leans on numbers I have withdrawn" reads worse than the situation actually is.

The argument for #934 was dimensional, not empirical. The old guard was 0.12 x session length, so its width scaled with how long you slept:

 1.02 h night  ->   7.3 min guard
 5.09 h median ->  36.6 min
11.74 h night  ->  84.5 min      = 11.5x spread

That is arithmetic on the shipped constant. It needs no harness, no cohort and no labels, and it is unaffected by anything in your correction. A prior about a ~70-100 min physiological latency has no business being 11x wider on a long night — and the replacement is a fixed 60 min window, spread 1.0x, by construction.

What you have withdrawn is the accuracy framing around the residuals. #934 never claimed accuracy: your own comment on it said "a robustness fix, not an accuracy win, and the PR says so up front." The rebuild agrees — your pre-#930 variant lands κ 0.356 → 0.356, REM F1 0.569 → 0.569, i.e. inert on that cohort. Which is exactly what a robustness fix should look like on 8-hour lab nights that cluster at 476 min: the defect it fixes cannot manifest in a cohort with no duration spread. The null is evidence the change is safe, not evidence it was unnecessary.

On the retraction itself. Rebuilding the harness, finding it does not reproduce your own published numbers, and posting that unprompted — including "I cannot reconstruct where it came from" — is the part of this thread I would most want kept. Catching that the old harness's predicted deep (19.25 %) and the rebuild's (19.24 %) agree to 0.01 pp, and therefore that the 4.7-6.2 % you had quoted could not have come from where you said, is a harder thing to notice than the original finding was.

Where things stand:

The guard fix: done, and standing on its own argument.
The REM residual, the deep over-call, and the +0.579 duration coupling: open and unverified, and they belong with Tools/SleepPSG: score SleepStagerV2 against PSG truth, reproducibly #991 rather than here — that is where the instrument lives now.
This issue can close once you are happy the guard question is settled; the residuals will get better attention under the harness that can actually measure them.

One forward-looking note, since #987 also touches REM: it zeroes wake -> rem in the transition matrix while this guard suppresses early REM in the emission. Both push the same direction on first-REM latency through different mechanisms. Individually each is defensible; together they are worth a single look at latency rather than two, so the combined effect is measured once rather than inferred twice.

Tools/SleepPSG: score SleepStagerV2 against PSG truth, reproducibly (#…

Measure the REM-latency guard in minutes since onset, not as a fracti…

Tools/SleepPSG: score SleepStagerV2 against PSG truth, reproducibly (r…

Verified fixed: remLatencyGuard is implemented on both platforms (Swift: Packages/StrandAnalytics/Sources/StrandAnalytics/SleepStager.swift, Kotlin: android/app/src/main/java/com/noop/analytics/SleepStager.kt) measuring REM latency in minutes since onset, not as a fraction of session length, with tests.

Navigation Menu

Repository navigation

SleepStagerV2.cyclePrior: the REM step is a fraction of session length, so the REM-latency guard varies 7-85 min across one user's nights #930

The code

What each term actually does

The actual bug: the guard is a fraction, but REM latency is an absolute quantity

What I'd suggest (and what I'd avoid)

Caveats on the above

1. The term isn't just untuned — it's dimensionally wrong, at both ends

2. κ is not sufficient evidence for a fix here, and this repo already has the scar

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

SleepStagerV2.cyclePrior: the REM step is a fraction of session length, so the REM-latency guard varies 7-85 min across one user's nights #930

Description

The code

What each term actually does

The actual bug: the guard is a fraction, but REM latency is an absolute quantity

What I'd suggest (and what I'd avoid)

Caveats on the above

Activity

vishk23 commented on Jul 28, 2026

1. The term isn't just untuned — it's dimensionally wrong, at both ends

2. κ is not sufficient evidence for a fix here, and this repo already has the scar

vishk23 commented on Jul 28, 2026

vishk23 commented on Jul 28, 2026

vishk23 commented on Jul 30, 2026

ryanbr commented on Jul 31, 2026

evoveotech commented on Sep 10, 2026

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions