Skip to content

sleep: compute the RSA transform's twiddle factors once per night - #2575

Merged
ryanbr merged 1 commit into
ryanbr:mainfrom
UtkuDenizAltiok:stager-dft-table
Sep 28, 2026
Merged

ryanbr merged 1 commit into
ryanbr:mainfrom
UtkuDenizAltiok:stager-dft-table

Conversation

@UtkuDenizAltiok

Copy link
Copy Markdown

What this PR does

SleepStagerV2.respRegularity (the RSA respiration-regularity term) runs once per 30-second epoch and evaluated cos/sin for every in-band bin and every resampled point, each time: ~53 bins × ~836 points, so ~45k pairs per epoch and ~45 million per night. The factors depend only on the 4 Hz grid length n, and a night's beat window is whole seconds wide, so a night sees only a handful of lengths. This computes each length's factors once per night in a small table (RespDFT / RespDft), with the very expressions the transform used inline, so the result is bit-identical. Both platforms.

Measured on a real WHOOP 5.0 store (a backup, 4.5 days; iPhone 17 Pro simulator, Release):

  • Time Profiler: in the first re-score of a process (every night staged cold), respRegularity + __sincos_stret were 0.64 s of 5.0 s CPU (13%).
  • First (cold) pass, alternating builds (upstream/main vs this), two rounds: 4.9 / 5.0 s → 4.5 / 4.4 s CPU. Warm passes are unchanged there, because that harness's synthetic offload lands after the night and the stager's own cache serves it; a pass that re-stages a growing night (every overnight re-score) saves the same per-epoch work.
  • Memory: one table per distinct n per night, ~53 × ~836 × 2 doubles ≈ 0.7 MB each, dropped with the night.

Type of change

  • Refactor / cleanup (same results, less work)

How it was tested

  • Swift: SleepStagerRespDFTTableTests keeps the function as it was, verbatim, and compares bit patterns with the table version over 3,000 random beat windows built the way features() builds them (whole-second times, 0–2 beats a second, full-width and short/sparse windows, >2,000 reaching the transform, >3 grid lengths sharing one table), plus a check that a warm and a fresh table give the same answer. Release and debug both pass. With the table's sine one ulp off (.nextUp), 2,125 comparisons fail; restored byte-identical (sha256).
  • Kotlin: SleepStagerRespDftTableTest, the same design against SleepStagerV2.respRegularity. Android CI on the fork: green (6,536 tests); with the table one ulp off (a throwaway branch), exactly this test fails.
  • Tools/SleepPSG: swift test (port validation, 33 tests) passes: the shipped stager and the recipe port still agree on every epoch label.
  • Parity: respRegularity is now a declared twin pair (both docs name the other), and the refreshed derived inventory is in the commit: function pairs 206 → 207, unpaired functions −2. parity_ledger.py: no new findings; parity_ratchet.py: 0 errors.
  • Full local run on 7c298e93 (on 0c982899), every step passed: WhoopStore 629 · StrandAnalytics 2083 · StrandImport 327 · doc lint · i18n · parity ledger · ratchet · governance (124) · macOS StrandTests 2,244 (only the two locale-dependent TodayCarryOverTests) · iOS build.

Checklist

  • Swift package tests pass for any package I touched (swift test in Packages/StrandAnalytics)
  • Android unit tests pass if I touched android/ (fork CI)
  • No new build warnings introduced
  • Follows the conventions in docs/CONTRIBUTING.md
  • I did not commit generated output (Strand.xcodeproj/) or any secrets/keystores

Related issues

Refs #1538

🤖 Generated with Claude Code

SleepStagerV2.respRegularity runs once per 30-second epoch and evaluated
cos/sin for every in-band bin and sample each time (~53 x ~836 pairs). On a
WHOOP 5.0 store that was 13% of a cold re-score's CPU (Time Profiler,
__sincos_stret), and every pass that re-stages a growing night pays it too.

The factors depend on the 4 Hz grid length alone, and a night's beat window
is whole seconds wide, so a night sees only a handful of lengths. A per-night
table (RespDFT / RespDft) computes each factor with the very expressions the
transform used inline, so the result is bit-identical. Swift and Kotlin each
keep the old function in a test and compare bit patterns over 3,000 random
beat windows; a table one ulp off fails 2,125 of them.

The derived parity inventory is refreshed: respRegularity is now a declared
twin pair (function pairs 206 -> 207, unpaired functions -2).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
UtkuDenizAltiok added a commit to UtkuDenizAltiok/noop that referenced this pull request Sep 28, 2026
UtkuDenizAltiok added a commit to UtkuDenizAltiok/noop that referenced this pull request Sep 28, 2026
), ryanbr#2569 merged, build 769113c; rescore-profile tools; STATE rewritten for a fresh session
@ryanbr

ryanbr commented Sep 28, 2026

Copy link
Copy Markdown
Owner

@UtkuDenizAltiok Reviewed. 18/18 green. The bit-identity argument is the whole change and it holds.

The table builds each factor with the same expressions the transform used inline, same w, same a = w * j, same accumulation order, so reading it back is bit-for-bit what recomputing gave. The cache key is n alone and that is sufficient because kLo and kHi are pure functions of n, so no two calls with the same n can want a differently shaped table. On the Kotlin side -2.0 * PI * k / n groups identically to the inline expression it replaces, so each platform preserves its OWN prior output exactly, which is the right contract here: this change is not making the two platforms agree, it is leaving each where it was.

The tests are the strongest part, and they do the two things a memo change needs.

XCTAssertEqual(now?.bitPattern, before?.bitPattern) against a verbatim copy of the old function is the correct assertion. A tolerance would have passed a change that moved stage labels. And two guards stop the equality from being vacuous: nonNil > 2000 of 3,000 windows, so most comparisons reach the transform at all, and dft.count > 3, so several grid lengths genuinely share one table rather than the cache being exercised once. I have shipped a vacuous equality test in this repository myself, so I went looking for exactly that and did not find it.

testTheAnswerDoesNotDependOnWhatTheTableHeldBefore is the hazard specific to memoisation: a warm table must not make a later window answer differently from a cold one. Having both tests means a correct implementation and a correct cache are checked separately.

One thing I checked that is not visible from your branch. #2576 landed after you opened this and widened Kotlin baseLogPrior from private to internal, in the same files. A visibility change can move what the ledger sees, and #2576 touched no Tools/parity_* path so parity-governance never ran on it. If that had left main's stored authority unreproducible, your re-derived authority would have been computed against the wrong tree. It did not: the ratchet on main reports errors=0, and main's authority is byte-identical to the base you derived against, so both trees carry the same declaration set and your twin_map delta stays correct across the rebase.

One note, not a defect. Each table is about 53 x 836 x 8 bytes twice over, so roughly 700 KB per distinct n. The window is fixed width, so a night sees one full-width n plus a few edge cases and holds a few megabytes that go when the night does. Worth a reader knowing; not worth changing.

The 13 % figure is consistent with what a field log shows independently: on a 5.0/MG install this morning, cold re-score passes spent 68 to 80 seconds in read and prep against 3 seconds scoring, so shaving the per-epoch transform is aimed at the right half of the pass.

Taking it.

@ryanbr
ryanbr merged commit f411030 into ryanbr:main Sep 28, 2026
18 checks passed
ryanbr added a commit that referenced this pull request Sep 29, 2026
`main` fails its own ratchet at 5de0519, on a clean tree, with no change
applied:

  ERROR: Tools/parity_ledger_baseline.json: accepted finding group regressed
  test-only-callsite|Packages/StrandAnalytics
  ERROR: Tools/parity_ledger_baseline.json: accepted finding group regressed
  test-only-callsite|android/analytics

I caused it by merging #2557. That change took the daily resting HR from the
primary night's gated five-minute floor instead of the whole-session mean,
which replaced the only production call to `primarySessionRestingHR` on both
platforms:

  -  ?? primarySessionRestingHR(physiologySessions, hr)?.roundToInt()
  +  ?: primarySession?.restingHR

The function is now reached from four test files and nothing else, so the
lexical call graph emits one new `test-only-callsite` finding per platform. The
checked-in baseline did not record them. This records them: 43 to 44 in
`Packages/StrandAnalytics`, 64 to 65 in `android/analytics`, two lines, nothing
else moved.

Why nothing caught it: #2557 touched no `Tools/parity_*` path, so
parity-governance never ran on it. Its seventeen green checks were real and did
not include the gate that governs call-site structure. #2575 and #2576 landed
the same way today and happened to be harmless.

Why my review did not catch it: I checked that
`primarySessionRestingHRWithCoverage` still had callers and concluded the old
mean was not dead code. That is true of the `WithCoverage` variant and false of
the plain function, which is the one #2557 orphaned. I verified the neighbouring
symbol.

Recording rather than deleting, deliberately. The findings are true, and the
baseline exists to hold accepted findings. Whether `primarySessionRestingHR`
should survive as a tested-but-unused function, or be deleted with its tests
repointed at the `WithCoverage` variant, is a change to code merged an hour ago
and belongs in its own review. A governance repair that also deletes product
code gets neither part looked at properly.

This does not make the debt go away, it makes it visible. The next change in
either scope inherits 44 and 65 as the ceiling, so a second orphaned function
there fires the gate again.

Refs #2557, #2567
@UtkuDenizAltiok
UtkuDenizAltiok deleted the stager-dft-table branch September 30, 2026 10:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants