Repository navigation
skin-temp: backfill absolutes for nights outside the 21-night rescore window (#1853) - #1919
Conversation
… window (ryanbr#1853) The nightly ABSOLUTE skin temperature (`skinTempC`) is written only by the scoring pass, which covers a rolling 21-night window. Nights already outside that window when the column shipped were never revisited, so a wearer sees "+0.5 Δ°C" with no way to learn what it moved from — the deviation is stored but the absolute is NULL. This adds a targeted historical backfill that re-derives the absolute from the SAME raw streams and the SAME `skinTempFunnel` the scoring pass uses, then writes it with a FILL-ONLY UPDATE that can only fill a NULL — never an upsert of a rebuilt row, never overwriting a measured value, never touching `skinTempDevC` or any other scored field. Three correctness rules (re-review findings from the Android twin, ryanbr#1852): 1. Attribute sessions by END. The night window is 54 h wide and contains the previous night; handing the funnel the whole window averages two nights into one temperature. `sessionsForDay` filters by session-end-in-day. 2. Page the candidates. A plain "oldest N" latches on page one (the nights most likely to have lost their raw samples) and never reaches the newer nights that can fill. `page(candidates:page:pageSize:)` walks oldest→newest. 3. The WHOOP 4.0 anchor comes from the CURRENT window. A re-learned anchor puts backfilled nights on a different offset from the stored ones and the chart plots two scales as one line. No anchor ⇒ DECLINE, never the global 826 fallback. Pure helper (`StrandAnalytics/SkinTempBackfill.swift`) + fill-only storage (`WhoopStore.fillSkinTempC`) + I/O walker (`Strand/Data/SkinTempBackfillWalker.swift`) + tests for session attribution, the anchor rule, the sample floor, the page rule, and the fill-only write (fills NULL, refuses to overwrite, no orphan). Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
|
Thanks @evoveotech — the three rules from the Android post-mortem are all here and correctly implemented. I verified them rather than reading the summary:
15/15 CI. The computation is not my concern here. The concern is that this is build fiveThis feature shipped four times and was reverted twice, on 3 and 4 September. The revert commit is explicit about the pattern:
That is not a criticism of the code. It is that the part which has failed four times is precisely the part this PR does not contain, and the three rules it does contain all guard the part that never failed. What would make this mergeable
Given the history, I would rather this waited for the wiring and the raw-sample check than became the fifth build that reads as "already done". |
…e action (ryanbr#1853 review) The walker had no call site — 646 lines that could not run, the same failure the four prior reverted builds arrived at a different way. This adds the wiring and the gate it sits behind. The call site: an on-demand "Backfill skin-temp absolutes" button in the Test Centre's Diagnostic Tools card (the issue's "diagnostic first" + safest trigger — doing it automatically on upgrade risks a long first launch on a deep history). The action is disabled while a pass is running and shows the result line under the button. The gate: the walker now owns the WHOOP 4.0 window-anchor scan in a self-contained `runBackfill()` entry point, so the call site does not duplicate engine internals. It replicates the scoring pass's window-wide `skinTempSamples` read + `Whoop4SkinTemp.deviceAnchorRaw` learn (rule 3: the anchor comes from the CURRENT scoring window, never the global 826 fallback), then pages through the candidate nights until a page is empty or the page budget is reached. Evidence the candidate nights still have raw samples: the result line is self-attesting. It reports filled / declined / no-raw-data counts. If `noRawData` equals the candidate count, the raw streams have aged out and the backfill fills nothing — the diagnostic the issue's unsettled unknown asked for, produced by the action itself rather than claimed in advance. Android note: the Android twin is currently reverted, so this PR ships Apple-only deliberately. The pure half (`SkinTempBackfill` in StrandAnalytics) and its tests are cross-platform; the Android wiring will follow once the Android twin's call-site question is settled. Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
|
Thanks for the review — all three points addressed in 1. The call site, and the gate it sits behindThe walker now has a call site: an on-demand "Backfill skin-temp absolutes" button in the Test Centre's Diagnostic Tools card (the issue's "diagnostic first" + safest trigger — doing it automatically on upgrade risks a long first launch on a deep history, exactly the #1854 failure mode). The action is disabled while a pass is running and shows the result line under the button. The gate it sits behind is NOT a new-data gate (the #1854 trap). It is a user tap. The backfill runs only when explicitly asked, so it can never be wired into a condition under which it has nothing to do — it runs precisely when there is something to do. The walker now owns the WHOOP 4.0 window-anchor scan in a self-contained 2. Evidence the candidate nights still have raw samplesThe result line is self-attesting. It reports If 3. A note on the Android sideThe Android twin is currently reverted, so this PR ships Apple-only deliberately. The pure half ( |
The i18n gate was red: the button's description was a new literal with no catalog entry, so it would have shipped English to ten locales. Added with translations. And the button asked for "therometer.medium", which is not an SF Symbol - the name is thermometer.medium, spelled correctly in LiquidTodayView and AppleWatchAboutView. An invalid symbol renders as nothing, so the button would have shipped without its icon and no check would have said so. While adding the translations I put an em-dash in the Russian, against the standing no-em-dash rule for user copy that I had just applied to another PR an hour earlier. Replaced. Worth writing down rather than fixing quietly: the rule is easy to keep in the string you are thinking about and easy to lose in the nine you are translating.
It was a @mainactor class doing a job with no UI: up to ten pages of fifty nights, three stream reads each. Every await yields the main thread, so this was never a freeze - but each resumption and all the per-night work between them landed on it, which is a long diagnostic pass competing with the UI for no reason. Nothing here needs the main actor. WhoopStore is an actor, DeviceRegistryStore is a Sendable struct, Repository.whoopSource is an immutable Sendable static, and the result types are already Sendable. The call site already awaited runBackfill(), so it is unchanged. Stated plainly: nothing local compiles this target, and `swiftc -parse` resolves no isolation, so this change is verified by the macOS and iOS build legs and nothing else. If they reject it the isolation goes back and the note stays.
The conflict was mine to cause: I appended a catalog entry to the tail of Localizable.xcstrings on this branch while ryanbr#1925 appended to the tail of the same file on main. Two appends to one anchor point is a conflict by construction, and the ordering was my choice - the branch entry could have gone in after the merge instead of before it. Resolved by taking main's file and re-splicing this branch's key onto it, so ryanbr#1925's two keys and the backfill description all survive. Verified rather than assumed: 3743 keys, all three present, JSON valid.
Summary
Addresses #1853.
The nightly ABSOLUTE skin temperature (
skinTempC) is written only by the scoring pass, which covers a rolling 21-night window. Nights already outside that window when the column shipped (#1636) were never revisited, so a wearer sees "+0.5 Δ°C" with no way to learn what it moved from — the deviation is stored but the absolute is NULL.This adds a targeted historical backfill that re-derives the absolute from the same raw streams and the same
skinTempFunnelthe scoring pass uses, then writes it with a fill-only UPDATE that can only fill a NULL — never an upsert of a rebuilt row, never overwriting a measured value, never touchingskinTempDevCor any other scored field.Three correctness rules (re-review findings from the Android twin, #1852)
Attribute sessions by END. The night window is 54 h wide and contains the previous night; handing the funnel the whole window averages two nights into one temperature.
SkinTempBackfill.sessionsForDayfilters by session-end-in-day, matching the engine'smatched = allSessions.filter { tsInDay(it.end) }.Page the candidates. A plain "oldest N" latches on page one (the nights most likely to have lost their raw samples) and never reaches the newer nights that can fill.
SkinTempBackfill.page(candidates:page:pageSize:)walks oldest→newest, 50 per page by default.The WHOOP 4.0 anchor comes from the CURRENT window. A re-learned anchor puts backfilled nights on a different offset from the stored ones and the chart plots two scales as one line. No anchor ⇒ decline, never the global 826 fallback.
What's in the change
StrandAnalytics/SkinTempBackfill.swift— the pure, database-free half: session attribution (rule 1), the anchor rule (rule 3), and the per-night plan that calls the existingAnalyticsEngine.skinTempFunnelso a backfilled absolute is byte-identical to what the scoring pass would have stored.WhoopStore/MetricsCache.swift—fillSkinTempC(deviceId:day:skinTempC:): a single-columnUPDATE … WHERE skinTempC IS NULLthat returns rows filled. Fill-only by construction.Strand/Data/SkinTempBackfillWalker.swift— the I/O layer: reads candidate computed rows (skinTempDevC != nil && skinTempC == nil), resolves the day owner / device family / worn tolerance via the sameIntelligenceEnginehelpers the scoring pass uses, reads the raw streams, calls the pure helper, and writes the fill-only result. On-demand (Test Centre trigger), paged, idempotent.StrandTests/SkinTempBackfillTests.swiftcovers session-by-end attribution, the inclusive/exclusive day boundaries, the WHOOP 4.0 anchor decline, the 5/MG fill path, the sample floor, the previous-night-not-matched case, and the page rule.DailySkinTempAbsoluteTestsgains three fill-only storage tests: fills NULL + touches nothing else, refuses to overwrite an existing absolute, no-op when the row doesn't exist.Trigger
On-demand from the Test Centre (the issue's "diagnostic first" + safest trigger). Doing it automatically on upgrade risks a long first launch on a deep history. The caller decides how many pages to run.
Test plan
swift testoverPackages/(WhoopStore fill-only tests)xcodebuild -scheme Strand build(app-target compile — the walker is app-target Swift)xcodebuild -scheme Strand test(StrandTests — pure helper tests)xcodebuild -scheme NOOPiOS build(iOS compile — the walker is shared with iOS)doc_comment_lint(no new detached doc comments)Generated with Devin