Activity

1. ryanbr commented on Jul 26, 2026

   

   ryanbr

   on Jul 26, 2026

   Owner

   More actions

   Good inventory, and the framing question is the right one to lead with. Answers where the tree actually knows something, and a plain "nobody knows" where it does not.

   1. The unclear ones.

   cardiac_flags@33 / cardiac_status@40 — the names are positional, not derived. BLE_REVERSE_ENGINEERING.md line 432 is explicit: carried raw, "meaning not pinned from observation" — a flag-ish byte and a status-like byte that happen to sit next to the HR fields. No decompile, no capture behind them. Your instinct to check before reading meaning into the names is right, and there is a worked example in that same row: whoop-local names @40 signal_quality, but it appears only in that project's code with no stated source and no supporting analysis, so we kept it raw. Four distinct values across your records is genuinely new information — worth adding.

   status_word_1@77 / status_word_2@79 — the channel-index reading is a real hypothesis and nobody has tested it. @75's low nibble is 0 across ~258k records, and 1/2 for the siblings, which is at least consistent with an index. Untested.

   aux_byte_82 — the ~258k corpus is not in-tree; it survives only as cited results in the doc and the decoder comments. Treat those numbers as attributions, not something you can re-query.

   2. hr_fixed_8_8@36 is not a plain wiring gap. The evidence is already in-tree — corr 0.989 vs heart_rate@22 over ~258k records, with real sub-bpm fractions. But hrSample.bpm is Int on both platforms, so consuming it means either a schema change or discarding the precision at the door, and it moves resting HR, the HR curve, zones and every derived metric at once. Worth doing, but as a deliberate metric change with a migration, not a one-line swap.

   The temp_aux pair is the better-shaped one, and your point that they are offset rather than redundant is the key constraint — any fusion has to be in deviation-space. A validity cross-check against @73 needs no new metric and would speak directly to the "skin temp shows nothing" reports.

   3. Agreed on spo2_candidate_82, and it is the strongest item here. The decoder comment states the evidence is split: an 8-night independent validation at corr +0.99 tracking both a 92 % desat and a 98 % high, against two nights on the original #103 device that moved opposite to the app value. That contradiction cannot be resolved from one device, and right now only the offline tooling can produce the data — so the app genuinely cannot answer its own question. Persisting it as instrumentation (no card, no score, guard test untouched) is the unblock. Please do open that.

   4. Yes — make the diagnostic pattern the default answer. It is not hypothetical: dynamic_acceleration@41 shipped exactly that way in #813 as DynAccelDiag, counting into the strap log on both platforms with no schema change. Roughly a day including the parity twin. A generic "count these N fields for a release" is the right next step, and it is much cheaper than N migrations — it also produces exactly the evidence that would tell us which of the 23 deserve one.

   If you want to pick these up: 3 first (it unblocks #103), then 4 as the vehicle for the rest. I would leave hr_fixed_8_8 until after, since it changes shipped numbers.
2. vishk23 commented on Jul 26, 2026

   

   vishk23

   on Jul 26, 2026

   Author

   More actions

   Answers to my own questions above — three of these field names are wrong

   Ran a census + bit-level analysis over 18,650 CRC-valid v18 records (18,602 from the official-app HCI/ATT trace shared in #103 — one contiguous 5.17 h sleep block at 1 Hz — plus 48 from NOOP). Cross-tabulated every field against in-frame ground truth: HR, rr_count, band sleep_state, dynamic_acceleration@41, the step counter, and time. Every association below was checked against circular-shift controls so a base rate can't masquerade as a finding.

   Corpus limits, stated up front: it is essentially one night of sleep — sleep_state is asleep in 99.4 %, HR ≤ 55 in 81 %, and there is no ambulation, no workout and no verified off-wrist period. So the cardiac-quality fields are well exercised and the activity/wear fields barely at all.

   cardiac_flags@33 — not cardiac. A beat-detection degradation flag byte.

   • bit0 is byte-for-byte identical to @81 bit0 — 18,650/18,650 agreement, across two sessions 15 days apart on two different hosts. Contingency is purely diagonal {(0,0):17643, (1,1):1007}. It is not an independent signal.
   • bits 1–3 are never set. bits 4–5 are thermometer-coded (bit4 only ever set when bit5 is set; state 01 never occurs) — a 3-level field.
   • The high nibble's popcount is monotone against two independent quality proxies:
     • P(rr_count == 0) by popcount 0→4: .180 / .207 / .301 / .427 / .612 (base rate .219)
     • frac(@40 == 255) by popcount 0→4: .926 / .755 / .338 / .167 / .000
   • bit7 lowers @40 (median 210 vs 255) but has no effect on RR detection (P(rr0|bit7) = .217 vs base .219) — a distinct, still-uncharacterized flag.

   Suggested name: hr_quality_flags.

   cardiac_status@40 — not a status enum. A saturating 0–255 confidence score.

   216 distinct values, 73.2 % of records read exactly 255. Bit-set rates rise monotonically bit0→bit7 (86.6 % → 95.2 %) and flip-rates fall monotonically (.157 → .021) — that is the signature of a continuous quantity pinned near its maximum, and is impossible for a flags byte.

   It behaves like confidence for the HR/RR pipeline:

   @40	P(rr_count == 0)
   == 255	.173
   100–149	.467
   < 100	.799

   It degrades under motion as a top-decile threshold effect (Spearman −0.158 vs controls −0.016/+0.001/+0.009; frac255 flat at .77–.80 for deciles 0–8, then .486 in decile 9), and falls as HR rises independently of motion — within the low-motion subset (n = 13,885) Spearman = −0.360 vs shifted controls −.027/−.026/+.074; frac255 goes .950 (< 45 bpm) → .383 (60–65 bpm).

   Suggested name: hr_confidence. This one looks immediately useful — an epoch-quality gate for which HR/HRV samples to trust.

   onwrist@81 bits 0–1 — not a wear indicator.

   bit1 is never set, so it is a 1-bit field, not 2-bit. And bit0 is not a sustained wear state: there are 401 separate ON runs with a median length of 2 s (max 15 s), HR stays valid throughout (mean 55.5 bpm, only 3 zeros in 1004 records). Off-wrist would be long contiguous blocks.

   It tracks motion artifact: P(dyn_acc > p95 | bit0=1) = .3625 vs base .05 and shifted controls .033–.042 — an ~11× lift — and P(rr_count == 0 | bit0=1) = .352 vs .211.

   Suggested name: motion_artifact (and note it is mirrored at @33 bit0).

   The in-tree fixture reading onwrist == 0 on a worn frame was never a polarity puzzle: 0 is simply the normal quiet state.

   wake_quality@81 bits 2–3 — zero in all 18,650 records.

   Untestable here: this corpus is 99.4 % asleep, with zero sleep_state == still. Needs daytime/wake data before anything can be said.

   status_word / _1 / _2 — the channel-index hypothesis is confirmed, with a refinement.

   It is the low nibble, and it is a hard constant per slot: @75→0, @77→1, @79→2, never varying. The payload is the upper 12 bits. Verified bit-by-bit (@75 bits 0–3 never set; @77 bit0 always set, 1–3 never; @79 bit1 always set, 0/2/3 never) in both sessions.

   But the three slots are not redundant copies of one status word. Slots 1 and 2 are perfectly coupled and duty-cycled; slot 0 is independent and slow (three long contiguous blocks across the night — 0x550 for 2.5 h, 0x6a0 for 1.4 h, 0x490 for 1.3 h — not tied to anything observable in-frame). Only three joint states of (@77,@79) ever occur.

   Better modelled as a 3-element array of (payload:12 << 4) | channel:4.

   @82 — currently read as a dead constant; it is actually a gated live payload.

   Slots 1/2 turn out to be a scheduled periodic subsystem: 30 s every 20 minutes, phase-locked — 15/15 windows are exactly 30 records long and every one starts at unix mod 1200 == 337, zero variance. @82 is nonzero only inside that window ({(0,0):18152, (1,1):450}, no exceptions), taking 24 distinct values in 1–160.

   Full write-up and the implications for the spo2_candidate_82 promote-gate are in #103. I'm deliberately not naming a physiological quantity for it — a 20-minute cadence is consistent with several and proves none.

   motion_wear_quality / activity_class@63 — still unresolved

   Only 2 distinct values in this corpus (0, and 1 in nine records), never 2. All nine @63 == 1 groups are 3 consecutive seconds starting exactly at a step-counter (u16@57) increment — though 2 of 5 increments produced no flag. In a corpus with no ambulation, activity_class (0 still / 1 walk / 2 run) and a graded wear-quality flag are indistinguishable. The step co-occurrence leans motion/step-detection.

   The captures that would settle the rest

   1. @63 — highest value. Wear the strap, walk 5 min then run 2 min, force a history offload, capture. @63 → 1 then 2 proves activity_class; anything graded proves wear quality.
   2. @81 bits 0–3. Take the strap off for 15+ min (a sustained bit0=1 while off would overturn the artifact reading), and separately capture an offload spanning a morning wake and daytime — the only way wake_quality and sleep_state == still can ever appear.
   3. @82. A full 24 h offload, to test whether the 20-min cadence is sleep-only or continuous.
   4. @75. Several nights, to see whether the 3-block structure recurs or aligns with sleep cycles.

   Happy to send a PR renaming cardiac_flags → hr_quality_flags, cardiac_status → hr_confidence, and onwrist → motion_artifact if that's welcome — the current names actively mislead, and hr_confidence in particular looks like it should be feeding an epoch-quality gate rather than sitting unread.
3. vishk23 commented on Jul 26, 2026

   

   vishk23

   on Jul 26, 2026

   Author

   More actions

   Three decoder corrections — hr_fixed_8_8, optical_baseline_106, and the amp sentinel

   Continuing the census on the same corpus (18,650 v18 records; 18,602 from @digitalerdude's public PacketLogger capture, plus 48 from a NOOP sync — not my own strap, noted since HR there is 40–93 and the strap advertises WG50_r52). Three in-tree readings don't survive contact with the bytes.

   1. hr_fixed_8_8@36 is not a fixed-point HR. @36 is a flags byte and @37 is a duplicate HR.

   The field is decoded as u16 LE @36, "higher-precision HR: bpm = value/256". But:

   @36 bit 4 set:            0 / 18,650      (a real .5-bpm fraction sets it ~50 % of the time)
   @36 in 0x80–0x8F:         95.02 %          (a uniform byte would be 6.25 %)
   @37 == heart_rate@22:     99.51 %
   

   So u16@36 / 256 is computing (flags << 8 | hr) / 256 — it recovers the integer HR because @37 is the HR, and the "fraction" is the flag byte. That also explains the documented corr 0.989 with heart_rate@22: it is an artifact of the construction, not evidence of a second estimator.

   @37 differs from @22 in only 79/18,602 records (−6..+2), and it is not a lead/lag — 3 match the next second, 5 the previous, 71 neither — and the disagreement does not grow with motion (corr with dyn_acc = +0.066). There is no sub-bpm HR in v18.

   @36 does carry real information as flags. Bit 7 is a validity bit: when clear (n = 748), rr_count == 0 in 70.32 %; when set, 19.83 % — a 3.5× lift.

   2. optical_baseline_106@106 is not a u16. @106 and @107 are two independent u8 channels.

   consecutive pairs where @107 changed but @106 stayed frozen:  3,517 / 18,649  =  18.86 %
   u16 wrap events (low byte crossing 0xFF):                      0 / 17,789
   Δ(u16) clusters at:                                            0, ±1, ±255, ±256, ±257, ±513
   corr(@106, @107):                                              +0.729
   

   A genuine little-endian u16 cannot change its high byte while the low byte is frozen — that requires a carry, and there are zero wraps. The ±255/±256/±257 clustering is the signature of two bytes moving by ±1 independently. The same pattern holds in the 48 NOOP records at different magnitudes.

   Their meaning is unresolved: correlation with the amps is +0.001/+0.115, and there is no beat-quality gradient (rr0 flat at 14.6–22.8 % across deciles).

   3. The 128 sentinel is record-level, not per-channel — and the "amps rise with HR" note is an artifact of including it.

   amp_a == 128: 757     amp_b == 128: 757     both: 757     one alone: 0
   

   They are never independently 128, so it is one invalid-record marker, not two per-channel sentinels. And the in-tree note that the amps "rise with HR (34 → 58)" reverses once the sentinel is excluded: 32.45 → 29.52, i.e. mildly down. The apparent rise was 128s being averaged in at high HR.

   optical_amp_a/_b do look like two detectors of one measurement: r = +0.790, exactly equal in 20.3 % of records, within ±2 in 79.8 %, modal Δ = +1, near-identical ranges (10–58 vs 2–61).

   The useful part: this is a working signal-quality gate

   The sentinel fires on 4.03 % of worn seconds and predicts the band's own beat-detection failure — rr_count == 0 in 79.44 % vs 19.41 %, a 4.09× lift.

   It is not a restatement of motion. Within dyn_acc < 0.009 g (motion effectively constant, n = 184 vs 15,114) the lift is still 4.11× (78.26 % vs 19.02 %), and it holds at 3.9–4.3× in every low-motion stratum. Motion alone gives only 3.22×, so the optical flag outperforms motion as a predictor. Controls are clean: shuffled null p50 1.003 / p95 1.106 / max 1.196; circular shifts (preserving autocorrelation) 0.90–1.23; observed 4.09×.

   Rule	Flags	Precision	Recall	Lift
   amp_a == 128	4.03 %	79.44 %	14.66 %	3.64×
   @36 bit 7 clear	3.98 %	70.00 %	12.76 %	3.21×
   either	5.25 %	70.90 %	17.05 %	3.25×
   motion only (> 0.05 g)	2.50 %	66.59 %	7.61 %	3.05×

   Where the value actually is: HRV is already self-protecting — the band withholds R-R when quality is bad (≥1 RR in only 20.56 % of sentinel seconds vs 80.59 % otherwise). But HR@22 keeps being emitted. During sentinel seconds it moves 2× as much second-over-second (mean |ΔHR| 0.825 vs 0.413) on 3.4× fewer supporting beats, and it is not a held value (35.9 % unchanged vs 64.5 %). Those are low-confidence HR points that every downstream consumer currently treats as equal to the rest.

   Combined with @40 (the saturating 0–255 confidence score from my previous comment, r = −0.80 with unknown_f32_113@113 — which itself floors at −5.2869 when quality is good and takes rr_count == 0 from 18.30 % to 78.00 %), v18 carries a coherent per-second quality group: @33 flags, @36 bit 7, @40 confidence, @108/@109 + sentinel, @113 graded metric.

   No SpO2, blood-pressure or arrhythmia claim is made or supported by any of this; wavelength identity remains unknown.

   Caveats

   Single subject, single night, 5 h 10 m contiguous, 99.48 % band-scored asleep and near-motionless (median dyn_acc 0.0073 g) — so no awake/asleep contrast is testable and the activity-side fields are barely exercised. The requested per-second join against a 100 Hz IMU corpus was not possible: the only IMU corpus available is four days later and from a different subject.

   What would settle it: one overnight capture with v18 offload frames and the deep-buffer capture running over the same hours, which makes the per-second join possible and lets motion be regressed out properly; plus a deliberate perturbation (loosen the strap in timed 2-minute windows) — if @106/@107 are AGC/LED-drive registers they should step, if they are detector DC they should drift smoothly. That is the one structural question passive data cannot answer.

   Happy to send the decoder corrections as a PR.
4. ryanbr commented on Jul 27, 2026

   

   ryanbr

   on Jul 27, 2026

   Owner

   More actions

   You are right, and my answer above was wrong. I repeated the in-tree 0.989 as if it were evidence of a second estimator. It is not.

   Verified from the doc's own worked example, which turns out to refute the doc:

   25997  ->  frame[36]=141  frame[37]=101
   25997/256 = 101.5508  =  frame[37] + frame[36]/256
   

   The integer part IS @37; the "sub-bpm fraction" is the flag byte over 256. A ~0.99 correlation with heart_rate@22 is guaranteed by construction whenever @37 tracks HR — it measures nothing. And that example has @37=101 against a documented @22=102, so the doc's single illustration is itself one of your ~0.49 % disagreements. Coherent with your census, and it means the error was visible in-tree the whole time without a capture.

   So the useful conclusion is not "do not wire it up" — it is there is no sub-bpm HR in v18, which closes the question rather than parking it. BLE_REVERSE_ENGINEERING.md:431 needs correcting, and it is the load-bearing kind of wrong: it invites exactly the change I was about to weigh.

   Yes to the PR — decoder + doc corrections together. Three things I would ask for in it:

   1. Correct the doc rows, not just the decoder, and keep a line saying what the field actually is. A future reader who finds @36 named hr_fixed_8_8 in git history should land on why that was wrong.
   2. Rename in the same change. hr_fixed_8_8 and cardiac_flags are actively misleading now; optical_amp_* survives your analysis, so keep it.
   3. Keep the quality group as instrumentation. The 4.09× lift holding at 3.9–4.3× within every low-motion stratum, with clean shuffle and circular-shift controls, is a properly-run result — but one subject, one night, 99.48 % asleep. That is enough to decode, store and log; not enough to gate a score. The repo has been burned by a single-night match before (Widget (iOS): throttle the live-HR republish to 60s (#114 follow-up) #194), and your own caveats already say this.

   On the structural question: a v18 offload plus deep-buffer capture over the same hours is a reasonable ask, and the timed strap-loosening is a genuinely good experiment — step vs drift discriminates AGC from detector DC in a way passive data cannot. Worth its own issue so it does not get lost in this thread.

   Also worth crediting @digitalerdude explicitly in the PR, since the corpus is their capture.

   ❤️React with ❤️1Reacted by digitalerdude
5. ryanbr commented on Jul 27, 2026

   

   ryanbr

   on Jul 27, 2026

   Owner

   More actions

   Suggested follow-up to #861 — document what the census pinned

   #861 corrects three wrong readings. The same census pinned a lot about three fields the doc
   still calls unknown, and that is a different kind of change: nothing here is a bug fix, so it does
   not belong in #861. Worth its own PR, docs-only.

   Why bother

   docs/BLE_REVERSE_ENGINEERING.md:432 still reads:

   33 / 38 / 40 — carried raw (meaning not pinned from observation): @33 a flag-ish byte,
   @38 a u16 beside the R-R fields, @40 a status-like byte.

   and :448:

   113 — a float32 (observed range ~ −5.3…0, 0 = unset); purpose unknown, carried raw.

   That was accurate this morning. It is not any more, and the gap is the expensive kind: the next
   person either re-derives it from scratch or, worse, invents a name for a byte we now have real
   structure on. Same reasoning as keeping the struck-through hr_fixed_8_8 row in #861.

   House precedent exists — the doc already cites the ~258k records corpus twice as attribution for
   numbers nobody can re-query.

   Suggested rows

   Splitting 33 / 38 / 40 (they are no longer one story) and rewriting 113:

   | 33 | `cardiac_flags` (u8) | a beat-detection quality byte, NOT cardiac. Over 18,650 v18 records
     (attribution: #845 census, digitalerdude's public HCI capture + a second strap) **bit 0 is
     byte-identical to `@81` bit 0** — 18,650/18,650 across two sessions 15 days apart on different
     hosts, so it is not an independent signal. Bits 1–3 are never set; bits 4–5 are thermometer-coded
     (bit 4 only ever set with bit 5, state `01` never occurs) — a 3-level field. High-nibble popcount
     is monotone against `P(rr_count == 0)`: .180 / .207 / .301 / .427 / .612 for popcount 0→4 against
     a .219 base rate. Name retained pending a better one; "cardiac" is positional, not derived. |
   
   | 38 | `rr_packed` (u16) | a u16 beside the R-R fields; meaning still not pinned. |
   
   | 40 | `cardiac_status` (u8) | a saturating 0–255 quality score, correlated **r = −0.80** with
     `@113` (#845 census). `whoop-local` names this `signal_quality` in code with no stated source or
     analysis; the census supports something quality-shaped, but the name here stays positional until
     someone pins the scale. |
   
   | 113 | `unknown_f32_113` (f32 LE) | a graded quality metric, not unknown any more: it **floors at
     −5.2869** when signal quality is good, and takes `P(rr_count == 0)` from 18.30 % to **78.00 %**
     across its range (#845 census). `0` = unset. Correlated r = −0.80 with `@40`. No physiological
     reading is asserted. |
   

   The caveat that has to travel with them

   vishk23 stated it themselves and it belongs in the doc, not just the issue thread: the corpus is
   one subject, one night, 5 h 10 m contiguous, 99.48 % band-scored asleep, median dyn_acc
   0.0073 g, with no ambulation, no workout and no verified off-wrist period.

   So the cardiac-quality fields are well exercised and the activity-side ones barely at all. A row
   that omits this invites someone to treat "monotone against rr_count == 0" as established across
   wake, exercise and off-wrist, which the data cannot support.

   Suggest a single line under the table rather than repeating it per row.

   What this deliberately does not do

   • No renames. cardiac_flags and cardiac_status are positional names for bytes that sit near
     the HR fields. The census says what they do, not what they are. Renaming to beat_quality_*
     would assert an interpretation one night cannot carry — and fix(protocol): v18 @36 is a flag byte, @106/@107 two u8 channels #861 has just spent its length
     showing what a premature name costs.
   • No consumption. Nothing reads these. The signal-quality group is instrumentation until a
     second corpus exists, per the Widget (iOS): throttle the live-HR republish to 60s (#114 follow-up) #194 rule.
   • No SpO₂/BP claim, and the wavelength identity stays unknown.

   Who should write it

   vishk23. The numbers are theirs and they should stand behind them; I can verify the arithmetic and
   the internal consistency but I cannot source the corpus. Happy to land it myself if they would
   rather not — the structure above is the whole PR.
6. ryanbr

   added 2 commits that reference this issue on Jul 27, 2026

   docs: use the census's own word for @40 — 'confidence score', not 'qu…

   ...

   759241e

   docs(v18): write down what the census pinned about the quality group (#…

   ...

   da92d4e
7. ryanbr commented on Jul 27, 2026

   

   ryanbr

   on Jul 27, 2026

   Owner

   More actions

   Both landed: #861 (6a60c53a) corrects the three wrong readings, #865 (da92d4e4) documents what the census pinned about @33 / @40 / @113.

   @vishk23 — the doc rows are attributed to your census rather than presented as this repo's measurement, since the corpus is @digitalerdude's capture plus your second strap and is not in-tree. Every figure was diffed against your comments and matches verbatim; the only derived number is the 4.3× lift, arithmetic on the two values beside it. Shout if any of it reads as overstated — it is your evidence and you should be comfortable with how it is worded.

   Names stayed positional deliberately. cardiac_flags and cardiac_status say where the bytes sit, not what they are, and #861 is a good argument for that restraint.

   Still open from this issue:

   • spo2_candidate_82 instrumentation — the item I called the strongest here. The evidence is split (8-night validation at corr +0.99 against two nights on the original device moving opposite), and only the offline tooling can currently produce the multi-device data that would settle it. Persisting the in-band value as instrumentation is the unblock, and nothing in fix(protocol): v18 @36 is a flag byte, @106/@107 two u8 channels #861 or docs(v18): write down what the census pinned about the quality group #865 touched it.
   • hr_fixed_8_8, now retired, still needs its slot fixed in feat(protocol): bank the v18 per-second fields the storage funnel was discarding #848 — the lookup there resolves a key that no longer exists, and because it is an optional dictionary read it banks nothing silently. Commented on that PR.
   • The generic field-counter diagnostic (framing question 4) — still the cheapest way to get evidence on the remaining unpinned bytes for roughly the cost of one migration.
8. ryanbr

   added a commit that references this issue on Jul 27, 2026

   tools(spo2): account for @82's duty cycle, screen for variance, and c…

   ...

   b8570f9
9. tanarchytan

   added a commit that references this issue on Jul 27, 2026

   docs(releases): 9.0.2 notes, with upstream attribution

   ...

   8c95f73
10. vishk23 commented on Jul 28, 2026

    

    vishk23

    on Jul 28, 2026

    Author

    More actions

    The second number in the title is now wrong, and it is wrong in the best possible way: nothing in the v18 path is dropped any more. #848 landed the other end of the pipe this issue asked about. Leaving the headline as-is invites someone to re-solve a solved problem, so here is the restatement with what I re-checked against main today.

    The 33 is correct and unchanged. It is the number of fb.add call sites in the v18 branch of decodeWhoop5Historical — Interpreter.swift:350–601 — and I counted 33 there again just now. (Counting distinct named channels instead gives 36, because three sites are loops: gravity_x/y/z, optical_baseline_a/b, and rr[i] collapsing to one rr channel. Same decoder, different convention; this issue picked call sites and should keep them.)

    "23 have no downstream consumer" has become "banked, awaiting one". #848 ("bank the v18 per-second fields the storage funnel was discarding") is merged. It added the v18AuxSample table plus four named columns — gravitySample.dynAccel, sleepStateSample.rawByte, skinTempSample.aux1Raw/aux2Raw — so the fields that previously existed only as decoder output now persist.

    Why that distinction is the whole point. This issue's own framing was "the strap trims its history once NOOP acks the offload, so those values are gone permanently." That was the real cost — not that nothing read them, but that nothing could, ever, retrospectively. #848 converts permanent loss into retained-but-unread. You cannot validate what you did not keep; you can always validate later what you kept.

    They are still unread, and that is deliberate. I checked every v18AuxSamples call site on both platforms: on Swift they are all in DeepCaptureChannelsTests and TimestampHealTests; on Kotlin the reader exists in WhoopDao/WhoopRepository with no caller at all. No analytic, no score, no gate, no UI, no export touches a row. That matches #848's own body ("Instrumentation only") and the repo's standing rule in CLAUDE.md — land unvalidated sensor work as instrumentation rather than score it. Several of these slots are bytes whose names deliberately assert nothing (status_word_1, aux_byte_82, cardiac_status); wiring them to anything before a census would be exactly the overclaiming this project has already had to retract more than once.

    I have pushed a small follow-up that adds the same "do NOT clean up the reader as dead code" note the ppgWaveformSample migration already carries, on both platforms — without it a future tidy-up pass reading the migration cold would delete the reader, and with it the only route back to the rows.

    Some names changed without changing the count, worth noting so the table above doesn't read as stale:

    • hr_fixed_8_8@36 split into hr_quality_flags@36 + heart_rate_alt@37 once the census disproved the sub-bpm-HR reading. The current note is honest: "duplicate of heart_rate@22 (99.6% exact); trust only when hr_quality_flags bit7 is set."
    • optical_baseline_106 became optical_baseline_a/b.
    • sleep_state_byte@81 was added by feat(protocol): bank the v18 per-second fields the storage funnel was discarding #848.

    And several rows in the table are recoverable rather than lost, which the table doesn't show: hist_version is the v18 dispatch gate; motion_wear_quality is the same byte as the persisted activity_class; onwrist and wake_quality are bits 0–1 and 2–3 of the persisted sleep_state_byte; spo2_candidate_82 is the gated 70–100 view of the raw aux_byte_82. A test enforces that no slot duplicates an already-persisted field (DeepCaptureChannelsTests.testNoSlotDuplicatesAnAlreadyPersistedField).

    Suggested retitle: "v18: 33 decoded fields — inventory, and which of the banked-but-unread ones deserve a consumer", or close it as done and open a narrower issue per candidate signal. The sleep_state precedent in the original post is still the right model: one signal, one analytic, measured against something. That is a per-signal question now, not an inventory question — the inventory part is finished.

    (I don't have write access to retitle; flagging rather than doing.)
11. vishk23 commented on Jul 29, 2026

    

    vishk23

    on Jul 29, 2026

    Author

    More actions

    Opened: #942. Sorry it took two asks — you were right that it was the strongest item here, and right to keep raising it.

    The ask split in two, and one half had already landed

    Working it, I found the thing worth reporting back: #848 merged 89 minutes after your "still open" note (2026-07-27T05:50:10Z vs 04:21:10Z) and banks the raw @82 into v18AuxSample. So the persistence half you asked for — "persisting the in-band value as instrumentation" — is done, and has been since that morning. Your note was accurate when you wrote it; it stopped being so almost immediately, and nobody said.

    The half that was actually still blocking was the read-back, and the tool says so itself:

    NOTE: this is the CAPTURE TOOLING's database, not the app's. Neither shipped app has a frames table — Android uses Room, iOS/macOS uses GRDB — so this cannot be pointed at a phone's store or a .noopbak.

    So a NOOP user who synced a 5/MG was already holding the evidence #103 waits on, with no way to run the gate over it — contributing still required linux-capture. #942 teaches the harness to read the app's own store. Details and limits are in the PR; the one that matters is that the offset-specificity scan cannot run from an app store (the app banks decoded slots, not frame bytes), so it is reported n/a rather than as a lost scan, and a capture is still required to settle specificity.

    Independent corroboration on @82 — with two limits on how far it goes

    tanarchytan/whoop-rs (HEAD 312908c) reads SpO2 from the same byte. crates/whoop-protocol/src/records/gen5.rs:31:

    spo2_pct: u8_at(b, 74).filter(|&v| (70..=100).contains(&v)),

    Inner offset 74 = frame-absolute 82 on Gen5 — our spo2_candidate_82, with the same 70–100 gate, medianed per night (crates/whoop-store/src/lib.rs:230-246). Their docs state in three places that this is the strap's own computed value riding v18 and is not derived (records/mod.rs:73-74, physio-algo/src/spo2.rs:1-3, docs/algorithms.md:311).

    Two things that stops short of, stated precisely because the difference matters:

    1. It is not a fully independent observer. docs/algorithms.md:3 describes itself as the source of truth for algorithms NOOP ships, and records/mod.rs:72 credits the v20 channel roles to "upstream RE" — so there is shared ancestry. What is independent is that a second team, working their own captures, landed on the same byte and shipped a plausibility filter around it. That is worth something; it is not two independent derivations.
    2. No captured frame in their repo carries a valid SpO2 percent. Their three v18 real-frame fixtures all read inner[74] = 0. In fairness all three are daytime captures (10:55, 09:21, 19:35 UTC) and the byte is documented sleep-only, so zero is consistent with the claim rather than a refutation — and it matches the duty-cycle finding, where @82 is identically 0x00 outside a short phase-locked window. Their README cites an overnight drain that "located the sleep SpO2 (363 readings, 95–100%)", but that capture is not in the repo. The honest label is claimed, unverified in-artifact.

    So: two implementations read this byte, neither has published a capture proving it carries real values on a worn strap overnight, and the instrumentation you asked for is what would settle it. That is a better motivation than the one this issue currently carries, and it argues for the gate rather than around it.

    (For completeness — my own earlier "@82 = 0" null was publicly retracted as underpowered and should not be read as evidence in either direction.)

    Item 4 is not done

    The generic N-field diagnostic counter following the DynAccelDiag/#813 pattern is still open. You put item 3 first because it unblocks #103; that is what #942 is. Item 4 is queued, not forgotten — it is the right vehicle for the remaining unpinned bytes and, as you said, much cheaper than N migrations.
12. ryanbr

    added a commit that references this issue on Jul 29, 2026

    tools(spo2): let the promote gate read a NOOP app store, not just a c…

    ...

    8fd42c7
13. ryanbr commented on Aug 2, 2026

    

    ryanbr

    on Aug 2, 2026

    Owner

    More actions

    Closing this — everything it asked for has landed, and the headline is now false in a way that invites someone to re-solve a solved problem.

    "23 of 33 have no downstream consumer" stopped being true when #848 landed. Your own restatement on 28 July had it right: the fields are banked, not dropped. v18AuxSample plus the four named columns exist, and Reads.swift selects aux1Raw, aux2Raw and dynAccel back out. The cost this issue actually named — "the strap trims its history once NOOP acks the offload, so those values are gone permanently" — no longer applies.

    Where each ask ended up:

    ask	outcome
    What is known about the unclear fields	answered in thread, written down in #865
    hr_fixed_8_8 "wiring gap"	#861 — not a gap: there is no sub-bpm HR in v18, and the 0.989 that justified the name was circular
    temp_aux_1/2, dynamic_acceleration, raw sleep_state	#848 — banked
    spo2_candidate_82 blocking #103	#942
    The framing question (log-count first?)	answered by #848 — banked rather than counted

    The one loose end from my last comment is closed too. I flagged that hr_fixed_8_8 was retired but #848 might still resolve the key, banking nothing silently. It does not: there is a dedicated regression test on both platforms (retired: Set<String> = ["hr_fixed_8_8", "optical_baseline_106"] in Swift, the same set in Kotlin) asserting nothing reads them, plus XCTAssertNil / assertNull in the decode tests and a struck-through row in the doc explaining why the name was wrong. Nothing in the shipping path touches either key.

    Still live, and deliberately separate:

    • Capture wanted: paired v18 + deep-buffer overnight, and a timed strap-loosening run, to settle the v18 optical/quality fields #858 — the paired v18 + deep-buffer capture, and the timed strap-loosening run. That is where the remaining unknowns (optical_baseline_106/@107, onwrist@81 bit 0, the wake/daytime gap) get settled, and it needs a second subject rather than more analysis.
    • WHOOP 5.0 raw HCI capture — where to submit? (docs reference #174, which doesn't exist) #103 — where to submit HCI captures.

    Worth saying plainly what this issue was: it asked the right question about the other end of the pipe, and the census that followed corrected three in-tree readings that had been wrong for months — including one where the doc's own worked example refuted the doc. Reopen if I have miscounted something.
14. ryanbr

    closed this as completedon Aug 2, 2026
15. ryanbr

    added 3 commits that reference this issue on Aug 5, 2026

    Android live workout: glanceable vertical-stack parity

    ...

    123df10

    Android live workout: glanceable vertical-stack parity

    ...

    30da36f

    Android live workout: glanceable vertical-stack parity

    ...

    e01bd2e
16. simoncad7

    added 3 commits that reference this issue on Aug 17, 2026

    docs(v18): write down what the census pinned about the quality group (r…

    ...

    933a0b4

    tools(spo2): account for @82's duty cycle, screen for variance, and c…

    ...

    35c0874

    tools(spo2): let the promote gate read a NOOP app store, not just a c…

    ...

    558fe30