From c4ed0344f1bfa10a9a9f223b7b03f49cd7b997bb Mon Sep 17 00:00:00 2001 From: ryanbr Date: Sun, 26 Jul 2026 21:15:22 -0700 Subject: [PATCH 1/2] docs(v18): write down what the census pinned about the quality group MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit #861 corrected three WRONG readings. The same census pinned a lot about three fields the doc still called unknown, which is a different kind of change and did not belong there. Until now `@33 / @38 / @40` shared one row saying "carried raw (meaning not pinned from observation)" and `@113` said "purpose unknown". Both were accurate this morning. The cost of leaving them is the expensive kind: the next person re-derives it from scratch, or invents a name for a byte we now have real structure on — which is what #861 spent its whole length undoing. Splits the shared row, because they are no longer one story: @33 bit 0 is byte-identical to @81 bit 0 in 18,650/18,650 across two sessions on different hosts, so it is NOT an independent signal. Bits 1-3 never set, bits 4-5 thermometer-coded. High-nibble popcount monotone against P(rr_count == 0): .180/.207/.301/.427/.612 against a .219 base rate. @38 unchanged — still not pinned. @40 a saturating 0-255 quality score, r = -0.80 with @113. @113 floors at -5.2869 when quality is good; takes P(rr_count == 0) from 18.30% to 78.00%. Not unknown any more. Names stay POSITIONAL. cardiac_flags and cardiac_status describe where the bytes sit, not what they are, and the census says what they do rather than what they measure. Renaming to beat_quality_* would assert an interpretation one night cannot carry. The corpus caveat now travels with the table instead of living in an issue thread: single subject, one night, 5 h 10 m, 99.48% asleep, no ambulation, no workout, no verified off-wrist period. The cardiac-quality fields are well exercised; the activity-side behaviour is not. Without that line someone reads "monotone against rr_count == 0" as established across wake and exercise, which the data cannot support. Attributed to the #845 census rather than presented as this repo's own measurement — the corpus is @digitalerdude's public capture plus a second strap, and is not in-tree. Consistent with how the ~258k-record figures are already cited here. Docs only. Nothing consumes these fields; they stay instrumentation per the #194 rule. --- docs/BLE_REVERSE_ENGINEERING.md | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/docs/BLE_REVERSE_ENGINEERING.md b/docs/BLE_REVERSE_ENGINEERING.md index ddd6cea1cb..56a2d01f03 100644 --- a/docs/BLE_REVERSE_ENGINEERING.md +++ b/docs/BLE_REVERSE_ENGINEERING.md @@ -431,7 +431,9 @@ garbage (HR `0`, gravity overflow). The fields below were read off real frames a | 36 | `hr_quality_flags` (u8) | a **flag byte**, *not* the low half of a fixed-point HR. Over **18,650** real v18 records bit 4 is **never** set (0/18,650 — a genuine 8.8 fraction sets it ~50 % of the time, and it is the only bit never set), **95.02 %** of values land in `0x80`–`0x8F` (uniform would be 6.25 %) across just **40 distinct values**, and sd = **26.5** vs 73.9 for a uniform byte. **Bit 7 = validity**: with it clear (n=748) `rr_count == 0` in **70.32 %** of records vs **19.82 %** with it set, and the `@108/@109` sentinel fires in **69.65 %** vs 1.32 %. Remaining bits unpinned; carried raw. | | 37 | `heart_rate_alt` (u8, bpm) | a **duplicate** of `heart_rate@22` — equal in **99.575 %** of records (18,523/18,602), differing only by −6…+2, and it tracks HR only while `@36` bit 7 is set (99.74 % exact vs 94.12 % when clear). | | ~~36–37~~ | ~~`hr_fixed_8_8` (u16 LE) — bpm = `value/256`~~ | **Retired.** The "corr 0.989 with `heart_rate@22`" that justified this name was **circular**: the u16 is literally `hr@22` (at `@37`) plus the `@36` flag byte over 256, so the residual is a flat **+0.504 ± 0.189** — i.e. `@36/256`, not a sub-bpm fraction. On records where `@36` bit 7 is clear it produced absurd readings (a fixture decodes to **227 bpm**). | -| 33 / 38 / 40 | raw bytes near the HR / R-R fields | carried **raw** (meaning not pinned from observation): `@33` a flag-ish byte, `@38` a u16 beside the R-R fields, `@40` a status-like byte. `whoop-local` names `@40` **`signal_quality`** (#715) — plausible for a byte sitting beside HR, but it appears only in that project's code with no stated source and no supporting analysis, so it stays raw here. | +| 33 | `cardiac_flags` (u8) | a **beat-detection quality byte**, not cardiac. Over **18,650** v18 records (#845 census — @digitalerdude's public HCI capture plus a second strap): **bit 0 is byte-identical to `@81` bit 0** in 18,650/18,650, across two sessions 15 days apart on different hosts, so it is **not an independent signal**. Bits 1–3 are never set; bits 4–5 are **thermometer-coded** (bit 4 only ever set with bit 5; state `01` never occurs) — a 3-level field. High-nibble popcount is monotone against `P(rr_count == 0)`: **.180 / .207 / .301 / .427 / .612** for popcount 0→4, against a **.219** base rate. The name is POSITIONAL (it sits near the HR fields), not derived — the census says what the byte does, not what it is. | +| 38 | `rr_packed` (u16) | a u16 beside the R-R fields; meaning still **not pinned**. | +| 40 | `cardiac_status` (u8) | a **saturating 0–255 quality score**, correlated **r = −0.80** with `@113` (#845 census). `whoop-local` names this `signal_quality` in its own code with no stated source or supporting analysis; the census independently supports something quality-shaped, but the name here stays positional until someone pins the scale. | | 41 | `dynamic_acceleration` (f32, g) | the strap's own **gravity-removed motion magnitude**, one scalar per second sitting immediately before the gravity triplet. Gated to `[0, 8] g` so a wrong offset stores nothing rather than garbage; reads 0.006–0.033 g across the resting oracle frames. Decoded on both platforms but **not persisted and not scored** — `step_motion_counter@57` and `activity_class@63` are what the motion paths actually consume. See the byte-43 note below. | | 45 / 49 / 53 | `gravity_x/y/z` (f32, g) | \|g\| ≈ 1.0 for 100 % of 500 records; v18 has **one** triplet (not v24's two) | | 57–58 | `step_motion_counter` (u16 LE @[57:59]) | a **cumulative** counter: climbs while moving, flat when still, low byte wraps at 256. **Steps = Σ wrap-aware diffs** `(cur-prev)&0xFFFF` — *not* the value summed per record (that over-counts massively — the WHOOP 5/MG step over-report). No per-record step count is in the record. | @@ -449,7 +451,9 @@ garbage (HR `0`, gravity overflow). The fields below were read off real frames a | 104 | (const) | observed **constant `1`** on two straps; carried raw, no metric. | | 106 / 107 | `optical_baseline_a` / `optical_baseline_b` (u8, u8) | two **independent u8** optical/ADC baseline channels — **not** one u16 LE. A u16 is structurally impossible here: across 18,599 consecutive-second pairs the **high byte changed while the low byte stayed frozen in 3,514 (18.89 %)**, and the corpus holds **zero** low-byte wrap events — a real u16 cannot step its high byte without a carry. The apparent u16 deltas are exactly `256·Δ@107 + Δ@106` (clustering at 0, ±1, ±255, ±256, ±257, ±513). Correlated but independent (corr **+0.73**; they move in **opposite** directions in 5.8 % of pairs where both move). **`0` — not `128` — marks off-wrist**: both bytes read 0 in exactly the 8 records that also carry `HR == 0`, while 128 occurs unremarkably while worn (`@106` in 10 records, `@107` in 103). Magnitudes are device-specific (102–255 / 119–247 on one strap vs 20–66 / 34–81 on another), so **no scale is asserted**. | | 108 / 109 | `optical_amp_a` / `optical_amp_b` (u8, u8) | a tightly-coupled **pair** (equal in 23.5 % of records, within ±2 in ~80 %). **`128` is a RECORD-level sentinel**, not per-channel: `amp_a == 128` in 757 records and `amp_b == 128` in 757 — the **same** 757, never one without the other. They do **not** rise with heart rate; that reading (~34 at HR 40–49 → ~58 at 80–89) was an **averaging artifact** of counting the 128 sentinel as a number — with sentinels excluded the trend is flat-to-declining (**32.45 → 29.52**). The real monotone trend is with **motion**: ~32.7 while still (`dyn_acc` < 0.02 g) → **37.4** at 0.05–0.2 g. The sentinel is a usable per-second **signal-quality** flag: it fires on 4.02 % of worn seconds and predicts the band's own beat-detection failure (`rr_count == 0`) at **79.44 % vs 19.40 %** — a **4.09×** lift that **survives holding motion constant** (4.11× within `dyn_acc` < 0.009 g), where shuffled and circular-shift nulls all sit at ~1.0×. **Not** SpO₂, blood pressure or a perfusion substrate — signal-quality/AGC is the supported reading and the wavelength identity is unknown. | -| 113 | `unknown_f32_113` (f32 LE) | a float32 (observed range ~ −5.3…0, `0` = unset); **purpose unknown**, carried raw. | +| 113 | `unknown_f32_113` (f32 LE) | a **graded quality metric**, no longer unknown: it **floors at −5.2869** when signal quality is good, and across its range takes `P(rr_count == 0)` from **18.30 % to 78.00 %** — a 4.3× lift (#845 census). `0` = unset. Correlated **r = −0.80** with `@40`, so the two report the same condition on different scales. Carried raw; no physiological reading is asserted. | + +**Corpus caveat for the `@33` / `@40` / `@108`–`@109` / `@113` quality group.** Those figures come from the #845 census over one contiguous capture: **a single subject, one night, 5 h 10 m, 99.48 % band-scored asleep**, median `dyn_acc` 0.0073 g, with **no ambulation, no workout and no verified off-wrist period**. The cardiac-quality fields are therefore well exercised and the activity-side behaviour is barely exercised at all. Read the monotone relationships as established *for still, asleep wear* — not across wake, exercise or off-wrist, which this corpus cannot speak to. A second corpus is what would promote any of this beyond instrumentation. The strongest check on the HR offset: where a historical record and a live `REALTIME_DATA` (§5, 2A37 ground-truth-verified) frame share a timestamp, the historical HR equalled the live HR at **96/96** From 759241e7afd204b6a7e9b726b1e5ea032227ad5a Mon Sep 17 00:00:00 2001 From: ryanbr Date: Sun, 26 Jul 2026 21:18:23 -0700 Subject: [PATCH 2/2] =?UTF-8?q?docs:=20use=20the=20census's=20own=20word?= =?UTF-8?q?=20for=20@40=20=E2=80=94=20'confidence=20score',=20not=20'quali?= =?UTF-8?q?ty=20score'?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A paraphrase, not a misstatement, but the row is explicitly attributed to the #845 census and #845 says confidence score. When a line carries someone else's attribution it should carry their term; a reader comparing the doc against the source should not have to decide whether a different word means a different claim. Checked the rest of the rows against the originals while I was there — 18,650/18,650 for the @81 bit-0 match, the thermometer-coded bits 4-5, the .180/.207/.301/.427/.612 popcount ladder against a .219 base rate, r = -0.80, the -5.2869 floor, and 18.30% -> 78.00% all match verbatim. The only derived figure is the 4.3x lift, which is arithmetic on the two numbers printed beside it. --- docs/BLE_REVERSE_ENGINEERING.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/BLE_REVERSE_ENGINEERING.md b/docs/BLE_REVERSE_ENGINEERING.md index 56a2d01f03..18007560be 100644 --- a/docs/BLE_REVERSE_ENGINEERING.md +++ b/docs/BLE_REVERSE_ENGINEERING.md @@ -433,7 +433,7 @@ garbage (HR `0`, gravity overflow). The fields below were read off real frames a | ~~36–37~~ | ~~`hr_fixed_8_8` (u16 LE) — bpm = `value/256`~~ | **Retired.** The "corr 0.989 with `heart_rate@22`" that justified this name was **circular**: the u16 is literally `hr@22` (at `@37`) plus the `@36` flag byte over 256, so the residual is a flat **+0.504 ± 0.189** — i.e. `@36/256`, not a sub-bpm fraction. On records where `@36` bit 7 is clear it produced absurd readings (a fixture decodes to **227 bpm**). | | 33 | `cardiac_flags` (u8) | a **beat-detection quality byte**, not cardiac. Over **18,650** v18 records (#845 census — @digitalerdude's public HCI capture plus a second strap): **bit 0 is byte-identical to `@81` bit 0** in 18,650/18,650, across two sessions 15 days apart on different hosts, so it is **not an independent signal**. Bits 1–3 are never set; bits 4–5 are **thermometer-coded** (bit 4 only ever set with bit 5; state `01` never occurs) — a 3-level field. High-nibble popcount is monotone against `P(rr_count == 0)`: **.180 / .207 / .301 / .427 / .612** for popcount 0→4, against a **.219** base rate. The name is POSITIONAL (it sits near the HR fields), not derived — the census says what the byte does, not what it is. | | 38 | `rr_packed` (u16) | a u16 beside the R-R fields; meaning still **not pinned**. | -| 40 | `cardiac_status` (u8) | a **saturating 0–255 quality score**, correlated **r = −0.80** with `@113` (#845 census). `whoop-local` names this `signal_quality` in its own code with no stated source or supporting analysis; the census independently supports something quality-shaped, but the name here stays positional until someone pins the scale. | +| 40 | `cardiac_status` (u8) | a **saturating 0–255 confidence score**, correlated **r = −0.80** with `@113` (#845 census). `whoop-local` names this `signal_quality` in its own code with no stated source or supporting analysis; the census independently supports something quality-shaped, but the name here stays positional until someone pins the scale. | | 41 | `dynamic_acceleration` (f32, g) | the strap's own **gravity-removed motion magnitude**, one scalar per second sitting immediately before the gravity triplet. Gated to `[0, 8] g` so a wrong offset stores nothing rather than garbage; reads 0.006–0.033 g across the resting oracle frames. Decoded on both platforms but **not persisted and not scored** — `step_motion_counter@57` and `activity_class@63` are what the motion paths actually consume. See the byte-43 note below. | | 45 / 49 / 53 | `gravity_x/y/z` (f32, g) | \|g\| ≈ 1.0 for 100 % of 500 records; v18 has **one** triplet (not v24's two) | | 57–58 | `step_motion_counter` (u16 LE @[57:59]) | a **cumulative** counter: climbs while moving, flat when still, low byte wraps at 256. **Steps = Σ wrap-aware diffs** `(cur-prev)&0xFFFF` — *not* the value summed per record (that over-counts massively — the WHOOP 5/MG step over-report). No per-record step count is in the record. |