From dc9243c27eb6d84244ca1709122df0e6dc480c36 Mon Sep 17 00:00:00 2001 From: Artur Iskakov Date: Fri, 25 Sep 2026 12:59:45 +0200 Subject: [PATCH 1/2] fix(ble): bank 5/MG IMU buffers into Raw Data Collector sessions W06-002. A Raw Data Collector session on a WHOOP 5/MG never recorded an IMU sample. The only writer, ImuSessionFileStore.append, sits in Collector.ingest(frame:parsed:), and nothing in the app calls that: #1709 moved IMU storage behind the ingest seam, which the 5/MG frame loop does not enter. The session started the strap's 100 Hz stream, then exported nothing. The 5/MG loop now hands every frame to Collector.bankImuForSessions before the offload branch, so a session takes buffers from the live stream and from history sync, as #1709 intended. Only intact 1244-byte buffers are banked, because Whoop5RawImu decodes without checking the CRC. V1: CollectorImuBankingTests (an intact real buffer is banked, a CRC-flipped copy is not); Strand and NOOPiOS (without the watch app) build. Not yet tested on hardware. --- Strand/BLE/BLEManager.swift | 5 +++- Strand/Collect/Collector.swift | 13 ++++++++ StrandTests/CollectorImuBankingTests.swift | 35 ++++++++++++++++++++++ docs/review/workstreams/W06-ble-collect.md | 2 +- 4 files changed, 53 insertions(+), 2 deletions(-) create mode 100644 StrandTests/CollectorImuBankingTests.swift diff --git a/Strand/BLE/BLEManager.swift b/Strand/BLE/BLEManager.swift index 1c0cf1524d..4f6eae07b3 100644 --- a/Strand/BLE/BLEManager.swift +++ b/Strand/BLE/BLEManager.swift @@ -7047,7 +7047,10 @@ extension BLEManager: @preconcurrency CBPeripheralDelegate { // BEFORE the offload branch so it catches the burst; no-op unless capture is on. puffinDeepBufferLog.appendIfDeepBuffer(frame: frame, char: characteristic.uuid, isOffload: isOffload) stopUnexpectedRealtimeImu(frame, isOffload: isOffload) - // #423: the queryable twin of that diagnostics line — persist the decoded 100 Hz 6-axis + // #423 / #1709: the queryable twin of that diagnostics line — bank the decoded 100 Hz + // 6-axis buffer into any open Raw Data Collector session, from the live stream and from + // history sync alike. BEFORE the offload branch, which `continue`s (W06-002). + collector?.bankImuForSessions(frame) if isOffload { // Same policy as WHOOP4: historical offload frames are bulk sync traffic. // Keep them out of the live UI parser during backfill and let Backfiller diff --git a/Strand/Collect/Collector.swift b/Strand/Collect/Collector.swift index a08578ce82..efbaf79359 100644 --- a/Strand/Collect/Collector.swift +++ b/Strand/Collect/Collector.swift @@ -366,6 +366,19 @@ final class Collector { rawCapture.open(at: monotonic(), duration: seconds) } + /// Bank one 5/MG 100 Hz IMU buffer into any open Raw Data Collector session (#1709). The 5/MG frame + /// loop calls this for every frame, live and from history sync, because that loop never calls + /// `ingest(frame:parsed:)`, where the only writer sat, so sessions recorded nothing (W06-002). + func bankImuForSessions(_ frame: [UInt8]) { + guard Self.isBankableImu(frame) else { return } + recordGroundTruthImu(frame) + } + + /// An intact 1244-byte IMU buffer. `Whoop5RawImu` decodes without checking the CRC, so the gate does. + nonisolated static func isBankableImu(_ frame: [UInt8]) -> Bool { + frame.count == Whoop5RawImu.bufferLength && verifyFrame(frame, family: .whoop5).ok + } + private func recordGroundTruthImu(_ frame: [UInt8]) { _ = ImuSessionFileStore.shared.append(deviceId: deviceId, frame: frame, receivedAtMs: Int64(Date().timeIntervalSince1970 * 1_000)) diff --git a/StrandTests/CollectorImuBankingTests.swift b/StrandTests/CollectorImuBankingTests.swift new file mode 100644 index 0000000000..73c173a0e0 --- /dev/null +++ b/StrandTests/CollectorImuBankingTests.swift @@ -0,0 +1,35 @@ +import XCTest +import WhoopProtocol +@testable import Strand + +/// W06-002: the 5/MG frame loop banks IMU buffers into Raw Data Collector sessions through +/// `Collector.bankImuForSessions`. The gate admits intact 1244-byte buffers only, because `Whoop5RawImu` +/// decodes without checking the CRC. +final class CollectorImuBankingTests: XCTestCase { + + /// A real 1244-byte buffer, the fixture `Whoop5RawImuTests` decodes. + private let realBuffer: [UInt8] = { + let hex = "aa01d40401005c702f1580e520af002d3f566a002004640064000300d308cc08c408c908c208cd08b708c208cc08c908cc08e108d608e408d208c508bd08d708c508d208cc08c908bc08b508a908c008cc08cb08d308e108dc08d408d108c108c208c208b708ce08c608e008c308d208bd08c908bb08b708be08c208cc08ce08c608cf08c408c808ca08cb08cf08cb08d508c708c908cc08bf08c308c408cc08cc08cc08c508c708d108c108c608c808c108c408c308c608cf08c808c608cf08ce08c808d008c008d008c608bb08cb08d208c008cb08c708c008c508c208c608cf08d008d3ffcaffc1ffc8ffcaffc5ffccffd4ffd3ffdeffddffd8ffdbffd6ffc7ffc7ffd0ffd5ffd6ffe2ffd4ffceffd6ffd2ffe2ffdbffdbffc7ffc6ffc9ffc1ffb1ffb7ffb7ffc9ffceffe4ffe7ffe4ffeaffe7ffdbffd7ffe0ffd7ffc4ffccffcdffbbffc2ffbeffb9ffccffd4ffd4ffc6ffcaffd3ffc8ffd6ffceffd7ffdaffdfffddffdaffd6ffddffdaffd3ffe1ffd0ffc9ffcdffd1ffcaffd3ffcfffd3ffd6ffcfffcaffc9ffc7ffcaffd7ffd5ffd0ffdaffd4ffddffd6ffd8ffdcffd8ffd4ffcaffe5ffceffccff690d840d810d760d7f0d7a0d730d7a0d800d8a0d820d8c0d800d750d740d740d620d690d800d7a0d7d0d6e0d710d780d790d890d770d810d7d0d760d7d0d7b0d7a0d890d810d830d7a0d6d0d6c0d6f0d690d6d0d790d6d0d730d760d770d850d790d810d760d7d0d750d720d760d740d720d820d750d890d840d830d7d0d7b0d770d7b0d820d6f0d830d6f0d770d6e0d7b0d820d700d760d7f0d6a0d780d790d7c0d830d780d7a0d840d780d6f0d7f0d740d800d7b0d860d7f0d7a0d840d7d0d820d770d810d7c0d6400640005020000000000000900090004000500060007000a000c000b000d000e000c000d000d000c000b00090008000c000d000d0011000e000c000c000b000b000e000f00130011001100100010000e000e000e000e000c000b000c000c000b000e000d0010000e000e000d000d000c000b000c000a000b000c000c000e000b000b000c000c000d000a000b000b000a000b000c000c000c000d000e000f000b000d000b000b000e000d000c000c000b000b000c000c000c000b000b000a000a000b000b000d000f000d000d000b000b000a00fcfffbfffbfffdfff9fffbfffcfffdfffdfffdfffcfffcfffffffcfffcfffdfffffffdfffcffffff01000000fefffdfffdfffefffeffffff0000ffff0200feffffffffff000000000100fefffffffdfffdfffefffdfffefffbfffdffffff0100fefffefffdfffdfffdfffefffdfffffffefffeff0000fefffdfffffffefffdff0000fefffefffefffdfffefffefffefffefffffffffffffffffffdff00000000fefffdfffffffefffdfffefffdfffdfffffffffffdfffefffffffcfffdfffefffdfffdfffefffdfffbfff9fff9fff8fffcfff9fff8fff9fff7fff7fffafffcfffbfffdfffbfffafffcfffcfffbfff9fff8fffcfff9fff8fffbfff9fff9fffcfffcfffcfffffffbfffcfffafff8fff7fff8fff7fff6fff9fff9fffafffcfffbfffdfffdfffcfffcfffcfffcfffbfffbfffafff9fffcfffafff7fffafffbfff9fffbfffafff8fff7fff8fffafff8fff8fffafffafffafffbfffcfffcfffafffafffbfffcfffafffafff9fffafffafff9fff8fff8fff9fff9fffbfff8fffafffafffafffbfffbfff9fffafffafffafff9ff7ae96eb8" + return stride(from: 0, to: hex.count, by: 2).map { + let s = hex.index(hex.startIndex, offsetBy: $0) + return UInt8(hex[s...hex.index(after: s)], radix: 16)! + } + }() + + func testAnIntactBufferIsBanked() { + XCTAssertEqual(realBuffer.count, Whoop5RawImu.bufferLength) + XCTAssertTrue(Collector.isBankableImu(realBuffer)) + } + + func testACorruptBufferIsNotBanked() { + var corrupt = realBuffer + corrupt[600] ^= 0x01 + XCTAssertNotNil(Whoop5RawImu.rawColumns(corrupt), "precondition: the decoder alone would accept it") + XCTAssertFalse(Collector.isBankableImu(corrupt)) + } + + func testOtherFramesAreNotBanked() { + XCTAssertFalse(Collector.isBankableImu(Array(realBuffer.prefix(124)))) + XCTAssertFalse(Collector.isBankableImu([])) + } +} diff --git a/docs/review/workstreams/W06-ble-collect.md b/docs/review/workstreams/W06-ble-collect.md index 7997d72d66..0ef81402f9 100644 --- a/docs/review/workstreams/W06-ble-collect.md +++ b/docs/review/workstreams/W06-ble-collect.md @@ -76,7 +76,7 @@ exactly what was tested on hardware. | ID | Sev | Status | Finding | Location | Evidence | PR | |---|---|---|---|---|---|---| | W06-001 | S4 | Reported | GET_CLOCK reply dropped for 5/MG | `BLEManager.swift` 5/MG handshake; `FrameRouter.swift` `COMMAND_RESPONSE` | State confirmed: sent on every connect, the reply is only hex-logged (10 replies in owner logs), no decode sets a clock, `clockRef` stays nil. No effect on history dating (records carry unix seconds) or safe-trim, so lowered from S3; what remains is readout truth (W06-019), Phase 3 | | -| W06-002 | S2 | Reported | A Raw Data Collector session on a 5/MG never records an IMU sample: `ImuSessionFileStore.append` is reachable only through `Collector.ingest(frame:parsed:)`, which nothing calls | `Collector.swift` `ingest`, `recordGroundTruthImu`; `BLEManager.swift` 5/MG frame loop | P5 confirmed (found by W1 and W6). #1709 moved IMU storage behind an ingest seam 5/MG never enters; upstream has the same gap. User-started from Test Centre | | +| W06-002 | S2 | Fixed | A Raw Data Collector session on a 5/MG never records an IMU sample: `ImuSessionFileStore.append` is reachable only through `Collector.ingest(frame:parsed:)`, which nothing calls | `Collector.swift` `ingest`, `recordGroundTruthImu`; `BLEManager.swift` 5/MG frame loop | P5 confirmed (found by W1 and W6). #1709 moved IMU storage behind an ingest seam 5/MG never enters; upstream has the same gap. V0: 0 production callers of `Collector.ingest(frame:parsed:)`, the only path to `ImuSessionFileStore.append`. Fix: the 5/MG frame loop calls `Collector.bankImuForSessions` for every frame, live and offload, gated to intact 1244-byte buffers. V1: `CollectorImuBankingTests` (intact banked, CRC-flipped refused); Strand and NOOPiOS (no watch) build. V3 pending: a Raw Data Collector session on the iPhone must report covered seconds | | | W06-003 | S3 | Reported | Mapped v20 (optical) and v21 (IMU) have no storage lane: every record goes to the 5 MB rolling reject archive, which one sync saturates, so only the newest ~630–720 of each survive after the strap has trimmed them | `Interpreter.swift` `mappedWhoop5HistoricalVersions`; `HistoricalStreams.swift` `rejectedHistoricalRecords`; `RawHistoryArchive.swift` | P5 confirmed with caveat: nothing consumes v20/v21, so no shown metric is wrong; a retention decision is missing. Every line in the 9 archives in the repo root is type 47 v16, v20 or v21. Root cause of W06-004 and W06-005. AD-6 | | | W06-004 | S3 | Reported | Every over-cap reject-archive rewrite runs on the main actor (about 290 ms for a 5 MB archive on an M-series Mac) and, with v20/v21 holding the archive at cap, recurs every 7–8 chunks through an offload | `RawHistoryArchive.swift` `archive`, `evictLines`; `Backfiller.swift` `rejectedSink` | Found by pass 5 (missed by pass 3). AD-3 evidence; measure on the iPhone in Phase 3 | | | W06-005 | S3 | Reported | Every sync that carries v20/v21 records ends with the error "couldn't be decoded (unrecognised strap firmware layout) … saved on this Mac", on iOS too; the strap log calls them "CRC/unmapped layout", and Sleep's freshness note takes its sync-failed branch | `BLEManager.swift` `exitBackfilling`; `Backfiller.swift` `finishChunk` log | P5 confirmed (found by W1 and W6). 410 such lines across 9 owner logs; 181 of 193 dumped rejects are intact v20 or v21 | | From 6a65375255afcc34d1badf1750cf3938392dd512 Mon Sep 17 00:00:00 2001 From: Artur Iskakov Date: Fri, 25 Sep 2026 13:41:02 +0200 Subject: [PATCH 2/2] docs(review): log the first W6 batch --- docs/review/workstreams/W06-ble-collect.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/docs/review/workstreams/W06-ble-collect.md b/docs/review/workstreams/W06-ble-collect.md index 0ef81402f9..4fa6fe23da 100644 --- a/docs/review/workstreams/W06-ble-collect.md +++ b/docs/review/workstreams/W06-ble-collect.md @@ -110,3 +110,7 @@ exactly what was tested on hardware. ack; a second HISTORY_START mid-session), the kill-mid-offload strap run, R22 deep-data routing, the RR source-precedence read filter. Next: V0 (pass 4) for every `Reported` row; the hold-ack tests (W06-007) come before any fix in this area. +- 2026-09-25 — Pass 4 and fixes started with the only S2: W06-002 fixed (IMU buffers banked into Raw Data Collector + sessions from the 5/MG frame loop); `StrandTests` and both app builds green. Branch `review/w06-fixes`. Next: the + strap run the owner agreed to (one sync, one short Raw Data Collector session, strap log attached), then the S3s, + starting with W06-007 (hold-ack tests) before any safe-trim change and W06-005 (the false sync warning).