Skip to content

Make the two hypnogram-coverage readers agree, and state the gate's real scope - #1721

Merged
ryanbr merged 1 commit into
mainfrom
hypnogram-coverage-parity
Aug 29, 2026
Merged

ryanbr merged 1 commit into
mainfrom
hypnogram-coverage-parity

Conversation

@ryanbr

@ryanbr ryanbr commented Aug 29, 2026 •

Copy link
Copy Markdown
Owner

Follow-up to #1717, from reviewing it. No behaviour change for any payload a producer can emit.

The two readers disagreed on shapes the oracle did not enumerate

That PR pins coverage with an oracle across 18 payload shapes — a good technique, and it found the 0.94999999999999996 boundary that reading the code could not have settled. The divergence it could not see is structural: Swift decodes the stage array as a whole (as? [[String: Any]]), so a single non-object element makes the entire payload unmeasurable, while Kotlin skipped per element and measured the remainder. Kotlin's optDouble also parsed a string bound that Swift's as? NSNumber rejects.

I compiled both and ran them rather than reasoning about it:

payload Swift Kotlin
[{seg},5] null → not holed 0.1 → holed
[{seg},null] null → not holed 0.1 → holed
[{"start":"0","end":"100"}] null → not holed 0.1 → holed

Unreachable from any current producer — every writer emits homogeneous {start,end,stage} segments or a timestamp-free shape — so this is latent, not a live defect. It is still a divergence in the one module whose stated contract is byte-parity.

Kotlin now bails on a non-object element and reads bounds the way NSNumber does. Seven rows added to the oracle, bringing it to 25.

The alignment direction was measured, not assumed. A bool bound converts on both sides, because JSONSerialization bridges true to NSNumber 1 — so the tidy-looking fix of rejecting anything non-numeric would have been wrong. Worse, under Foundation {"start":0} reports is Bool == true, so expressing it as a bool exclusion would have discarded every timeline whose first segment starts at zero — which is most of them, including the oracle's own tiling case. That one is pinned with the reason attached.

Two scope corrections

Both documentation; neither changes code.

Xiaomi is in scope. The module said the gate stays away from imports because their minute-dict shape carries no timestamps. That is true for the WHOOP CSV, Apple and Health Connect, but it is not a blanket exemption: XiaomiBandImporter emits real {start,end,stage} segments, and takes its span from bedtime/wake_up_time fields that are independent of the items array it builds those segments from. A Xiaomi night whose items do not reach its own bed/wake bounds is judged holed. That is arguably the right answer — the night genuinely is only partly described — but it is a real behaviour change for that importer, unvalidated against a Xiaomi export, and it is now pinned by test rather than left to be discovered. Flagging in case you want it gated differently.

The engine's protection is a different mechanism than documented. At the merge site, fraction(stagesJSON:) returns nil for timestamp-free shapes — the rule as described. At the engine site coverage is summed off decoded segments, not off stagesJSON, so that rule never runs. What actually keeps those imports safe is coveredSeconds > 0 returning nil for a group with no timestamped stages at all. Same outcome, different mechanism — and it holds only while a group is single-sourced, which the day merge ensures by choosing one side. A group mixing a staged fragment with a minute-dict one would read as holed; both halves are now pinned.

Two rows added on re-review

Both were shapes I had argued rather than run — the same gap this PR exists to close, so leaving them would have been inconsistent.

Health Connect's real payload was never pinned. The oracle's minute-dict row is the throwing {light,deep,rem,awake} object, but hcStagesJson does not emit that — it emits an array of {stage,min} objects, which parses cleanly and reaches the segment loop. So the one live producer shape this gate claims exemption from was reaching null by an entirely different path (every element is an object, so no bail; none carries bounds, so every segment is skipped; cover stays zero) with nothing asserting it. Pinned now along the path it actually takes. A JSON null bound inside a well-formed object is pinned beside it.

The inline comment saying "the dict shape throws here" was true of the dict and false of Health Connect's array, sitting six lines under the doc this PR corrects. Reconciled.

Verification

Android 4752 pass, 0 failures — including the extended 25-row oracle, which is the actual proof the alignment landed: the Kotlin implementation now reproduces the Swift twin's output on every new shape.

Swift package tests cannot build on this machine (GRDB's CSQLite module), so the Swift half is syntax-checked locally and rests on CI. The new Swift expectations (0.099, 0.125) were generated by running the twin standalone, not written by hand.

Unrelated, worth knowing: the two RecoveryDriversTest float failures reported as a pre-existing baseline in #1717 do not reproduce here — the full Android suite was clean at that base and is clean now, so that looks environmental.

…eal scope

Follow-up to #1717, from reviewing it.

The coverage gate is pinned by an oracle across 18 payload shapes, and the two
readers still disagreed on shapes the oracle did not enumerate. Swift decodes
the stage array as a WHOLE (`as? [[String: Any]]`), so one non-object element
makes the entire payload unmeasurable; Kotlin skipped per element and measured
the remainder. Kotlin's `optDouble` also parsed a string bound that Swift's
`as? NSNumber` rejects. Compiled both and ran them:

  [{seg},5]                        Swift null/not-holed   Kotlin 0.1/HOLED
  [{seg},null]                     Swift null/not-holed   Kotlin 0.1/HOLED
  [{"start":"0","end":"100"}]      Swift null/not-holed   Kotlin 0.1/HOLED

Unreachable from any current producer - every writer emits homogeneous
segments or a timestamp-free shape - so this is latent, not a live defect. It
is still a divergence in the one module whose contract is byte-parity, and the
oracle would never have caught it.

Kotlin now bails on a non-object element and reads bounds the way NSNumber
does. Both directions were checked against the real Swift, not against my
reading of it: a bool bound CONVERTS on both sides, because JSONSerialization
bridges `true` to NSNumber 1, and the tidy-looking alignment of rejecting
non-numbers would have been wrong - under Foundation `{"start":0}` reports
`is Bool == true`, so a bool exclusion would have discarded every timeline
whose first segment starts at zero. Five rows added to the oracle.

Two scope corrections, both documentation:

- The module said the gate stays away from imports because their minute-dict
  shape has no timestamps. True for the WHOOP CSV, Apple and Health Connect,
  but not a blanket exemption: the Xiaomi Band importer emits real
  {start,end,stage} segments and takes its span from bedtime/wake_up_time
  fields independent of the items array it builds them from, so a Xiaomi night
  IS judged here. Arguably right - such a night genuinely is only partly
  described - but it is a real behaviour change for that importer, unvalidated
  against an export, and now pinned by test rather than left to be found.

- At the ENGINE call site coverage is summed off DECODED segments, not off
  stagesJSON, so the payload-shape rule never runs there. What actually keeps
  timestamp-free imports safe is `coveredSeconds > 0` returning nil for a group
  with no timestamped stages at all. Same outcome, different mechanism, and it
  holds only while a group is single-sourced - which the day merge ensures.

Re-review added two more rows, both shapes previously argued rather than run.
Health Connect's REAL payload is an ARRAY of {stage,min} objects, not the
throwing {light,deep,rem,awake} dict the oracle already had: it parses, reaches
the loop, and exits with zero cover. That is the live producer shape this gate
claims exemption from, so it is now pinned along its actual path rather than by
assertion. A JSON null bound inside a well-formed object is pinned beside it.
The inline comment naming only the throwing dict was reconciled with the doc.

No behaviour change for any payload a producer can emit. Android 4752 pass.
Swift package tests cannot build on this box (GRDB CSQLite), so the Swift half
is syntax-checked locally and rests on CI; the new Swift expectations were
generated by running the twin standalone rather than by hand.
@ryanbr
ryanbr force-pushed the hypnogram-coverage-parity branch from b10c638 to 7cdd796 Compare August 29, 2026 00:40
@ryanbr
ryanbr merged commit e152f76 into main Aug 29, 2026
15 checks passed
@ryanbr
ryanbr deleted the hypnogram-coverage-parity branch August 29, 2026 01:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant