Repository navigation
Release 1.13.0 - #321
Merged
Merged
Release 1.13.0#321
Conversation
…-columns fix: select histogram bins by column name
…s (Refs #266) Reads a <label>_export.csv written by the bloodflow-app History -> Export (SessionPlayback.materialize_corrected_csv) — both the per-camera and the reduced clinical side-average layouts — and renders per-cam traces + nan-aware side averages for BFI/BVI/mean/contrast/temp plus a per-camera quality strip, with a console summary of frame gaps, active-cam masks, and quality counts. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: boringethan <ethan@openwater.cc>
…scripts (Refs #266) scripts/visualize_scan.py now writes a single self-contained interactive HTML page instead of a static PNG. Every panel shares one linked time axis, hovering shows per-sample values, and clicking a camera in the legend hides it across all five metric panels at once. It detects the format from the header and accepts any CSV the current app writes: the corrected/History-export scan CSV (with per-frame quality on hover), the reduced clinical side-average CSV, the raw histogram CSV (reduced to per-camera image mean and standard deviation), and the console telemetry CSV. Several can be passed together — a scan plus its telemetry lands on one axis, which is the point. Legacy layouts are rejected by name rather than mis-parsed. Supersedes and removes view_corrected_scan.py, view_scan_export.py and plot_telemetry.py. plot_dark_drift.py stays: dark-schedule drift is a distinct analysis, not a view. plotly is a new optional viz extra, deliberately not a runtime dependency of the wheel the apps ship. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: boringethan <ethan@openwater.cc>
- Raw CSV: scheduled dark frames are drawn as their own marker series instead of dipping the light mean/std line every dark interval. - Raw CSV: truncated/ragged rows are counted and skipped instead of raising IndexError (a scan interrupted mid-write leaves one). - Folder mode keeps the newest raw CSV per side, not just the newest. - Scan summary counts quality only for active cameras; masked-off cameras no longer inflate the "(blank)" count. - Ragged rows dropped by the column reader are reported in the summary. - Docstring: the per-scan corrected CSV is opt-in (writeCorrectedCsv). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: boringethan <ethan@openwater.cc>
…isualization-ae8610 feat: one interactive scan/telemetry CSV viewer (replaces four plot scripts)
…d Root install (Refs #216) Squashed onto next with DCO sign-off; original commits: ## feat: EV-sign the WinUSB driver MSI + catalogs via eSigner CKA; retire the self-signed Root install CI now prefers the org's SSL.com EV code-signing cert: driver-msi.yml sets up eSigner CKA (key held in SSL.com's cloud HSM, exposed through Windows CNG) from the ES_USERNAME/ES_PASSWORD/ES_TOTP_SECRET secrets and exports CODESIGN_THUMBPRINT, which flips build_driver_msi.ps1 into EV mode: signtool-sign the 4 catalogs AND the MSI (RFC3161-timestamped), derive the shipped .cer from the store cert, and wix build with -d EvSigned=1. EvSigned MSIs install the cert into TrustedPublisher only (the EV chain is publicly trusted, so no more planting a private root on user machines) and additionally delstore the retired self-signed CN=Openwater WinUSB cert (6E8D9AAC...) alongside the already-removed leaked one (D3A6A27A...). Package version bumped 1.1.0 -> 1.2.0 so installed machines upgrade. While the ES_* secrets don't exist, CI falls back to the legacy PFX path with a loud warning so PR builds stay green; the legacy self-signed mode also remains available for local bench builds (never ship it). Refs #216 ## chore: EV-sign the driver only on manual dispatch — PR runs use the legacy PFX eSigner cloud signings are metered; the shipped driver is signed once per driver change (dispatch -> download artifact -> vendor into bloodflow-app). PR-triggered runs now always take the zero-cost legacy self-signed path, even with the ES_* secrets present. Refs #216 ## feat: commit the canonical EV-signed driver zip in-repo winusb-driver/OpenMotionDriver-x64.zip is now tracked content: the EV-signed artifact from driver-msi.yml dispatch run 32908313110 (Open Water Internet, Inc., ADFCD7CB..., sha256 162792c6...), signed once per driver change since eSigner signings are metered. bloodflow-app resources/ vendors a byte-identical copy. Local/PR rebuilds overwrite the zip with a legacy-signed one -- the build script now warns to revert it; never commit a legacy-signed zip. Refs #216 ## fix: commit the EV-signed catalogs; drop driver_install.cmd (Refs #216) The four .cat files are now the EV-signed ones from inside the canonical zip (byte-identical, extracted from its cab, no new signing), so the repo matches what ships and an EV dispatch re-signs none of them (1 signing, the MSI, instead of 5). driver_install.cmd, the unused manual install path that still added the cert to the Root store, is removed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: boringethan <ethan@openwater.cc>
…gning feat: EV-sign WinUSB driver MSI + catalogs via eSigner CKA; retire self-signed Root install
#182) Comments and docs only; no behavior change. - dark.py / moments.py and the sibling pipeline stages cite docs/SciencePipeline.md by current heading name + number (e.g. "§5.8.2 HybridRealtimePredictor"), replacing section numbers that no longer exist (§4.2, §6, §7.x, §8.x, §9, §11, §12, §16). - _emit_interval comments no longer claim it applies the D_prev stencil: it emits a raw CorrectedInterval and DarkFrameHoldStage._apply_stencil applies the stencil downstream. - CorrectedFrame.raw_u1 is write-only (nothing downstream reads it). - FrameBatch.tcm/tcl are int64 trigger counters (0 when no sample), not TEC temperatures; contrast_raw's comment and moments.py name ShotNoiseCorrectionStage, not BfiBviStage, as where contrast is computed. - CAMERA_GAIN_MAP comment: only ShotNoiseCorrectionStage uses it. - sinks.py: the raw channel is gated by the pipeline's Tee("raw") (no ScanMetadata.write_raw_csv exists); ScanWorkflow.start_scan, not the runner, forces the corrected CSV on without a scan DB; the app-side _FinalBatchSink no longer exists. - dark.py module docstring: DarkCorrectionStage is in this module. - docs/API.md: remove the nonexistent ScanDatabase.iter_raw_frames / get_raw_frame; raw histograms live only in the raw CSVs. Based on @HussainAther's #238 (and #213). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: boringethan <ethan@openwater.cc>
- test_telemetry_listener_fires: wait on a threading.Event set by the listener's second callback (up to 5 s) instead of a fixed 2.5 s sleep. The slow refresh is every 10th 100 ms poller tick, each refresh's UART reads stretch the period, and the session-scoped poller can be at any phase when the listener registers, so two callbacks did not reliably fit in 2.5 s. The listener is removed in a finally block. - test_tec_voltage_read: tec_voltage() reads back the TEC DAC setpoint. The AD5761R is configured for -5..+5 V in console firmware and the setter accepts [-5.0, 5.0], so assert that range rather than the 0..3.3 V ADC range (a -0.07 V readback is valid). - docs/TestPlan.md: describe both tests as they now behave. Based on @HussainAther's 3dc7d3e from #224. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: boringethan <ethan@openwater.cc>
…op frames The scan thread runs every pipeline stage and sink inline. While it is busy (a dark-interval close with its scan-DB writes, a GC pause, a slow disk), nothing drains the queues behind it. Each side's packet queue held 64 USB reads, about 0.65 s of total slack with the shared batch queue and the sensor's own 5-frame buffer. Past that the USB reader stops reading and the sensor drops frames on both modules at the same instant; past ~0.9 s the gap exceeds 8 frames and the frame classifier locks the side out (#286). Size each packet queue for PACKET_BUFFER_SECONDS (10 s) of output instead: 3600 reads per side, sized for uncompressed packets (9 reads each), at most ~15 MB per side and only while actually backed up. The queue holds compact raw bytes, so this is the cheap place to buffer; the batch queue behind it holds dense per-row arrays and stays at 4. Also record each side's queue high-water mark, log it when streaming stops, and warn once it passed half full, so field logs show how close a scan came to losing frames. Refs #116 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: boringethan <ethan@openwater.cc>
…acket-queue fix: size the USB packet queue by time so scan-thread stalls don't drop frames
When a whole sensor module skipped more than 8 frames, the frame classifier quarantined every later frame until the 8-bit frame id wrapped back to its stale anchor: 6.4 s of good data discarded, and the side's absolute ids left 256 low for the rest of the scan, so every scheduled dark after it landed on a lit frame (#286; field: bloodflow-app#516, 2026-09-18 MotionDemo log). Re-anchoring after a large gap required the delivery monitor to have seen one camera go missing from otherwise-flowing packets, which a whole-module gap never shows because there are no packets at all. Re-anchor from the device clock instead. A frame that fails direct acceptance (step > 8, backward, or a multi-step claim the clock contradicts) is held as a candidate at the absolute id the clock implies, if that id's low byte is the wire id (+-1 frame for crystal drift over long gaps). The stream re-anchors there only if the next frame continues it by exactly one frame and ~25 ms. A corrupted frame id fails the first test and a single capture with matched counter+timestamp corruption fails the second, so the #220 protections hold (the etch-a-sketch suite passes unchanged). The epoch comes from the clock, so gaps past 127 frames, which the 8-bit id alone can't express, also land on the right ids, including a gap of 256*n + 1 frames that reads as a single +1 step. That evidence covers a single camera's outage too, so the separate camera-outage permission (_resync_allowed / allow_large_resync) is removed. CameraStreamGap alerts are unchanged. test_unannounced_large_gap_fails_closed_without_delivery_evidence encoded the old lockout and is replaced by tests for the new behaviour. Refs #286 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: boringethan <ethan@openwater.cc>
fix: re-anchor the frame counter from the device clock after a gap
…retired) The console firmware never implemented OW_CTRL_READ_ADC or OW_CTRL_READ_GPIO; both IDs fall through to its default case and return no payload, so these methods could only ever raise. The only callers were two xfail tests. Remove the constants, both MotionConsole methods, the tests and their TestPlan.md entries. Console ADC and GPIO data stay available through the dedicated commands (TECADC, PDUMON, GET_TEMPS, BOARDID, GET_FAN). Closes #302 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: boringethan <ethan@openwater.cc>
…adc-gpio refactor: remove read_adc_value / read_gpio_value (console 0x1E/0x1F retired)
libusb 1.0.30 fixes denial-of-service bugs that a malicious USB device
can trigger with malformed descriptors. The package shipped 1.0.29 for
pyusb and a 1.0.24-era build for dfu-util.
- omotion/_vendor/libusb/windows/{x64,x86}: VS2022 builds from the
official libusb-1.0.30.7z (signature and GitHub digest verified).
- omotion/dfu-util/{win64,win32}/libusb-1.0.dll: the MinGW builds from
the same archive. dfu-util.exe loads the DLL next to it; the MinGW
build needs only msvcrt, and it exports every symbol dfu-util.exe
imports. This also drops the old win32 DLL's dependency on
libgcc_s_sjlj-1.dll, which never shipped.
- Removed the copies nothing loads: dfu-util-static / lsusb-static
(static 1.0.24-era libusb), the libusb static and import libraries, and
the macOS dylib, which the macOS dfu-util never loaded because it links
Homebrew's libusb by absolute path.
- Removed the repo-root libusb-1.0.29/ distribution and the 1.0.28
libusb-1.0.dll. Neither shipped. Provenance now lives in
omotion/_vendor/libusb/README.txt.
- tests/test_vendored_libusb.py fails if a package DLL is older than
1.0.30 or if any other libusb binary reappears.
Refs OpenwaterHealth/openmotion-bloodflow-app#669
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: boringethan <ethan@openwater.cc>
fix: vendor libusb 1.0.30 for pyusb and dfu-util
apply_laser_power() wrote the limit registers on the Safety EE and OPT FPGAs but never read them back, so a register that did not take the intended value went unnoticed. Every load now reads back each register it wrote on those two FPGAs (PULSE_WIDTH_LL/UL, RATE_LL, DRIVE_CL and the PWM/CW current limits) and compares it with the bytes it wrote, whether from the baseline, a user config override, the EE/OPT_THRESH DRIVE CL write or the force_fault set. Each register gets up to 3 reads. The result is latched as console.laser_limits_check (MotionInterface.laser_limits_check). A mismatch, an unreadable register or an incomplete load returns False and names each register with its expected and read values. While the check is failed, ScanWorkflow.start_scan refuses laser scans in its pre-flight with the reason in last_scan_error, and MotionConsole.start_trigger raises LaserSafetyLimitError without sending anything. Only a later load that verifies clean clears it. Refs #310 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: boringethan <ethan@openwater.cc>
Read the firmware's OW_CMD_RESET_HISTORY (0x10, openmotion-sensor-fw#137) record and add it to the sensor's connect-time ==== block. The block shows how the previous session ended (power off, host reset, watchdog, fault, ...), how long that session ran, boots since power-on with a per-reason breakdown, and uptime, USB recoveries, ECC/DMA errors and the RCC flags. The bloodflow app already calls log_sensor_info() on every sensor connect, including the reconnect after a mid-scan drop, so the app log answers "why did it drop?" with no app change. A shutdown that means something went wrong (watchdog, fault, ECC, Error_Handler) is repeated as a separate WARNING record, so it shows up in a failure grep. Firmware without the command answers OW_UNKNOWN, and the block says the reason is unavailable. - omotion/reset_history.py: parse/describe/format, pure and unit-tested - MotionSensor.get_reset_history(): never raises; None on old firmware Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: boringethan <ethan@openwater.cc>
Same checks and the same refusals, with less machinery:
- The latch is a string, console.laser_limits_error (None = verified or
no load yet), instead of LaserLimitCheck/LimitReadback dataclasses.
apply_laser_power sets it on entry and clears it only at the end of a
load that verifies clean, so the existing write-failure returns stay
as they were and no try/finally split of the body is needed.
- One helper reads the written safety registers back and returns a line
per failing register ("EE_RATE_LL: expected 70313, read 0"). Values
print as raw little-endian integers; the unit conversion and the
FpgaMap "unit" key are gone.
- No read retries: a failed read fails the load, as a failed write
already does.
- MotionInterface is untouched; callers read
iface.console.laser_limits_error.
Refs #310
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: boringethan <ethan@openwater.cc>
…-safety-limits fix: verify the laser safety limits after every load (#310)
feat: log the sensor's last shutdown reason at connect (#313)
…comments docs: fix stale pipeline comments and API.md phantom raw-frame methods (#182)
…y-test-flake test: de-flake console TEC voltage and telemetry listener tests (#178)
A Stop that landed while the scan worker was still in its pre-flight sent
cancel_scan's stop_trigger before the worker's start_trigger. The trigger
and laser then ran on after Stop, the terminal fsync count was read
mid-run and named a lit frame (TERMINAL DARK CONTAMINATED on every
camera), and the app never saw trigger OFF ("trigger clock still open").
cancel_scan now sets _cancel_requested before stopping, and a new
_trigger_lock orders its stop against the worker's trigger start: the
worker skips start_trigger once a cancel has arrived, and a Stop that
races an in-flight start_trigger waits for it and then stops it.
Refs #318
Refs OpenwaterHealth/openmotion-bloodflow-app#726
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: boringethan <ethan@openwater.cc>
…ight-race fix: don't start the trigger after a Stop during scan pre-flight (#318)
vpennam
approved these changes
Oct 14, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Release 1.13.0:
next→main. This SDK ships in Open-Motion app 1.5.4. The app's rc and production builds install exactly the release pinned in itssdk-version.txt, so 1.5.4-rc.0 can't be cut until this is on PyPI.12 PRs since 1.12.0. It's a minor bump because of the new features (#313, #266, #216). Note that #302 also removes two public methods. Neither app calls them; checked on bloodflow-app
nextand test-appnext.Scan data integrity
Safety and security
Diagnostics
Removed
read_adc_value/read_gpio_value: console commands 0x1E/0x1F are retired (refactor: remove read_adc_value / read_gpio_value (console 0x1E/0x1F retired) #303, closes Remove read_adc_value / read_gpio_value (console commands 0x1E/0x1F retired) #302).Tests and docs
Before merging
maincarries two README commits thatnextdoesn't have (1541f04, 01938d8: the disclaimer under the title). The release merge is clean (git merge-treereports no conflicts). Mergemainback intonextafter the release sonextpicks them up.Post-release checklist
1.13.0on the merge commit onmain. release-build then creates the GitHub Release with the wheel and sdist.publish-pypi.yml -f tag=1.13.0by hand, because it doesn't fire on its own. Then check thathttps://pypi.org/pypi/openmotion-sdk/1.13.0/jsonreturns 200.sdk-version.txt→ 1.13.0 PR intonextbefore tagging 1.5.4-rc.0.🤖 Generated with Claude Code