NVCM Programmed check always returns true

No description provided.

Root cause found (static analysis; bench rig in use, so no hardware runs — verified against the sensor-fw hardware record instead).

The check's verdict came from the auto-boot test bytes of the OW_FACTORY_NVCM_CHECK blob: _interpret_nvcm_blob (motion_connector.py:551) treated boot_0x40_responds == 0 ("0x40 stopped ACKing after a key-less CRESETB release") as proof of a programmed NVCM. That signal is unconditionally "programmed": the CrossLink I2C slave config port at 0x40 only becomes active after receiving the activation key, so it never ACKs in that phase regardless of NVCM state. Result: every camera with a valid IDCODE reported PROGRAMMED — blank parts included.

This is the exact failure sensor-fw already hit and fixed in its runtime detector — openmotion-sensor-fw commit c40a6b4 ("fix(nvcm): rewrite fpga_detect_nvcm to use Done bit instead of boot test", hardware-verified: blank cam8 read Done=0 while the boot test claimed programmed). The app's interpreter was ported from the SDK's scripts/nvcm_probe.py, whose verdict logic predates that finding.

Fix (draft PRs up):

App: fix: NVCM programmed check reads STATUS Done bit, not the auto-boot test #50 — verdict now from the STATUS register Done bit (bit 8 → blob[8] bit 0), which the blob already carries; interpreter extracted to utils/nvcm_verdict.py with 16 unit tests incl. a regression case from the real blank-camera blob (STATUS 00 00 02 08 + boot test claiming "0x40 gone" → must be BLANK).
SDK: fix: NVCM probe verdict from STATUS Done bit, not the auto-boot test openmotion-sdk#118 — same correction to the reference script scripts/nvcm_probe.py + nvcm_check docstring, so the wrong verdict logic can't be re-ported.

Hand-test: run "Check NVCM (all)" on a module with known-blank cameras → expect BLANK per camera (previously PROGRAMMED); an NVCM-burned camera (Done fuse set) → PROGRAMMED; absent/unpowered → NO RESPONSE/INCONCLUSIVE.

fix: NVCM verdict reads STATUS bit 19, not the SRAM Done bit

Fix merged to next in cb08844 (PR #50; SDK probe companion in openmotion-sdk 5220d94, PR #118).

Verified with a blind hardware test (2026-07-02): Ethan installed one blank-NVCM camera in an undisclosed slot; a 16-camera SDK sweep + the app's Check NVCM both had to find it.

First cut of the fix (STATUS Done bit, bit 8) was falsified on the bench: in the probe's ISC flow the part is held unconfigured, so Done reads 0 on every camera → all 16 reported BLANK.
Raw-blob sweep found the real discriminator: STATUS bit 19 (BE word 0x00080000) — set on all 15 programmed cameras (00 08 02 08), clear on exactly one (00 00 02 08), which matched the known-blank blob from the sensor-fw notebook. Blind call: right module, slot 1 — confirmed correct.
Feature row / feabits / NVCM-row reads return all-0xFF for programmed AND blank parts in this mode (bare ISC_ENABLE doesn't read-enable the array) — not usable as corroboration.
Verdict updated to bit 19 in both the app (utils/nvcm_verdict.py, 10/10 unit tests incl. both real hardware signatures) and the SDK probe script; re-ran through the app UI against ground truth — correct per-camera verdicts confirmed by Ethan.

In review for release-time verification. Note for the record: bit 19's datasheet name is unconfirmed — the claim is empirical (15+1 samples, two independent blanks). If a camera ever reads amiss, re-run openmotion-sdk/scripts/nvcm_probe.py and compare against the two signatures above.

TestApp_1.4.1

Issue Fixed with Sensor 1.5.0v

TestApp_1.4.1
Sensor 1.8.1v

Issue is not fixed

ow-testapp-20260710_111656.log
ow-testapp-20260710_111601.log
ow-testapp-20260710_110118.log

ow-testapp-20260710_141103.log
ow-testapp-20260710_140833.log

@vpennam

The fix is correct — it has simply never shipped. Re-validated on the bench today (2026-07-15) against a third independent blank camera.

Why @vpennam saw "issue not fixed" on 2026-07-10

TestApp 1.4.1 does not contain the fix. The 1.4.1 tag is 353cfe2, dated 2026-06-22; the fix landed on next on 2026-07-02 (cb08844). No tag contains it, and origin/main does not even have utils/nvcm_verdict.py. So 1.4.1 still runs the original auto-boot-test logic and reports PROGRAMMED for everything, blank parts included — exactly the reported symptom.

The earlier "Issue Fixed with Sensor 1.5.0v" observation was also not the fix working: OW_FACTORY_NVCM_CHECK (0x6C) first shipped in sensor-fw 1.6.0, so on 1.5.0 the firmware rejected the command outright. Neither of Vinay's runs ever exercised the corrected code.

Bench re-validation (sensor fw 1.8.1-rc.3, TCA-safe: one camera powered at a time)

Module with a known-blank camera in slot 1, verdicts computed by the shipped utils/nvcm_verdict.py:

cam	boot test (old logic)	STATUS bit 19 (fix)	STATUS
1	PROGRAMMED ✗	BLANK ✓	`00 00 02 08`
2–8	PROGRAMMED	PROGRAMMED	`00 08 02 08`

The two signals disagree on exactly the blank camera, and bit 19 is the one that is right. Reproduced on a repeat run. A second module (all-programmed) read 00 08 02 08 on all 8 — correctly all PROGRAMMED, no false BLANKs. boot_0x40_responds = 0 on every camera measured, including the confirmed blank: the boot test is degenerate, which is the root cause of this issue.

Correction to the record

openmotion-sensor-fw/docs/fpga-nvcm-autodetect.md states "Camera 8's NVCM is confirmed PROGRAMMED ... reproduced 5×", with STATUS 00 00 02 08. That conclusion rests on the auto-boot test and had no blank negative control (the notebook says as much: "use it as the negative-control reference in place of a blank camera"). Since boot_0x40=0 on genuinely blank parts, that verdict was itself a false positive from this bug — cam8 was most likely blank, and its 00 00 02 08 agrees with the bit-19 reading. Filing a doc correction against sensor-fw so this doesn't mislead the next reader.

Empirical basis for bit 19 is now three independent datasets: the 07-02 blind sweep (15 programmed + 1 blank), an all-programmed module, and today's blank-slot-1 module. Unit tests: 10/10.

Blocked on a release only — next → main + tag. Nothing further to implement.

Try this with test app 1.4.1-dev.0

Reopened investigation — the bit-19 fix has a failure class the blind sweep couldn't catch. Field report: right sensor camera 8 reads PROGRAMMED in the app while every scan SRAM-loads it.

New root cause (hardware-verified today)

STATUS bit 19 is "SDM Enable" per the Lattice CrossLink I2C programming spec status-register map (p27): it is set when self-download is enabled because the NVCM Done fuse is programmed. It tracks the fuse, not bootability. Right cam 8 is a Done-fuse-burned, non-bootable part:

bit19=1 (00 08 02 08, byte-identical to a good NVCM part) even probed straight from a cold camera power-up — and camera power-off verifiably depowers the FPGA (config port unreachable), so this is not a stale-latch artifact;
firmware pin test: C8: NVCM detect clk=1 data=1 → not driving → SRAM load (13.9 s / 12.5 s across two runs). Good left cam 8: clk=0 data=0, "NVCM programmed, skipping SRAM load", 0.11 s;
after a keyless boot attempt, right cam 8's config port stops responding until power cycle — consistent with a boot that starts (fuse says go) and fails.

The 2026-07-02 blind validation used truly-blank (fuse-unburned) cameras, which bit 19 does discriminate — this burned-but-dead class is invisible to it, and nothing else in the OW_FACTORY_NVCM_CHECK blob reflects boot success.

Fix (PR #61)

The check now drives the firmware's own detector — the same signal the scan path trusts: OW_FPGA_RESET (clears the isProgrammed cache) + timed non-forced OW_FPGA_PROG_SRAM; verdict = skip (<2 s ⇒ PROGRAMMED) vs SRAM load (⇒ BLANK/unbootable). Margin is two orders of magnitude on both sides. Side effect: the check is no longer read-only — a non-booting camera ends up SRAM-configured and running, same as after a scan.

Also worth noting for hardware triage: right sensor camera 8's NVCM appears partially/badly burned (OTP) — it can likely never be NVCM-flashed to a bootable state and will need SRAM loads (or a camera swap) permanently.

fix: log per-camera NVCM check verdicts for log-based triage

Correction to my 2026-07-17 comment: the claim that camera 8 "can likely never be NVCM-flashed again" is retracted as unproven (flagged by Ethan). It stacked two inferences that don't hold up: (1) bit 19 = Done fuse burned — June bench data shows a programmed part reading bit19=0 in the same probe mode, so the bit's semantics aren't established; (2) fuse burned = array unwritable — the NVCM array is unreadable in this probe flow, so re-burn feasibility is simply unknown. What remains established: cam 8 does not boot from NVCM (behavioral pin test), which is what the checker now reports. All 'cannot be re-flashed' wording has been removed from the app (feature/62 d311970) and the SDK probe (fix/158 de4c33c); bit 19 is now surfaced as a neutral indicator only.

upgrade sensors to 1.8.2-dev.1 and 1.4.1-dev.1 test app

TestApp_1.4.2-dev.0 and Sensor 1.8.2-dev.1

Issue Not fixed,

TestApp_1.4.2-dev.0 and Sensor 1.8.2-dev.1

Issue Fixed

Navigation Menu

Repository navigation

NVCM Programmed check always returns true #44

Why @vpennam saw "issue not fixed" on 2026-07-10