Replay scorer: FFT angular-fragmentation metric for graph consistency by MJohnson459 · Pull Request #90 · ClachDev/Mote

MJohnson459 · 2026-08-02T09:31:40Z

Score how geometrically self-consistent a replayed SLAM map is, from the FFT
wall-orientation spectrum the declutter pass already computes, and put the
numbers in the bag-replay report so parameter sweeps rank map quality instead of
the operator eyeballing PNGs.

Branch: angular-fragmentation-metric (commit b785a19, NOT pushed - per the
global rule that remote git writes need explicit approval).

What changed

NEW mote_bringup/mote_bringup/map_cleanup/angular_stats.py - numpy-only, no
cv2. Holds the four spectrum helpers (moved verbatim from
structure_extraction) plus angular_stats(wall, params=None,
reference_directions=None).
structure_extraction.py imports the two helpers it uses; declutter behaviour
unchanged.
metrics.map_quality gains the angular keys via a lazy import that degrades to
omitting them on ImportError, keeping the module numpy-only/ROS-free.
bag_replay/report.py: three new ranked columns with arrows/bolding, plus
per-set wall-direction and orthogonal-frame tables in ## Maps, plus four new
Limitations bullets.
NEW mote_bringup/test/test_angular_stats.py (14 cases, synthetic grids only);
test_metrics.py gains a case pinning the keys and the graceful degradation.
Both READMEs gain a paragraph on what the score does and does not prove.

Design

The flat has an angled hallway, so it genuinely has three dominant wall
directions. A peak count cannot be a score, and a single Manhattan-frame residual
charges the building for being itself with the same signal it should raise for
drift - and is not rotation-stable anyway (measured: off-frame 0.778 -> 0.547 ->
0.671 on an unchanged map rotated +17/-31).

So the ranked scalars are frame-model-free and rotation-invariant by
construction: angular_support_deg, angular_entropy_norm, unassigned_energy_frac
(all lower-better). Direction and frame tables are structure, not score.
manhattan_* is kept unranked and marked not-rotation-stable;
orthogonality_error_deg is omitted as instructed. The defect verdict is optional
and needs a prior via reference_directions.

Stated limit, not papered over: within one map a coherent drift-rotated section
is angularly indistinguishable from real architecture; neither raises unassigned
energy. The frame table separates one extra direction from a duplicated frame;
only a prior convicts below its merge tolerance.

Three deviations from the task's stated defaults, each measured

Peak threshold for the stats is 0.15, not Params' 0.45. Decision 4 asked me
to switch peak-picking from e to q and verify. The family table improves
(zero-share noise peaks vanish) but at 0.45 a whole family is dropped on some
rotations of an unchanged map, and its energy lands in unassigned_energy_frac
(0.099 -> 0.181 -> 0.163). At 0.15: 0.099 / 0.071 / 0.084. Params itself is
untouched, so declutter is unaffected.
Frame merge tolerance is 10 deg, not decision 5's stated default of 12.
Acceptance requires the 13-deg rotated fixture to show n_directions>=2 on the
secondary frame; at 12 it merges back into the dominant frame and the case
is indistinguishable from the angled corridor. 8/10/11 all work; 10 is above
the 7.5 deg measured intra-frame shear. Run 3's real 23-38 deg tear reads
identically at 10 or 12.
Decision 8 asked for angular_stats to be cv2-free AND to import Params from
structure_extraction, which imports cv2 (and would be circular). Resolved by
duck-typing: angular_stats carries its own SpectrumParams with the six
spectrum-scan fields, Params stays exactly where it is and passes straight
in, and a test pins the two default sets together.

Also: decision 2's crop is to the wall bounding box, since the public signature
takes a wall mask alone. That is contained in the decided-cell bbox, so it is the
stricter crop; documented, and pinned by an extent-invariance test.

Verification

pixi run test (colcon test mote_bringup): 336 tests, 0 failures, 2 skipped
(foxglove, needs a live bridge). test_metrics.py: 11 pass with mote_bringup on
the path and 11 without it.
Declutter byte-identical: cleaned PNG and diagnostics panel match by sha256
before/after the helper move, over four real maps (both run-3 legs, hospital
and mote sim sites). diff -r reported no differences at all.
segment-eval unchanged: 30/33 mapped hospital rooms, 10/10 office, 1/1 mote,
zero merges - the documented baseline exactly. test_room_segmentation passes.
2026-07-29 evidence reproduces to 3 dp pre-crop: concentration 0.627/0.252,
dispersion 13.51/24.04, off-frame 0.778/0.658, support 38.11/42.04, and the
direction table identically (2.25/0.198, 24.75/0.324, 92.25/0.082,
118.75/0.297).
Post-crop the numbers move as decision 2 predicted (support 38.11 -> 37.72,
42.04 -> 43.05) and the two-frame tear signature survives: 0.632/0.277 at
22.5 deg, 0.557/0.297 at 41 deg (task quoted 0.621/0.280 ~23 deg and
0.564/0.297 ~38 deg). unassigned differs (0.091 vs 0.068) because peaks are
now picked on q at 0.15 rather than e at 0.45 - see deviation 1.
A demonstration report.md was built offline from the npz pair (no replay
re-run); new columns arrow and bold correctly and both tables render.
pre-commit clean.

Not done, deliberately

The run-3 pair is not an acceptance gate - both legs are damaged maps, and the
ranked scalars do not separate them (the better leg by loop drift scores worse on
support, having explored 59 m2 against 81 m2). Documented as a coverage confound
in both the report and the README rather than tuned away. Persisting a per-site
reference direction set is out of scope and filed as voro #334.

…rispness The truth-free map metrics (wall thickness, speckle, unknown fraction) catch blur, noise and incompleteness. They are blind to the failure that actually matters on the flat's bags: a section of the map drawn at the wrong angle. A drift-rotated room is crisp, unspeckled, and wrong, and until now only the eye caught it. The whole design turns on one fact about the building being mapped: the flat is mostly square but has an angled hallway, so it genuinely has three dominant wall directions and always will. That kills a peak count as a score, and it kills fitting one Manhattan frame and charging the map for deviating from it -- the hallway's walls sit off the main frame by construction, so a single-frame residual charges the building for being itself, with exactly the signal it is meant to raise for drift. Measured, that single-frame fit is not rotation-stable either: rotating an unchanged map +17/-31 deg moves its off-frame fraction 0.778 -> 0.547 -> 0.671, the fitted frame jumping between competing families. So the ranked scalars are frame-model-free -- angular_support_deg (exp(H) x bin width, the effective degrees of wall direction a map uses), angular_entropy_norm and unassigned_energy_frac -- and rotation-invariant by construction, since a rotation is a circular shift of the spectrum. The direction and frame tables are reported as structure, not score. The defect verdict needs a prior and is optional: pass reference_directions and get the offset, the off-reference energy and the dispersion, with the reference set free to rotate as a whole because a map frame's absolute rotation is an accident of where SLAM started. State the limit plainly: within one map a coherent drift-rotated section is angularly indistinguishable from real architecture. Both are a narrow extra wall family, and neither raises unassigned energy. The frame table is what separates *one* extra direction (a hallway) from a *duplicated orthogonal frame* (a rotated section), and only a prior convicts below its merge tolerance. Three things the measurements decided rather than taste: - Floor subtraction is load-bearing. On the raw spectrum, broadband clutter dominates and the torn leg scores *better*; subtracting a 45-deg circular smooth is what makes the numbers mean anything. - Peaks are picked on the floor-subtracted residual, which drops the zero-share noise families the raw energy invents, at a relative threshold of 0.15 rather than the declutter's 0.45. At 0.45 a whole family is dropped on some rotations of an unchanged map and its energy lands in unassigned_energy_frac (0.099 -> 0.181 -> 0.163); at 0.15 the same map gives 0.099 / 0.071 / 0.084. The declutter's own threshold is untouched -- it selects wedges to keep, a different job where being conservative is right. - Frame merge tolerance is 10 deg, not 12. It must exceed the shear a genuine frame carries (the conservative leg's own frame is internally sheared 7.5 deg) and stay under the tear being resolved. At 10 a 13-deg rotated section reads as a second frame of 2 directions while an angled corridor stays 1; at 12 the two become indistinguishable. Run 3's real 23-38 deg tear reads the same at either. The spectrum now has one implementation: _angular_energy, _smooth_circular, _pick_directions and _angdist move verbatim into map_cleanup/angular_stats.py, which is numpy-only and imports no cv2, and structure_extraction imports the two it uses. angular_stats carries its own copy of the six spectrum-scan defaults rather than importing Params, because the dependency has to run that way -- structure_extraction imports cv2, and metrics.py must keep its numpy-only contract. Params itself is unmoved and duck-types straight in; a test pins the two default sets together. metrics.map_quality imports angular_stats inside the function and omits the angular keys on ImportError, so a benchmark run in an environment without mote_bringup on the path still scores. The bag-replay report gains the three ranked columns with their arrows, and renders the direction and frame tables per parameter set. Confounds documented rather than tuned away: coverage confounds the ranked scalars (the run-3 leg that is better by loop drift, 0.551 m vs 8.776 m, scores worse on angular support, 43.0 vs 37.7, having explored 59 m2 against 81 m2), a multi-angle building is not a defect, and n_peaks is threshold-bound. The run-3 pair is deliberately not an acceptance gate -- both legs are damaged maps, so they show only the torn end of the scale. Verified: - pixi run test: colcon test mote_bringup, 336 tests, 0 failures, 2 skipped (foxglove, needs a live bridge), including 14 new test_angular_stats.py cases on synthetic grids with no data-file dependency. - The declutter pass is unchanged: cleaned PNG and diagnostics panel are byte-identical (sha256) before and after the helper move, over four real maps (both run-3 legs, the hospital and mote sim sites). - segment-eval unchanged: 30/33 mapped hospital rooms, 10/10 office, 1/1 mote, zero merges -- the documented baseline exactly. - The 2026-07-29 measured evidence reproduces to 3 dp before the crop (concentration 0.627/0.252, dispersion 13.51/24.04, off-frame 0.778/0.658; support 38.11/42.04; the direction table identically), and the two-frame tear signature survives it: 0.632/0.277 at 22.5 deg and 0.557/0.297 at 41 deg. - test_metrics.py passes both with mote_bringup on the path and without it. - pre-commit clean. Co-authored-by: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019atjx6AYGCum2Sp82cGQVq

MJohnson459 · 2026-08-02T17:40:59Z

Re-scope note (2026-08-02, from the mapping-pipeline design):

The original purpose — ranking param sweeps — is now covered elsewhere: lockstep replay (#295) + the loop drift metric + candidate preview (#339) decided the 2026-08-02 sweep without these scalars. But this work gets a sharper job under docs/design/mapping-pipeline.md (branch mapping-pipeline-design):

The build report's open-trajectory tear alarm. loop drift is only meaningful when the trajectory closes — the 2026-08-02 session happened to end at its start; a budget-exit explore session gives the scorer no drift number at all. For those, the frame-grouping diagnostic here (a drift-rotated section duplicates its whole orthogonal frame; an angled hallway adds one direction) is the only automated tear signal map-build (#343) will have. We watched exactly this signature live on today's torn control map.
One shared spectrum implementation. The angular_stats.py consolidation is independently valuable: the map-build alignment step uses the same angular-energy fold (windowed energy, folded 0/90, sub-bin interpolation) to measure wall rotation before re-solving. It was hand-rolled twice during the 2026-08-02 session — the first attempt wrongly, because the un-windowed FFT's axis-aligned leakage pinned the fold to 0°. One canonical, tested implementation prevents that class of error.
The reference direction set is a site fact — same category as the zone vocabulary — so its eventual home is the site bundle (out of scope here, as the task already says; noting the destination).

Review accordingly: the ranked scalars matter less than (a) the frame-grouping table being trustworthy for ≥~20° tears with its ~12° merge-tolerance limit stated, and (b) angular_stats.py being clean enough for the alignment step to import. Consumers: #343 (scoring input), #347 (report rendering).

Re-scoped against docs/design/mapping-pipeline.md. Ranking parameter sweeps is now covered by lockstep replay (#295) + loop drift + candidate preview (#339), so the scalars this started as are no longer the deliverable. The two jobs that remain are the frame-grouping tear alarm (#343 scoring input, #347 rendering) and one canonical spectrum implementation for the map-build alignment step. The ranked columns are demoted. Loop drift bolds slam-conservative (0.551 m vs 8.776 m) while all three angular scalars bolded the other leg -- a bolded row asserting the opposite of the correct answer, on the only real pair available. The confound is coverage (59 m2 vs 81 m2: fewer long walls read as tighter), and since ranking is someone else's job now, the honest fix is to stop claiming it. `angular_support_deg` stays as an unbolded descriptive column beside a new `n_strong_frames` (frames holding >=15% of the energy), which is the tear signal an operator actually reads. Every scalar is still computed and in run.json. The tear alarm's working band is pinned. It is relied on where the trajectory does not close and there is no drift number at all, so 20/25/30/40 deg tears are now tested directly -- secondary frame present, >=2 directions, >15% energy, and the reported offset matching the true tear angle -- rather than resting on the 13 deg case. Its floor is pinned too, as a fact rather than a caveat in prose: a 5 deg tear reads as ONE frame and must, because the merge tolerance has to exceed the 7.5 deg shear a genuine frame carries or honest shear would be reported as damage. New `wall_rotation()` is the alignment primitive the design asks for: windowed energy, folded 0/90, sub-bin interpolated, with `fold_90` and `refine_peak` exposed separately so a caller can use the pieces. Three things were measured rather than assumed. - **The leakage failure does not reproduce here.** A hand-rolled fold reportedly read 0 deg because the un-windowed FFT's axis-aligned leakage dominated. With this implementation the peak lands at 59.25 deg un-tapered and 58.75 tapered on a 31 deg room outline (truth 59.0), and the energy within 2 deg of 0/90 is 0.026-0.030 across every combination of taper and low-cut. `_angular_energy` already drops the DC neighbourhood and uses magnitude rather than power, which is the likely reason. Pinned as a regression so it stays true -- this is the concrete payoff of consolidating on one implementation. - **Tapering a tight crop is worse than not tapering at all** (0.68 deg mean rotation error vs 0.39 untapered): the Hann roll-off cuts into the walls, and a rotated shape's corners reach the crop edge. So `wall_rotation` pads by ROTATION_PAD_FRAC before tapering, which gives 0.14 deg mean / 0.26 worst, and a test asserts the un-padded path is more than twice as bad so a future simplification cannot quietly drop it. Extent invariance is exact. - **Sub-bin interpolation is required, not polish.** 0.5 deg bins cannot resolve the 1-2 deg shear this eventually has to measure. Refinement beats the bin grid, and the docstring states plainly that 0.14 deg is good enough to drive a re-solve and NOT good enough to certify a 1-2 deg shear absent. The reference direction set remains out of scope; its destination is the site bundle, same category as the zone vocabulary (voro #334). Verified: - colcon test mote_bringup: 354 tests, 0 failures, 2 skipped (was 336; +18). - Declutter still byte-identical (sha256, cleaned map + diagnostics, four real maps) -- `wall_rotation`'s windowing is opt-in and off for `angular_stats`. - segment-eval unchanged: 30/33 hospital, 10/10 office, 1/1 mote, zero merges. - test_metrics passes with and without mote_bringup on the path. - Demo report rebuilt from the run-3 npz pair: no angular column is bolded, and `wall frames (>=15% energy)` reads 2 for both legs -- both torn, correctly. - pre-commit clean. Co-authored-by: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019atjx6AYGCum2Sp82cGQVq

Both PR #90 review comments were correct. Verified each before acting; the first is a genuine false-positive generator on exactly the signal this module exists to report. **1. A non-square transform canvas invented tears.** The angular scan measures orientation in array *index* space, and a frequency-domain index maps to a real frequency divided by that axis' length -- so on an oblong array both axes carry different scales and every angle is skewed towards the long one. A genuinely perpendicular pair of wall families then stops looking perpendicular: at bbox aspect 0.8 they split by 12.7 deg, at 0.66 by 23.2, at 0.5 by 36.9 -- all past FRAME_MERGE_DEG, so `_frames_table` reported two orthogonal frames for one intact building. Reproduced on an elongated but perfectly rectilinear building: 2 frames at 10, 20 and 30 deg rotation, 1 after the fix. Note which maps the bug spared -- 0 and 90 deg are fixed points of the distortion, so it is invisible on an axis-aligned map and appears only once the map frame is rotated, which a real SLAM frame always is. The run-3 legs escaped by luck (aspect 0.915 and 0.968). `room_segmentation.py` already pads to square for this reason, citing an 8 deg skew on a 58 x 38 m map, and its comment already drew the distinction that matters: the declutter pass is immune because it puts its wedges back in the same index space it found them in, but a number *reported* as an angle is not. Everything this module emits is in the second category. `_crop_to_content` now pads to square, so both entry points get it. **2. Reported "wall directions" were wall normals, 90 deg off.** A wall's Fourier energy lies on the ridge perpendicular to it -- a horizontal line transforms to a vertical frequency ridge -- so a spectrum peak is the wall's normal. Confirmed directly: horizontal walls reported 88.25 deg. It never broke internal consistency, since frames and `wall_rotation` work mod 90 where a constant +90 cancels, but the direction table and the report's "Wall directions" column presented normals under a label promising orientations, and `reference_directions` asked callers for real wall angles it then compared in the wrong frame (the fit's global offset search hid it). `spectral_to_wall` / `wall_to_spectral` now convert at the reporting boundary; horizontal walls read 2.25 deg. The `_angular_energy` docstring asserted the opposite of the truth and is corrected. **Consequences, re-measured rather than re-tuned.** Squaring moved every number, and it also invalidated an earlier claim of mine: "tapering a tight crop is worse than not tapering" was partly measuring the aspect bug. Re-derived over rotations of 3 deg and up: pad 0.0 -> 0.215 mean / 1.062 worst, pad 0.5 -> 0.068/0.102. Padding still wins, by more than the test demands. It also exposed a floor worth stating plainly, found by chasing a single outlier: **`wall_rotation` under-reports below about 2 deg.** A wall line rotated less than ~2 deg rasterises into runs long enough that its dominant spectral content is still axis-aligned, so the peak is pulled onto the axis. Measured true -> reported: -0.5 -> -0.13, -1.0 -> -0.46, -1.5 -> -0.46, -2.0 -> -2.26, -3.0 -> -2.90. A bigger building does not help; it is rasterisation, not resolution. So it is usable at 2 deg and up and **cannot measure the 1-2 deg residual shear** -- an alignment step correcting from this number would silently under-correct. Pinned by a test that asserts the shortfall, not just documented. Scalar rotation invariance loosened from 10% to 20% for `angular_support_deg`, measured at 3.6-15.2% over +17/-31/+23/+45: rotating a building changes its bounding box, hence how much padding squaring adds, hence the broadband floor. Entropy stays within 1.0-4.6%. Another reason support is not a ranking column. Real run-3 numbers after the fix, with the aspect skew removed: support 39.32 / 42.15 (was 37.72 / 43.05), and the tear offsets 25.0 deg / 41.0 deg (the first was 22.5 before -- the padding corrected it). Both legs still report 2 strong frames, so the tear signature survives; docs and report prose updated to the corrected figures. Verified: - colcon test mote_bringup: 364 tests, 0 failures, 2 skipped (was 354; +10). - Declutter still byte-identical (sha256, cleaned map + diagnostics, four real maps) -- confirming the fix touches only the reporting path, as intended. - segment-eval unchanged: 30/33 hospital, 10/10 office, 1/1 mote, zero merges. - test_metrics passes with and without mote_bringup on the path. - Demo report rebuilt: frame offsets now 25.0 and 41.0 deg. - pre-commit clean. Co-authored-by: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019atjx6AYGCum2Sp82cGQVq

claude Bot reviewed Aug 2, 2026

View reviewed changes

Comment thread mote_bringup/mote_bringup/map_cleanup/angular_stats.py

claude Bot reviewed Aug 2, 2026

View reviewed changes

Comment thread mote_bringup/mote_bringup/map_cleanup/angular_stats.py

MJohnson459 merged commit da0cca1 into main Aug 2, 2026
3 checks passed

MJohnson459 deleted the angular-fragmentation-metric branch August 2, 2026 20:54

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Replay scorer: FFT angular-fragmentation metric for graph consistency - #90

Replay scorer: FFT angular-fragmentation metric for graph consistency#90
MJohnson459 merged 3 commits into
mainfrom
angular-fragmentation-metric

MJohnson459 commented Aug 2, 2026

Uh oh!

MJohnson459 commented Aug 2, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Uh oh!

Conversation

MJohnson459 commented Aug 2, 2026

What changed

Design

Three deviations from the task's stated defaults, each measured

Verification

Not done, deliberately

Uh oh!

MJohnson459 commented Aug 2, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant