Skip to content

Read Q' by dimension name in the summed-Q' baseline - #196

Merged
taddyb merged 1 commit into
DeepGroundwater:masterfrom
taddyb:fix/summed-qprime-axis-order
Sep 11, 2026
Merged

taddyb merged 1 commit into
DeepGroundwater:masterfrom
taddyb:fix/summed-qprime-axis-order

Conversation

@taddyb

@taddyb taddyb commented Sep 10, 2026

Copy link
Copy Markdown
Collaborator

Completes the axis-order work from #195 by fixing the same defect on the MERIT path.

The defect

scripts/summed_q_prime.py read Qr positionally in two places, taking axis 0 as divides and axis 1 as time:

qr_hourly = hourly["Qr"].values          # assumed (divides, hours)
n_days = qr_hourly.shape[1] // 24        # ...so shape[1] must be time

That is correct only for a store written in the contract order Qr(divide_id, time), and the failure mode is silent rather than loud. zarr answers an out-of-range read with fill values instead of raising, so a transposed store yields an all-NaN lateral inflow and a baseline of zero — a number that looks like a result rather than an error. The summed-Q′ baseline is what every routing skill claim is measured against, so a silently zero baseline would make routing appear to beat it everywhere.

This is the same class of bug fixed for the gridded path in #195. It has not produced a wrong number in practice, because every store in use happens to be in contract order; the point is that nothing was enforcing that.

The fix

Both reads go through a new ddr.io.readers.qr_as_divide_time, which selects and transposes by dimension name and raises on a variable whose dimensions match neither layout, rather than quietly returning fill values.

The helper lives in the reader module rather than in the script because summed_q_prime.py imports cupy at module scope and so cannot be imported without a GPU. That is also why the existing tests for this script reimplement its logic inline instead of calling it; putting the helper in ddr.io.readers makes the behaviour directly testable.

Three tests: a contract-order store passes through unchanged, a transposed store produces an identical matrix, and dimensions matching neither layout raise.

Verification

812 tests pass against 809 on master, the difference being exactly the three added here. No behaviour change is expected or observed, since the stores in use are already in contract order.

Still open

The gridded benchmark builds its forcing by flowline-midpoint assignment while training uses the area-weighted store, so the two measure with different inflow. Roughly 40% of catchments straddle a cell edge. Not addressed here.

scripts/summed_q_prime.py indexed Qr positionally in two places, taking axis
0 as divides and axis 1 as time. That is correct only for a store written in
contract order, and the failure is silent rather than loud: zarr answers an
out-of-range read with fill values instead of raising, so a transposed store
yields an all-NaN lateral inflow and a baseline of zero.

Both reads now go through ddr.io.readers.qr_as_divide_time, which selects by
dimension name and refuses a variable whose dimensions match neither layout.
The helper lives in the reader module rather than the script because the
script imports cupy at module scope and so cannot be imported without a GPU,
which is why the existing tests reimplement its logic inline.

Same class of defect as the gridded fixes in DeepGroundwater#195, on the MERIT path.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YZBLnrygRbcnyQvu2KL4vz
@taddyb
taddyb merged commit 186584e into DeepGroundwater:master Sep 11, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant