Repository navigation
Read Q' by dimension name in the summed-Q' baseline - #196
Merged
taddyb merged 1 commit intoSep 11, 2026
Merged
Conversation
scripts/summed_q_prime.py indexed Qr positionally in two places, taking axis 0 as divides and axis 1 as time. That is correct only for a store written in contract order, and the failure is silent rather than loud: zarr answers an out-of-range read with fill values instead of raising, so a transposed store yields an all-NaN lateral inflow and a baseline of zero. Both reads now go through ddr.io.readers.qr_as_divide_time, which selects by dimension name and refuses a variable whose dimensions match neither layout. The helper lives in the reader module rather than the script because the script imports cupy at module scope and so cannot be imported without a GPU, which is why the existing tests reimplement its logic inline. Same class of defect as the gridded fixes in DeepGroundwater#195, on the MERIT path. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YZBLnrygRbcnyQvu2KL4vz
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Completes the axis-order work from #195 by fixing the same defect on the MERIT path.
The defect
scripts/summed_q_prime.pyreadQrpositionally in two places, taking axis 0 as divides and axis 1 as time:That is correct only for a store written in the contract order
Qr(divide_id, time), and the failure mode is silent rather than loud. zarr answers an out-of-range read with fill values instead of raising, so a transposed store yields an all-NaN lateral inflow and a baseline of zero — a number that looks like a result rather than an error. The summed-Q′ baseline is what every routing skill claim is measured against, so a silently zero baseline would make routing appear to beat it everywhere.This is the same class of bug fixed for the gridded path in #195. It has not produced a wrong number in practice, because every store in use happens to be in contract order; the point is that nothing was enforcing that.
The fix
Both reads go through a new
ddr.io.readers.qr_as_divide_time, which selects and transposes by dimension name and raises on a variable whose dimensions match neither layout, rather than quietly returning fill values.The helper lives in the reader module rather than in the script because
summed_q_prime.pyimports cupy at module scope and so cannot be imported without a GPU. That is also why the existing tests for this script reimplement its logic inline instead of calling it; putting the helper inddr.io.readersmakes the behaviour directly testable.Three tests: a contract-order store passes through unchanged, a transposed store produces an identical matrix, and dimensions matching neither layout raise.
Verification
812 tests pass against 809 on master, the difference being exactly the three added here. No behaviour change is expected or observed, since the stores in use are already in contract order.
Still open
The gridded benchmark builds its forcing by flowline-midpoint assignment while training uses the area-weighted store, so the two measure with different inflow. Roughly 40% of catchments straddle a cell edge. Not addressed here.