Repository navigation
Leakance and learned n(d) coexist; Juniata and regional curvature landscapes - #50
Merged
Merged
Conversation
zeta multiplies leakance_factor by K_D, so only their product reaches the physics: an exact scaling degeneracy. Measured over 346,321 CONUS reaches on a trained arm, rho(leakance_factor, K_D) = +0.9986 and all four learned outputs of that head hold 97 percent of their variance in one direction. Physically a reach either sits above a losing aquifer or it does not, so the factor should select where leakance acts rather than scale how much. A gate and a conductance are not degenerate with each other. The gate is a sharpening sigmoid on the head's normalized output, g = sigmoid(logit(clamp(u, eps, 1-eps)) / tau), annealed by an epoch-keyed schedule that mirrors the learning rate. tau = 1 is special-cased to return u untouched, so every config without params.leakance_gate is bit-exact unchanged. Differentiable at every positive tau, so autograd gives the exact gradient and no straight-through estimator is introduced. Applied to the head output before setup_inputs, so no routing operator or backward changes. Replicated across all three hand-written head readers rather than unified, per the WET preference, with the eval reader and dump_parameters using the FINAL temperature so the scored model is the one training ended on. A previous bug shipped because an eval reader silently dropped a learned head output. Gates: lib 377 passed; leakance_gate 11; gamma_eval_parity 4; sp8_gradcheck 20; leakance_gradcheck 16; leakance_off_parity 3; zeta_accum 11; leakance_reference_match 5; ddr_sandbox_match 1. compare_ddr_sandbox ABSOLUTE MATCH at 1.525879e-4 m3/s, unchanged. verify_doc_paths 0 unresolved. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0143bx7gVz8ug2oc6DTy5jtz
104 numbered equations over 13 sections, each derived from the code rather than from prior docs, with symbol tables giving units and whether each quantity is learned per reach, set by config, read from data, or derived. Covers the head and its normalization, the new gate, the geometry chain with a catalogue of every clamp and what it does to the gradient, stage-dependent roughness, the Muskingum coefficients, the sparse adjoint, the leakance flux with all six partials, the gradient-path labels, all five losses, and the diagnostics. It found ten code-versus-documentation discrepancies. The load-bearing one: the leakance term computes its width as (p*d)^q while the routing geometry computes the same reach's width as p*d^q, so the area multiplying K_D is smaller by the depth-independent factor p^(q-1) = 0.3445 at the prescribed p = 21, q = 0.65, and a learned K_D absorbs it. The flux and skill results are unaffected because a constant rescaling is absorbable, but K_D is not directly comparable to a measured streambed leakance without dividing it out. Inherited from DDR, so the port is faithful; leakance_reference_match pins that. validate_params.py now divides the factor out and records the caveat. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0143bx7gVz8ug2oc6DTy5jtz
…gradient-sink check The equations reference flags that nothing in the repo measures how often the geometry clamps saturate, which matters because a saturated clamp zeroes the gradient to the parameter behind it, so training cannot identify that parameter at that reach. Measured on the 2026-09-16T04-44-15Z arm over 64,892 eval reaches: the depth floor takes 2.1 percent, and the side-slope bounds, the bottom-width floor and both roughness range bounds take none. The side-slope clamp cannot fire in this configuration: with q fixed at 0.65 its lower bound needs a depth of 1,751 m and its upper bound a depth of 3.2 mm, which the 10 mm depth floor already forbids. So clamps are not the reason the learned outputs collapse onto one direction. That is consistent with the head-topology screen, which found capacity is not the constraint either, leaving the gradient and the objective. Figures are a lower bound: computed from each reach's mean eval depth, so a clamp that fires only at low flow is invisible. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0143bx7gVz8ug2oc6DTy5jtz
… a decision Three plotting faults, all of which misled the reader. A prescribed constant is not a learned field. dump_parameters writes every parameter the run used, learned or not, so the width exponent and coefficient were being mapped as if the head had produced them when both come from params.defaults. They are now detected by zero variance, reported as constants in the caption line, and left off the figure. A gate is a decision, not a magnitude. Percentile colour limits rescaled a field that never leaves 0.96 to 1.0 into a full rainbow, which made a gate that is ON everywhere look richly structured. The gate now uses a fixed 0 to 1 scale and the panel title states the fraction actually switched off, which is 0.0 percent. The roughness label depends on the arm. With a learned stage exponent n is n_0, roughness at the reference depth; without one it is Manning's n. The title now says which, since config rejects a learned gamma alongside leakance so these arms are always the latter. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0143bx7gVz8ug2oc6DTy5jtz
Equal flux requires K_phys * A_phys = K_D * A_ours, so K_phys = K_D * p^(q-1) = K_D * 0.3445. Our area is SMALLER than the physical one, so the physically equivalent conductivity is SMALLER than the learned number. I had it inverted, which overstated the implied streambed conductivity by 2.9x and put the gated arm at the 52nd percentile of the measured range when it is at the 13th. Also reports the distance of the learned median from the box centre in decades, which is the test that matters: a sigmoid head with near-zero output bias reports an unconstrained parameter at its box centre, so proximity to the centre is evidence the data did not move it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0143bx7gVz8ug2oc6DTy5jtz
Before this, `TimestepLeakanceOp` had eight parents and none of them was gamma, so a learned stage-roughness exponent would have silently received no gradient while leakance was on. `validate_learned_gamma` rejected the combination at config load rather than let that happen, which made the two features mutually exclusive. `TimestepLeakanceGammaOp` is the nine-parent sibling, following the same pattern `TimestepGammaOp` already uses for gamma without leakance. The backward math is shared through the new `leakance_backward_body`, so the two leakance ops differ only in parent bookkeeping and the eight-parent path is untouched. Gamma and zeta couple through the geometry accumulators that `timestep_backward_core` already shares: gamma re-weights the chain below depth (B5/B15/B17) while zeta reads depth, p_spatial and q_eps and is folded into b_rhs (B27). No new cross-term is needed, which is a claim, so `tests/leakance_gamma_gradcheck.rs` measures it: all nine parents swept against central differences with both features active (worst rel 9.3e-4), plus two identities — a learned gamma of zero is bit-identical to plain leakance, and a learned constant gamma matches the global stage-roughness path to 1e-5. Also restores the gitignored .cargo/config.toml CUDARC_CUDA_VERSION override lost when the build directory was cleared (trap T12). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0143bx7gVz8ug2oc6DTy5jtz
…ked for `sr_n0_gamma_leakance.yaml` is the first configuration in which both mechanisms that can absorb depth-dependent volume error are allowed to compete: leakance removes water, gamma changes how fast water moves at a given stage. Every prior leakance run used a depth-independent n, so the comparison was never possible. Three boxes widened, from research/findings/2026-09-17-parameter-range-audit.md: gamma [0, 0.5] -> [-0.2, 0.85] K_D [1e-8, 1e-6] -> [1e-8, 1e-3] d_gw [-2, 2] -> [-5, 1] The gamma floor is the substantive one. Zero encoded "channels get smoother as they fill", which holds in-bank and fails on a vegetated floodplain or any composite section, where effective roughness rises with stage. Those reaches were not discouraged, they were unrepresentable. `validate_learned_gamma` now admits [-0.5, 1]; the guard floor is far from the only real singularity, at gamma = -(5+3q)/3 (about -2.3 at q = 0.65). Both signs are gradient-checked. The default gamma box is deliberately NOT changed, so no existing config moves; the widened box is opt-in per experiment. lr is five times cooler than the no-leakance arms. A CONUS full-recipe leakance run at 0.005 diverged to NaN at epoch 15 with a finite loss and a non-finite gradient, and 5x lower removed it. Gates: ddr_sandbox_match + compare_ddr_sandbox ABSOLUTE MATCH (1.53e-4, the same value as before this branch), --lib 373, and 91 tests across mmc, sparse_gradcheck, sp8_gradcheck, stage_roughness, the four leakance suites, gamma_eval_parity and leakance_reference_match. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0143bx7gVz8ug2oc6DTy5jtz
head = min(depth - d_gw, depth + M) Once the water table falls more than the streambed thickness M below the bed, an unsaturated zone opens beneath the channel, stream and aquifer stop being hydraulically connected, and the flux is set by the head across the bed layer alone rather than by the distance down to the table. The previous linear head had no such saturation. This is not a refinement, it is the difference between a d_gw that can be learned and one that cannot. 9.3% of CONUS reaches have the table more than 5 m below the bed (p95 14 m, p99 71 m, max 148 m), so representing them needs a deep box; but under a linear head a deep box is actively harmful. At depth 1.5 m a d_gw of -80 m gives 12x the flux of -5 m for the same conductance, and a large head with a small K_D is indistinguishable from a small head with a large one. Widening the box without the cap buys degeneracy, not coverage. With the cap, every state below -M is the same physics, so the box does not need to be deep at all. What it needs instead is to CENTRE in the connected region: connection is exactly `d_gw > -M` (depth cancels from both sides), and the d_gw gradient is identically zero once disconnected. A box centred below -M would have given d_gw no gradient at epoch 1 and it would never have moved. Hence M = 1.0 with d_gw in [-2, 1]: centre -0.5, connected, two thirds of the box gradient-bearing. `leakance_bed_thickness` is absent by default, and absent means no cap and a byte-identical forward, so the DDR c2bd0f9 reference match and every prior leakance run stay valid. The eval zeta diagnostic recomputes the head independently and was updated in step, or it would have reported a flux the routing never applied. Six new unit tests pin the semantics: back-compat when absent, flux independent of d_gw once disconnected, unbounded growth WITHOUT the cap (so a silent disable fails loudly), connection independent of depth, the vanishing d_gw gradient, and central differences through the capped forward in both regimes. Gates: --lib 383, compare_ddr_sandbox ABSOLUTE MATCH 1.525879e-4 (unchanged), 91 tests across the eleven routing and leakance suites. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0143bx7gVz8ug2oc6DTy5jtz
Attribution needs all four cells of {leakance on/off} x {n(d) on/off}, and the
existing sr_n0_only.yaml and sr_n0_gamma.yaml cannot serve: they run the hot
learning rate (0.005 vs 0.001) and the old gamma box [0, 0.5], so they differ
from the combined arm in three ways at once, not one.
Each control is derived from sr_n0_gamma_leakance.yaml by removing exactly one
mechanism. Same gauges, seed, windows, head, schedule, and identical boxes for
every parameter the two arms share.
ctl_neither n_0 alone the floor
ctl_gamma n_0 + gamma n(d) only
ctl_leakance n_0 + K_D/d_gw/factor the cell every prior
leakance run occupied
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0143bx7gVz8ug2oc6DTy5jtz
…he init The 2026-09-17T15-47 CONUS run went NaN on the FIRST optimizer step. Loss was finite through all four micro-batches of mb=0 (0.51, 0.40, 0.42, 0.81) and every value after the step was NaN, so the forward was fine and the gradient was not. Cause, measured by re-running with the K_D box as the only difference, same seed: K_D [1e-8, 1e-6] centre 1e-7 loss 0.149 2.9% reaches negative pre-clamp K_D [1e-8, 1e-3] centre 3.16e-6 loss 0.513 30.8% reaches negative pre-clamp The head is a sigmoid with near-zero output bias, so every parameter starts at its box centre, and for a log-space parameter that is the GEOMETRIC mean. Widening the ceiling 1000x moved the starting conductance 31.6x. zeta is linear in K_D, so leakance drained a third of the network below zero at epoch zero, the S28 clamp rewrote all of them, and the gradient through that many saturated clamps went non-finite immediately. This is the same mistake as the d_gw box: widening a box does not just add reach, it relocates where training begins. [1e-9, 1e-5] has its geometric centre at exactly 1e-7, the historically stable init, while spanning four decades and covering the measured streambed range (8.8e-8 to 2.1e-5 1/s). The same diagnostic run carried the widened gamma box AND the disconnection cap and was stable through the optimizer step, so both are exonerated. It also exposed a real flaw, previously masked because the denormalize bug kept K_D frozen at a tiny constant in every leakance run ever executed: NOTHING BOUNDS ZETA BY THE WATER IN THE REACH. `losing_only` fixes the sign of the exchange, not its magnitude, so a large enough conductance removes more water than exists. Recorded in the config; `reaches with negative discharges pre clamp` is the instrument (no-leakance rate is 2-4%). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0143bx7gVz8ug2oc6DTy5jtz
Records the single-variable measurement behind e2f7b96: a K_D box widened to [1e-8, 1e-3] moved the geometric init 31.6x, drove 30.8% of reaches negative pre-clamp against a 2-4% baseline, and went non-finite on the first optimizer step with a finite forward. Also records the flaw that exposed: nothing bounds zeta by the water in the reach. losing_only fixes the sign of the exchange, not the magnitude. Masked for the whole history of the feature by the denormalize bug that froze K_D. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0143bx7gVz8ug2oc6DTy5jtz
zeta <- min(zeta, alpha * relu(b_rhs_base))
`leakance_losing_only` constrains the SIGN of the exchange but never its
MAGNITUDE, so a large enough conductance removes more water than the reach
carries, the solve returns negative discharge, and the S28 clamp manufactures
mass to conceal it. That is what produced the NaN in 2026-09-17T15-47: 30.8%
of reaches driven negative against a 2-4% baseline, and a non-finite gradient
through that many saturated clamps on the first optimizer step.
Measured on the four-reach harness, sweeping conductance:
K_D alpha=0.9 total alpha=0.1 total
1e-4 42.50 42.50
1e-3 20.43 33.92
5e-3 0.004 33.92
At 5e-3 an effectively unbounded leakance drains the whole network to the
discharge floor. The bound holds it.
TWO THINGS THE TESTS CAUGHT, both of which would have been silent:
1. `b_rhs_base` is NOT non-negative. With `enforce_positivity: false` (the
default) the Cunge Muskingum coefficients go negative often, so a raw
`alpha*b_base` cap is negative, binds against a positive zeta, and rescales
the entire RHS by (1-alpha) — a mass bound turned into a silent 10x
attenuation. The `relu` is the physics: no water available, no loss
possible.
2. The forward splits `b_rhs` two ways, so the backward must split `gb_rhs`
two ways with DIFFERENT factors:
d(b_rhs)/d(b_base) = 1 - s*alpha where b_base > 0, else 1
d(b_rhs)/d(zeta) = -(1 - s)
Getting one right and the other wrong is undetectable from the loss. Both
factors are computed in the FORWARD and saved, so they cannot drift.
Off by default (`leakance_max_rhs_fraction: None`), so the DDR c2bd0f9
reference match and every prior run are byte-identical. Validator rejects it
without leakance and outside (0, 1).
Gates: --lib 383, compare_ddr_sandbox ABSOLUTE MATCH 1.525879e-4 (unchanged),
95 tests across the eleven routing and leakance suites.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0143bx7gVz8ug2oc6DTy5jtz
…nest Gaining reaches were clamped to zeta = 0 by `leakance_losing_only: true`, which every prior leakance run used. Measured against the independent water table on 63,365 CONUS reaches, using this model's own routed depths (p50 = 0.29 m): should be GAINING (table above stream stage): 63.7% should be losing: 36.3% So the clamp disabled the majority regime. It also revises a result previously attributed to physics: the KGE decomposition showed leakance only helps gauges whose inflow over-delivers, because a sink can only remove a surplus. Half of that was the flag, not the physics. Exchange direction is set by stream STAGE, not bed: a reach gains when `d_gw > depth`. The old d_gw ceiling of +1.0 m sat below the true water table on 45.3% of reaches, so the box is now [-2, 3] (80.4% of the field distribution, against 39.3% before). Centre +0.5 is connected, so d_gw carries gradient at epoch 1, and against a 0.29 m median depth it starts the model gaining, matching the dominant regime. THE MASS BOUND IS NOW SYMMETRIC, and that matters more than the loss side. An unbounded sink empties the channel, which is a numerical failure and shows up immediately. An unbounded SOURCE is a scientific failure and shows up as success: it can manufacture water wherever the model runs short, absorbing any inflow deficit and flattering the skill while meaning nothing. dHBV's q' already carries baseflow, which is the gaining contribution, so the right prior is that leakance is a CORRECTION on local flow, not a source: |zeta| <= alpha * relu(b_rhs_base), alpha = 0.25 Backward handles both arms: an upper bind gives `b_rhs = (1-alpha)*b_base` and a lower bind `(1+alpha)*b_base`, so the base factor is `1 - (s_up - s_dn)*alpha` where `b_base > 0`, and zeta's parents are gradient-free under either. What two-way buys scientifically is a SIGN constraint. The model must place the water table correctly relative to stream stage, not merely pick a plausible magnitude, which is a far sharper test of whether d_gw is physical than anything run so far. The read-out stays the partial correlation against the withheld water table, NOT skill. Five new tests, including a gradcheck through the gaining branch, which the losing_only gate meant had never been exercised in training. Gates: --lib 383, compare_ddr_sandbox ABSOLUTE MATCH 1.525879e-4 (unchanged), 99 tests across the eleven routing and leakance suites. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0143bx7gVz8ug2oc6DTy5jtz
…%, not 2-4% I wrote 2-4% into the config comments, src doc comments and the findings doc as the no-leakance baseline for `reaches with negative discharges pre clamp`. That figure was actually the rate of a leakance-ON run with a narrow K_D box. Read from six prior no-leakance CONUS runs, the true rate is 0.013% to 0.114%, typically 0.04-0.06%. The error made leakance's excess look ~50x smaller than it was, and it set the watch threshold 50x too loose, so a badly draining run would not have tripped it. With the correct baseline the two-way bounded run reads: 51.0% K_D ceiling 1e-3, no bound 30.8% same, later batches 1.0-4.1% narrow K_D box, still no bound 0.045% two-way + symmetric mass bound 0.043-0.056% no-leakance baseline i.e. the exchange now contributes no excess negative discharge at all. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0143bx7gVz8ug2oc6DTy5jtz
Registers what the running arm must achieve, written while it trains and before any number is available, because leakance has a documented history of absorbing bias while meaning nothing and two-way exchange makes that risk worse rather than better. Skill is explicitly NOT the read-out: a source term that can manufacture water wherever the model is short will improve NSE for uninteresting reasons. Baseline measured on the prior losing-only arm with the same script, 154,145 reaches, all ten head inputs as controls: magnitude, raw +0.212 magnitude, all controls +0.027 <- no independent signal sign agreement 60.8% majority-class baseline 63.6% <- worse than guessing regime partial correlation +0.011 Success requires the magnitude partial correlation above +0.10 AND the sign test beating the majority baseline with a regime partial correlation above +0.10. Also records four patterns that would make a PASS untrustworthy, and the caveat that the reference field is itself a model, not measurements. Adds experiments/leakance/validate_water_table.py, which makes last night's ad-hoc partial correlation reproducible and adds the sign test that only became possible once gaining reaches stopped being clamped to zero. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0143bx7gVz8ug2oc6DTy5jtz
…easured The landscape study refused any arm with `use_leakance: true`, inherited from `InfluenceContext::open` when leakance was declared out of scope in July. The guard turned out to be protecting something real: `landscape/objective.rs` hardcoded `k_d: None, d_gw: None, leakance_factor: None` when building `SpatialParameters`, so simply lifting it would have evaluated a NO-LEAKANCE model and reported the result as the leakance arm's landscape. A silent wrong answer is worse than a refusal. Fixed properly rather than bypassed: - `objective.rs` carries `K_D`/`d_gw`/`leakance_factor` at their trained fields through the existing `carried` mechanism, which already reads trained fields by name from the checkpoint. Carried, not swept: the question is where the roughness parameters sit with everything else held where the model is. - `param_range` learned those three names. - The guard is now opt-in. The ADJOINT study still refuses leakance, and deliberately: its functionals are gradients w.r.t. lateral inflow, and a zeta sink on the RHS is not accounted for in them. ALSO FIXES A THIRD COPY OF THE DENORMALIZE BUG. `objective.rs` had `let log_lo = (lo + 1e-6).ln()`, the same expression PR #46 fixed in `routing/utils.rs::log_space_lower` and `training/forward.rs`. It was harmless while `p_spatial` (lo = 1) was the only log-space axis, where the distortion is 1e-6, and severe for `K_D` (lo = 1e-8), where `lo + 1e-6` is 100x the bound and collapses the box. Two latent hazards had been cancelling: the guard kept K_D away from the bug. Now neither is latent. `log_space_lower` is `pub`. Journal: the two failed attempts are recorded with what each one taught, including that leaving `axes` at its default would have swept a PRESCRIBED p_spatial and reported it as an unidentifiable flat direction. Gates: --lib 383, compare_ddr_sandbox ABSOLUTE MATCH, ddr_sandbox_match, leakance_gamma_gradcheck 21, leakance_reference_match, leakance_off_parity. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0143bx7gVz8ug2oc6DTy5jtz
…s that use them `AXIS_PARAMS` was ["n","p_spatial","q_spatial","gamma"], so the leakance directions were invisible: a landscape of a leakance arm could only report where the ROUGHNESS parameters sat with the exchange parameters pinned, which is not an answer to whether leakance is identifiable. `leakance_factor` is deliberately NOT sweepable. It is the gate input, and the anneal already drives its gradient to zero once a reach commits (d(gate)/d(u) = 3.2e-8 at u = 0.1, tau = 0.1), so a swept curvature along it would report the schedule rather than the physics. Guard: a leakance axis on a non-leakance arm is rejected in the objective, where the arm's config is resolved, rather than in `validate_axes`, which runs before it. Carry loops skip anything being swept, so nothing is both. Three bundles, all reading `epoch_30_mb_9` of the two-way run so they describe one model state: landscape-twoway-gamma-wide alpha_max 4.605, find gamma's optimum landscape-twoway-leakance-axes axes [n, K_D, d_gw], dense 17^2 landscape-twoway-strat400 400-gauge census, axes [n, gamma, q_spatial] RESULTS from the first two, at the TRAINED point (hess0), Juniata pair: gamma: optimum pins at the wall at BOTH alpha_max 1.5 and 4.605, so it has no interior optimum at all. corr(n, gamma) = -0.947 / -0.732: n and gamma are near-perfectly anti-correlated in how they move the loss, which is §31's "one latent direction relabelled as two" measured as a near-singular Hessian. leakance: corr(n, d_gw) = +0.028 / +0.015, i.e. d_gw is essentially ORTHOGONAL to roughness and is not n relabelled. But K_D has NEGATIVE diagonal curvature (-1.28e-3 / -1.62e-3) and its optimum pins at the search ceiling, the same failure mode q_spatial showed in §29. The 18-30% loss gains come from riding K_D to a boundary, not from a determined value. Gates: --lib 383, ddr_sandbox_match, leakance_gamma_gradcheck 21, compare_ddr_sandbox ABSOLUTE MATCH. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0143bx7gVz8ug2oc6DTy5jtz
Records the registered failure (both pre-registered bars, the sign test worse than the one-way arm it was built to improve on), gamma having no optimum at any box width with corr(n,gamma) = -0.947, and the split leakance result: d_gw orthogonal to roughness, K_D with negative curvature and no interior minimum. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0143bx7gVz8ug2oc6DTy5jtz
Moved from /tmp into research/findings/ to match the existing convention (2026-06-06-gpu-device-config-handoff.md and four siblings). Records what was built (nine-parent op, disconnection cap, symmetric mass bound, two-way exchange, leakance axes in the landscape), that the CONUS arm finished clean at NSE 0.7300 / KGE 0.7567, and that BOTH pre-registered tests failed with the sign test worse than the one-way arm it was meant to improve. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0143bx7gVz8ug2oc6DTy5jtz
…s names `MeritGagesDataset::gauge_obs_std_required` computes the per-gauge observation std regardless of the arm's configured loss, sharing the one-shot cache with `gauge_obs_std`. The landscape objective uses it so a synthetic-n student trained with L1 can be scored with nse-batch at a gauge (before this the objective refused such arms). plot3d.py labels its axes from the file's `param_names` attribute and centres the planes at alpha = 0 when `slice_center: trained`, so the leakance-frame (n, K_D, d_gw) landscapes render with the right labels. Gate: cargo test --lib (383 passed), cargo test --test ddr_sandbox_match. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AhTffypSqtbq3BLkW1SQH1
…ero control Four landscape bundles over every gauged location in the Juniata basin: - landscape-jrb-all: axes (n, p, q) on uh-seed42, 8 gauges, 9 to 213 reaches. Stiff eigenvalue rises 30x with network size (Spearman +0.905); stiff direction (+0.8 n, -0.5 p) everywhere; flat direction q. - landscape-jrb-leakance: axes (n, K_D, d_gw) on the two-way arm, 10 gauges, planes through the trained point. K_D at the sweep ceiling at 8 of 10 with negative curvature; d_gw orthogonal to n. - landscape-jrb-zero / -zero-e20: the same cascade on student zero, a head trained on the teacher's own Q' and noise-free synthetic observations (configs synthetic_n_student_zero*.yaml). n sits at its per-gauge optimum but the CONUS level is 1.3x (epoch 5) to 1.6x (epoch 20) the truth, along an (n up, q up, p down) valley; the repeated alpha*_q values are the clamp wall. The pinned-geometry student (zero_pq) recovers the level (ratio 0.99) but against the consensus-geometry teacher. jrb_compare.py and jrb_leakance_compare.py produce the cross-gauge tables and panels. Journal carries the fd-step check (stiff eigenvalue converged to 2 %; flat eigenvalue is h^2 truncation, future bundles use 0.025) and the strat400 read. Outputs under output/jrb_landscapes*/ (gitignored). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AhTffypSqtbq3BLkW1SQH1
…apes experiments/landscape-huc-leakance: ten gauges in each of the 18 HUC2 regions (180, from the 2,365-gauge WY2000 census, HUC02 via the GAGES-II shapefile, 3 to 300 reaches, evenly spaced in network-size rank), axes (n, K_D, d_gw) on the two-way arm through the trained point. Launched 2026-09-22 ~02:00 UTC as 16 detached shards; huc_leakance_summary.py builds the per-region table and figure from the shard outputs. curvature3d.py renders, for one gauge file, the loss surface on each parameter plane and beneath it the local curvature of that surface (largest and smallest eigenvalue of the 2x2 Hessian differenced from the 25x25 loss grid), so where the bowl is sharp, flat, or a saddle is visible across the whole box rather than at one point. On the Juniata leakance files: roughness is 100 to 300x stiffer than K_D or d_gw at the trained point and the loss is a plateau in both leakance parameters over a factor of ten, bending only where both compound into a mass sink. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AhTffypSqtbq3BLkW1SQH1
180 gauges, 16 shards, 4 h. The Juniata "leakance is flat" read holds as a humid-region statement (n 7 to 400x stiffer than K_D or d_gw; 85 %+ of the K_D x d_gw box within 5 % of the trained loss) and is refuted in the plains (Upper Mississippi, Missouri, Souris-Red-Rainy: plateau 0.25 to 0.44, leakance alone reaches +0.03 to +0.15 NSE at well-fit gauges). The arid West is inconclusive: 6 to 9 of 10 gauges per arid region have a negative trained NSE, and their apparent leakance gains track the broken fit (Spearman -0.88), so curvature cannot be read there on this arm. huc_leakance_summary.py gains --min-nse0 to drop broken fits. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AhTffypSqtbq3BLkW1SQH1
Owner
Author
|
Regional leakance-frame curvature census landed: |
`landscape.additive_axes: {name: half_width}` sweeps a slot as
`x0 + alpha * half_width / alpha_max` (physical units) instead of
`x0 * exp(alpha)`. The alpha coordinate, grid, box, Newton search and
Hessian are untouched; only the field mapping in `Objective::fields_at`
and `forward_loss` changes, and the netCDF carries `additive_scale`
(units per alpha, 0 for a log slot) so readers can convert. Needed
because a multiplier can never change the sign of `d - d_gw`: the
regional census (2026-09-22) found every leakance gain at the
maximal-losing corner with the gaining regime untestable by construction.
Bundle `experiments/landscape-plains-dgw-additive/`: the 14 well-fit
plains gauges (HUC 07, 09, 10) of the census with `d_gw` swept +/- 2 m.
`plains_additive_summary.py` reads it against the census.
Gate: cargo test --lib experiment (78 passed, incl.
additive_axes_must_name_an_axis_with_positive_half_width).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AhTffypSqtbq3BLkW1SQH1
7.1 Additive d_gw sweep on the 14 well-fit plains gauges: leakance-only dNSE +0.090 vs +0.099 in the multiplicative census, minimum within 0.2 m of the trained water table at 10 of 14, raise:lower preference 10:4 with near-equal drops on both sides. The plains see the conductance magnitude, not the direction of exchange. 7.2 Arid West: the summed-Q' baseline is as broken as the trained model (median NSE -2.05). Volume-deficit subset (0.3 to 0.7) is the reservoir and diversion gauges; volume-excess subset (2 to 7,860, mostly reference class) is transmission loss the trained term cannot reach. Regulation is a subset explanation, not the population one. 7.3 Leakance leverage tracks the baseline volume error at Spearman +0.54 over 105 well-fit gauges; 36 of 38 "more losing" gauges have inflow above observed. The plains leverage is inflow-volume bias. Journal: entry for the plains run, ledger rows for its 14 shards and the dry run. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AhTffypSqtbq3BLkW1SQH1
Owner
Author
|
Follow-ups 1 to 3 worked (findings §7, commits |
…able term Offline fit of dMC-dev's `_first_order_euler_storage` at Raystown Dam with observed inflow (Saxton + local summed Q') and observed release. The law reaches NSE 0.18 train / 0.15 test at the box corner (theta 3, b 3) with an indefinite Hessian and no interior optimum, against pass-through 0.65 / 0.53 and a fitted 0.66-day linear reservoir 0.74 / 0.65. Capacity-as-S0 bounds the law's response time near 45 days while the observed release responds in under a day: model-class misspecification, not the sum-observation problem. Capacity sensitivity x0.5 / x2 does not change the ordering. Verdict: do not port the dMC reservoir operations; a reservoir term would need a release rule, not a fill-fraction law. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AhTffypSqtbq3BLkW1SQH1
Three checks after the flat hydrographs were questioned: the sandbox loop matches dMC-dev's `_first_order_euler_storage` verbatim to 2e-5 m3/s over 5,479 days; the step response e-folds in 93 to 509 days, consistent with the analytic linearised timescale; and a capacity sweep collapses the law onto pass-through (0.754 at 1 MCM vs 0.650) and then down to 0.175 at the real 940 MCM, monotone. The flatness is dMC's capacity-as-S0 design, not the implementation: the response floor S0/(b*theta*q_ref) is 67 days at the box corner, while the dam's daily buffer is ~2 MCM. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AhTffypSqtbq3BLkW1SQH1
…storage law by form Generic above/below dam sandbox plus a candidate search over the training population (67 dam-named arid gauges, 20 with an upstream gauge). At the two arid flood-control dams every model sits at NSE ~ 0 at every capacity: the release is zero for months and then a flat-topped evacuation (Alamo, ~180 m3/s) or a rectangular irrigation block (Santa Rosa), shapes a fill-fraction power law cannot produce. Abiquiu repeats Raystown: a 5.5-day linear reservoir (0.73 / 0.66) beats the law at any capacity. Do not port the law; prescribe observed release at dam rows, and drop the volume-deficit regulated gauges from training. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AhTffypSqtbq3BLkW1SQH1
…ion, literature) The linear reservoir in dam_sandbox.py was driven by the previous day's inflow; same-day it scores 0.800/0.732 at Raystown (was 0.737/0.650). It is exactly a Muskingum row with X = 0, K = T, so ddrs could host it by a coefficient override. Its main error is a release cap: min(S/T, Q_max) adds +0.13 test NSE at Raystown with observed inflow but only +0.02 with the trained model's inflow. Arid scheduled dams stay at NSE 0 under every storage-state rule. Population: 347 of 2,365 eval gauges have DOR > 0.5 (lower bound, NWM reservoir set), median NSE 0.495 vs 0.739 unregulated, 0.12-0.38 below in every area bin, mostly timing not volume. 216 have a gauge between them and their main reservoir. Literature (22 DOIs checked on Crossref): the fill-fraction law and level pool are lake laws; operating rules are not identifiable from downstream Q and the error lands in other parameters (Manning's n included). Options of record in the findings doc and .claude/RESERVOIRS.md: mask DOR > 0.5 gauges first (A/B), then one identity-row mechanism for observed releases and a linear/capped reservoir with prescribed Q_max. Corrects the arid doc's lagged linear column and its Abiquiu 17 % claim; adds §Reservoirs and do-not-use rows to research-status. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Option C of .claude/RESERVOIRS.md. MuskingumCunge::set_reservoir_rows(rows, t_days), called after setup_inputs, marks rows as linear reservoirs S = T*Q. forward_chain_inner (S19'') then replaces the Muskingum K with T*86400 s and X with 0 on those rows, after the S18'/S19' positivity clamps and before the coefficients, x_eff_out and the saved K. timestep_backward_core (B19'') zeroes the gradient into X before the Cunge/cap split and into K after the gk_from_x_cap fold, so a dam row's n, q_spatial and p_spatial get exactly 0. mask_where / mask_fill copy off-mask elements unchanged, and None adds no tensor op: compare_ddr_sandbox stays ABSOLUTE MATCH with a byte-identical diff CSV. Validation rejects length mismatch, out-of-range and duplicate rows, and non-finite or sub-hour T. route_timestep panics if rows are set on the leakance or CUDA-graph path; Task 2 rejects both at config load. tests/reservoir_override.rs: linear-reservoir recurrence (worst rel 1.2e-7) with the upstream reach bit-identical, disabled == never-set bitwise, mass balance after 2,000 steps, FD gradcheck of n and q_spatial on the 5-reach sandbox topology, exact-zero dam-row gradients with and without enforce_positivity, input validation, and the leakance panic. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015TepsZAS2LkFC5EmvuZrz9
…on C) Makes Task 1's linear-reservoir override usable from config and data. params.use_reservoirs (default false) plus data_sources.reservoirs, a CSV with header COMID,T_days (extra columns ignored), route the table's dam reaches as S = T*Q in every train, eval and train-and-test forward. - src/data/store/reservoirs.rs: read_reservoir_table (DataError naming the path on a missing column, unparseable row, non-finite or sub-hour T_days, duplicate COMID, empty table) and reservoir_rows, the single COMID -> row mapping onto a network's divide_comids order. - MeritGagesDataset reads the table at open; collate and the eval static network map it and carry RoutingBatch/RoutingTensors::reservoir_rows. training::forward::apply_reservoir_rows arms them right after setup_inputs at all four engine sites (forward, forward_eval_core, forward_with_frozen_params, probe_forward). The eval network logs "reservoirs: <k> of <m> table COMIDs are in the network" once. - validate_reservoirs rejects use_reservoirs with no table, or with use_leakance, use_cuda_graphs, ddr_match, subdivision or gridded_network. Paper studies refuse reservoir arms. - plan.rs and sources.rs fingerprint the table into sources.lock. - MuskingumCunge::setup_inputs clears reservoir rows of the previous network. - Fixture examples/juniata/data/juniata_reservoirs.csv (Raystown Lake, COMID 73005301, T 1.23 d). The release Juniata test runs the bundle with and without it: 1 of 1 matched, all 5,477 gauge days differ. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015TepsZAS2LkFC5EmvuZrz9
…be, docs) - testing.md: reservoir gates in Tier A/C, the reservoir e2e test in the Juniata acceptance section, a Reservoirs row in "What covers what". - dataset open logs `reservoirs: table <path> has <m> COMIDs` (fd 2, like the match line), so train-only runs carry the table in run.log; the e2e test asserts it. - probe_courant refuses use_reservoirs: it drives forward_chain_inner with no reservoir override, so applying rows to the engine would be a no-op. - reservoir_override test 1 ramps the dam's lateral inflow, pinning the row t-1 convention in the recurrence check. - juniata_acceptance header timing (~53 s, three trainings); CLAUDE.md pointer to RESERVOIRS.md and config.md §Reservoirs. - traps.md T18: training_step_layer_b step3 / layer_c step2 fail because fixture_config routes pre-#192 fixtures on the default ddr_match: false; verified by rerunning with ddr_match: true (all pass). Tests unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015TepsZAS2LkFC5EmvuZrz9
Option C (linear reservoir via K := T, X := 0 on dam rows) is implemented and off by default; research-status and the options doc now say so, and record where ResOpsUS, ISTARF-CONUS and the GRanD to MERIT COMID crosswalk live (/mnt/ssd1/data/resops/) with the 298 / 289 inflow-outflow coverage count that a per-dam T fit would use. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015TepsZAS2LkFC5EmvuZrz9
… pairs) GRanD dams of the NWM table paired with the nearest eval gauge below them (drainage area within 1.5x the dam watershed), required to carry NWIS peak code 6 in WY1996-2010, ResOpsUS dams first then even drainage-area rank. Nine HUC2 regions reach ten; short regions keep all and report n. Current trained median NSE 0.458 [0.387, 0.508] vs summed Q' 0.371; regional medians at n = 10 carry 0.2 to 0.6 NSE of bootstrap uncertainty, so comparisons should be paired per gauge. Each dam is tagged with its release data (ResOpsUS coverage, ISTARF fit, GloFAS Qf, GRanD purpose) and USGS regulation evidence (peak codes, GAGES-II dams). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015TepsZAS2LkFC5EmvuZrz9
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Work-in-progress branch, opened as a draft to save state. 23 commits over master.
Physics and infrastructure
TimestepLeakanceGammaOpso leakance and a learned stage-roughness exponent gamma can coexist; gradcheck intests/leakance_gamma_gradcheck.rs.leakance_bed_thickness), symmetric mass bound (leakance_max_rhs_fraction), two-way exchange (leakance_losing_only: false),K_Dbox fixed to[1e-9, 1e-5].K_Dandd_gware sweepable axes;nse-batchcan be scored on L1-trained arms (gauge_obs_std_required).Results recorded on the branch
2026-09-17T16-38-16Z: NSE 0.7300 / KGE 0.7567 vs baseline 0.6781 / 0.7172. Both pre-registered water-table tests failed (research/findings/2026-09-18-two-way-leakance-handoff.md).fd_step: 0.025.In flight
experiments/landscape-huc-leakance: ten gauges per HUC2 region (180) in the leakance frame, launched 2026-09-22 as 16 shards. Results and a findings doc to follow on this branch.Test plan
cargo test --lib(383 passed) andcargo test --test ddr_sandbox_matchafter theobjective.rs/dataset.rschangetests/leakance_gamma_gradcheck.rs,tests/leakance_reference_match.rson the physics commits (per their commit messages)cargo testandcompare_ddr_sandboxbefore undrafting🤖 Generated with Claude Code
https://claude.ai/code/session_01AhTffypSqtbq3BLkW1SQH1