Repository navigation
Learned dam release: per-dam calibration, additive dam row, rule curve, water accounting - #51
Merged
Merged
Conversation
zeta multiplies leakance_factor by K_D, so only their product reaches the physics: an exact scaling degeneracy. Measured over 346,321 CONUS reaches on a trained arm, rho(leakance_factor, K_D) = +0.9986 and all four learned outputs of that head hold 97 percent of their variance in one direction. Physically a reach either sits above a losing aquifer or it does not, so the factor should select where leakance acts rather than scale how much. A gate and a conductance are not degenerate with each other. The gate is a sharpening sigmoid on the head's normalized output, g = sigmoid(logit(clamp(u, eps, 1-eps)) / tau), annealed by an epoch-keyed schedule that mirrors the learning rate. tau = 1 is special-cased to return u untouched, so every config without params.leakance_gate is bit-exact unchanged. Differentiable at every positive tau, so autograd gives the exact gradient and no straight-through estimator is introduced. Applied to the head output before setup_inputs, so no routing operator or backward changes. Replicated across all three hand-written head readers rather than unified, per the WET preference, with the eval reader and dump_parameters using the FINAL temperature so the scored model is the one training ended on. A previous bug shipped because an eval reader silently dropped a learned head output. Gates: lib 377 passed; leakance_gate 11; gamma_eval_parity 4; sp8_gradcheck 20; leakance_gradcheck 16; leakance_off_parity 3; zeta_accum 11; leakance_reference_match 5; ddr_sandbox_match 1. compare_ddr_sandbox ABSOLUTE MATCH at 1.525879e-4 m3/s, unchanged. verify_doc_paths 0 unresolved. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0143bx7gVz8ug2oc6DTy5jtz
104 numbered equations over 13 sections, each derived from the code rather than from prior docs, with symbol tables giving units and whether each quantity is learned per reach, set by config, read from data, or derived. Covers the head and its normalization, the new gate, the geometry chain with a catalogue of every clamp and what it does to the gradient, stage-dependent roughness, the Muskingum coefficients, the sparse adjoint, the leakance flux with all six partials, the gradient-path labels, all five losses, and the diagnostics. It found ten code-versus-documentation discrepancies. The load-bearing one: the leakance term computes its width as (p*d)^q while the routing geometry computes the same reach's width as p*d^q, so the area multiplying K_D is smaller by the depth-independent factor p^(q-1) = 0.3445 at the prescribed p = 21, q = 0.65, and a learned K_D absorbs it. The flux and skill results are unaffected because a constant rescaling is absorbable, but K_D is not directly comparable to a measured streambed leakance without dividing it out. Inherited from DDR, so the port is faithful; leakance_reference_match pins that. validate_params.py now divides the factor out and records the caveat. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0143bx7gVz8ug2oc6DTy5jtz
…gradient-sink check The equations reference flags that nothing in the repo measures how often the geometry clamps saturate, which matters because a saturated clamp zeroes the gradient to the parameter behind it, so training cannot identify that parameter at that reach. Measured on the 2026-09-16T04-44-15Z arm over 64,892 eval reaches: the depth floor takes 2.1 percent, and the side-slope bounds, the bottom-width floor and both roughness range bounds take none. The side-slope clamp cannot fire in this configuration: with q fixed at 0.65 its lower bound needs a depth of 1,751 m and its upper bound a depth of 3.2 mm, which the 10 mm depth floor already forbids. So clamps are not the reason the learned outputs collapse onto one direction. That is consistent with the head-topology screen, which found capacity is not the constraint either, leaving the gradient and the objective. Figures are a lower bound: computed from each reach's mean eval depth, so a clamp that fires only at low flow is invisible. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0143bx7gVz8ug2oc6DTy5jtz
… a decision Three plotting faults, all of which misled the reader. A prescribed constant is not a learned field. dump_parameters writes every parameter the run used, learned or not, so the width exponent and coefficient were being mapped as if the head had produced them when both come from params.defaults. They are now detected by zero variance, reported as constants in the caption line, and left off the figure. A gate is a decision, not a magnitude. Percentile colour limits rescaled a field that never leaves 0.96 to 1.0 into a full rainbow, which made a gate that is ON everywhere look richly structured. The gate now uses a fixed 0 to 1 scale and the panel title states the fraction actually switched off, which is 0.0 percent. The roughness label depends on the arm. With a learned stage exponent n is n_0, roughness at the reference depth; without one it is Manning's n. The title now says which, since config rejects a learned gamma alongside leakance so these arms are always the latter. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0143bx7gVz8ug2oc6DTy5jtz
Equal flux requires K_phys * A_phys = K_D * A_ours, so K_phys = K_D * p^(q-1) = K_D * 0.3445. Our area is SMALLER than the physical one, so the physically equivalent conductivity is SMALLER than the learned number. I had it inverted, which overstated the implied streambed conductivity by 2.9x and put the gated arm at the 52nd percentile of the measured range when it is at the 13th. Also reports the distance of the learned median from the box centre in decades, which is the test that matters: a sigmoid head with near-zero output bias reports an unconstrained parameter at its box centre, so proximity to the centre is evidence the data did not move it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0143bx7gVz8ug2oc6DTy5jtz
Before this, `TimestepLeakanceOp` had eight parents and none of them was gamma, so a learned stage-roughness exponent would have silently received no gradient while leakance was on. `validate_learned_gamma` rejected the combination at config load rather than let that happen, which made the two features mutually exclusive. `TimestepLeakanceGammaOp` is the nine-parent sibling, following the same pattern `TimestepGammaOp` already uses for gamma without leakance. The backward math is shared through the new `leakance_backward_body`, so the two leakance ops differ only in parent bookkeeping and the eight-parent path is untouched. Gamma and zeta couple through the geometry accumulators that `timestep_backward_core` already shares: gamma re-weights the chain below depth (B5/B15/B17) while zeta reads depth, p_spatial and q_eps and is folded into b_rhs (B27). No new cross-term is needed, which is a claim, so `tests/leakance_gamma_gradcheck.rs` measures it: all nine parents swept against central differences with both features active (worst rel 9.3e-4), plus two identities — a learned gamma of zero is bit-identical to plain leakance, and a learned constant gamma matches the global stage-roughness path to 1e-5. Also restores the gitignored .cargo/config.toml CUDARC_CUDA_VERSION override lost when the build directory was cleared (trap T12). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0143bx7gVz8ug2oc6DTy5jtz
…ked for `sr_n0_gamma_leakance.yaml` is the first configuration in which both mechanisms that can absorb depth-dependent volume error are allowed to compete: leakance removes water, gamma changes how fast water moves at a given stage. Every prior leakance run used a depth-independent n, so the comparison was never possible. Three boxes widened, from research/findings/2026-09-17-parameter-range-audit.md: gamma [0, 0.5] -> [-0.2, 0.85] K_D [1e-8, 1e-6] -> [1e-8, 1e-3] d_gw [-2, 2] -> [-5, 1] The gamma floor is the substantive one. Zero encoded "channels get smoother as they fill", which holds in-bank and fails on a vegetated floodplain or any composite section, where effective roughness rises with stage. Those reaches were not discouraged, they were unrepresentable. `validate_learned_gamma` now admits [-0.5, 1]; the guard floor is far from the only real singularity, at gamma = -(5+3q)/3 (about -2.3 at q = 0.65). Both signs are gradient-checked. The default gamma box is deliberately NOT changed, so no existing config moves; the widened box is opt-in per experiment. lr is five times cooler than the no-leakance arms. A CONUS full-recipe leakance run at 0.005 diverged to NaN at epoch 15 with a finite loss and a non-finite gradient, and 5x lower removed it. Gates: ddr_sandbox_match + compare_ddr_sandbox ABSOLUTE MATCH (1.53e-4, the same value as before this branch), --lib 373, and 91 tests across mmc, sparse_gradcheck, sp8_gradcheck, stage_roughness, the four leakance suites, gamma_eval_parity and leakance_reference_match. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0143bx7gVz8ug2oc6DTy5jtz
head = min(depth - d_gw, depth + M) Once the water table falls more than the streambed thickness M below the bed, an unsaturated zone opens beneath the channel, stream and aquifer stop being hydraulically connected, and the flux is set by the head across the bed layer alone rather than by the distance down to the table. The previous linear head had no such saturation. This is not a refinement, it is the difference between a d_gw that can be learned and one that cannot. 9.3% of CONUS reaches have the table more than 5 m below the bed (p95 14 m, p99 71 m, max 148 m), so representing them needs a deep box; but under a linear head a deep box is actively harmful. At depth 1.5 m a d_gw of -80 m gives 12x the flux of -5 m for the same conductance, and a large head with a small K_D is indistinguishable from a small head with a large one. Widening the box without the cap buys degeneracy, not coverage. With the cap, every state below -M is the same physics, so the box does not need to be deep at all. What it needs instead is to CENTRE in the connected region: connection is exactly `d_gw > -M` (depth cancels from both sides), and the d_gw gradient is identically zero once disconnected. A box centred below -M would have given d_gw no gradient at epoch 1 and it would never have moved. Hence M = 1.0 with d_gw in [-2, 1]: centre -0.5, connected, two thirds of the box gradient-bearing. `leakance_bed_thickness` is absent by default, and absent means no cap and a byte-identical forward, so the DDR c2bd0f9 reference match and every prior leakance run stay valid. The eval zeta diagnostic recomputes the head independently and was updated in step, or it would have reported a flux the routing never applied. Six new unit tests pin the semantics: back-compat when absent, flux independent of d_gw once disconnected, unbounded growth WITHOUT the cap (so a silent disable fails loudly), connection independent of depth, the vanishing d_gw gradient, and central differences through the capped forward in both regimes. Gates: --lib 383, compare_ddr_sandbox ABSOLUTE MATCH 1.525879e-4 (unchanged), 91 tests across the eleven routing and leakance suites. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0143bx7gVz8ug2oc6DTy5jtz
Attribution needs all four cells of {leakance on/off} x {n(d) on/off}, and the
existing sr_n0_only.yaml and sr_n0_gamma.yaml cannot serve: they run the hot
learning rate (0.005 vs 0.001) and the old gamma box [0, 0.5], so they differ
from the combined arm in three ways at once, not one.
Each control is derived from sr_n0_gamma_leakance.yaml by removing exactly one
mechanism. Same gauges, seed, windows, head, schedule, and identical boxes for
every parameter the two arms share.
ctl_neither n_0 alone the floor
ctl_gamma n_0 + gamma n(d) only
ctl_leakance n_0 + K_D/d_gw/factor the cell every prior
leakance run occupied
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0143bx7gVz8ug2oc6DTy5jtz
…he init The 2026-09-17T15-47 CONUS run went NaN on the FIRST optimizer step. Loss was finite through all four micro-batches of mb=0 (0.51, 0.40, 0.42, 0.81) and every value after the step was NaN, so the forward was fine and the gradient was not. Cause, measured by re-running with the K_D box as the only difference, same seed: K_D [1e-8, 1e-6] centre 1e-7 loss 0.149 2.9% reaches negative pre-clamp K_D [1e-8, 1e-3] centre 3.16e-6 loss 0.513 30.8% reaches negative pre-clamp The head is a sigmoid with near-zero output bias, so every parameter starts at its box centre, and for a log-space parameter that is the GEOMETRIC mean. Widening the ceiling 1000x moved the starting conductance 31.6x. zeta is linear in K_D, so leakance drained a third of the network below zero at epoch zero, the S28 clamp rewrote all of them, and the gradient through that many saturated clamps went non-finite immediately. This is the same mistake as the d_gw box: widening a box does not just add reach, it relocates where training begins. [1e-9, 1e-5] has its geometric centre at exactly 1e-7, the historically stable init, while spanning four decades and covering the measured streambed range (8.8e-8 to 2.1e-5 1/s). The same diagnostic run carried the widened gamma box AND the disconnection cap and was stable through the optimizer step, so both are exonerated. It also exposed a real flaw, previously masked because the denormalize bug kept K_D frozen at a tiny constant in every leakance run ever executed: NOTHING BOUNDS ZETA BY THE WATER IN THE REACH. `losing_only` fixes the sign of the exchange, not its magnitude, so a large enough conductance removes more water than exists. Recorded in the config; `reaches with negative discharges pre clamp` is the instrument (no-leakance rate is 2-4%). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0143bx7gVz8ug2oc6DTy5jtz
Records the single-variable measurement behind e2f7b96: a K_D box widened to [1e-8, 1e-3] moved the geometric init 31.6x, drove 30.8% of reaches negative pre-clamp against a 2-4% baseline, and went non-finite on the first optimizer step with a finite forward. Also records the flaw that exposed: nothing bounds zeta by the water in the reach. losing_only fixes the sign of the exchange, not the magnitude. Masked for the whole history of the feature by the denormalize bug that froze K_D. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0143bx7gVz8ug2oc6DTy5jtz
zeta <- min(zeta, alpha * relu(b_rhs_base))
`leakance_losing_only` constrains the SIGN of the exchange but never its
MAGNITUDE, so a large enough conductance removes more water than the reach
carries, the solve returns negative discharge, and the S28 clamp manufactures
mass to conceal it. That is what produced the NaN in 2026-09-17T15-47: 30.8%
of reaches driven negative against a 2-4% baseline, and a non-finite gradient
through that many saturated clamps on the first optimizer step.
Measured on the four-reach harness, sweeping conductance:
K_D alpha=0.9 total alpha=0.1 total
1e-4 42.50 42.50
1e-3 20.43 33.92
5e-3 0.004 33.92
At 5e-3 an effectively unbounded leakance drains the whole network to the
discharge floor. The bound holds it.
TWO THINGS THE TESTS CAUGHT, both of which would have been silent:
1. `b_rhs_base` is NOT non-negative. With `enforce_positivity: false` (the
default) the Cunge Muskingum coefficients go negative often, so a raw
`alpha*b_base` cap is negative, binds against a positive zeta, and rescales
the entire RHS by (1-alpha) — a mass bound turned into a silent 10x
attenuation. The `relu` is the physics: no water available, no loss
possible.
2. The forward splits `b_rhs` two ways, so the backward must split `gb_rhs`
two ways with DIFFERENT factors:
d(b_rhs)/d(b_base) = 1 - s*alpha where b_base > 0, else 1
d(b_rhs)/d(zeta) = -(1 - s)
Getting one right and the other wrong is undetectable from the loss. Both
factors are computed in the FORWARD and saved, so they cannot drift.
Off by default (`leakance_max_rhs_fraction: None`), so the DDR c2bd0f9
reference match and every prior run are byte-identical. Validator rejects it
without leakance and outside (0, 1).
Gates: --lib 383, compare_ddr_sandbox ABSOLUTE MATCH 1.525879e-4 (unchanged),
95 tests across the eleven routing and leakance suites.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0143bx7gVz8ug2oc6DTy5jtz
…nest Gaining reaches were clamped to zeta = 0 by `leakance_losing_only: true`, which every prior leakance run used. Measured against the independent water table on 63,365 CONUS reaches, using this model's own routed depths (p50 = 0.29 m): should be GAINING (table above stream stage): 63.7% should be losing: 36.3% So the clamp disabled the majority regime. It also revises a result previously attributed to physics: the KGE decomposition showed leakance only helps gauges whose inflow over-delivers, because a sink can only remove a surplus. Half of that was the flag, not the physics. Exchange direction is set by stream STAGE, not bed: a reach gains when `d_gw > depth`. The old d_gw ceiling of +1.0 m sat below the true water table on 45.3% of reaches, so the box is now [-2, 3] (80.4% of the field distribution, against 39.3% before). Centre +0.5 is connected, so d_gw carries gradient at epoch 1, and against a 0.29 m median depth it starts the model gaining, matching the dominant regime. THE MASS BOUND IS NOW SYMMETRIC, and that matters more than the loss side. An unbounded sink empties the channel, which is a numerical failure and shows up immediately. An unbounded SOURCE is a scientific failure and shows up as success: it can manufacture water wherever the model runs short, absorbing any inflow deficit and flattering the skill while meaning nothing. dHBV's q' already carries baseflow, which is the gaining contribution, so the right prior is that leakance is a CORRECTION on local flow, not a source: |zeta| <= alpha * relu(b_rhs_base), alpha = 0.25 Backward handles both arms: an upper bind gives `b_rhs = (1-alpha)*b_base` and a lower bind `(1+alpha)*b_base`, so the base factor is `1 - (s_up - s_dn)*alpha` where `b_base > 0`, and zeta's parents are gradient-free under either. What two-way buys scientifically is a SIGN constraint. The model must place the water table correctly relative to stream stage, not merely pick a plausible magnitude, which is a far sharper test of whether d_gw is physical than anything run so far. The read-out stays the partial correlation against the withheld water table, NOT skill. Five new tests, including a gradcheck through the gaining branch, which the losing_only gate meant had never been exercised in training. Gates: --lib 383, compare_ddr_sandbox ABSOLUTE MATCH 1.525879e-4 (unchanged), 99 tests across the eleven routing and leakance suites. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0143bx7gVz8ug2oc6DTy5jtz
…%, not 2-4% I wrote 2-4% into the config comments, src doc comments and the findings doc as the no-leakance baseline for `reaches with negative discharges pre clamp`. That figure was actually the rate of a leakance-ON run with a narrow K_D box. Read from six prior no-leakance CONUS runs, the true rate is 0.013% to 0.114%, typically 0.04-0.06%. The error made leakance's excess look ~50x smaller than it was, and it set the watch threshold 50x too loose, so a badly draining run would not have tripped it. With the correct baseline the two-way bounded run reads: 51.0% K_D ceiling 1e-3, no bound 30.8% same, later batches 1.0-4.1% narrow K_D box, still no bound 0.045% two-way + symmetric mass bound 0.043-0.056% no-leakance baseline i.e. the exchange now contributes no excess negative discharge at all. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0143bx7gVz8ug2oc6DTy5jtz
Registers what the running arm must achieve, written while it trains and before any number is available, because leakance has a documented history of absorbing bias while meaning nothing and two-way exchange makes that risk worse rather than better. Skill is explicitly NOT the read-out: a source term that can manufacture water wherever the model is short will improve NSE for uninteresting reasons. Baseline measured on the prior losing-only arm with the same script, 154,145 reaches, all ten head inputs as controls: magnitude, raw +0.212 magnitude, all controls +0.027 <- no independent signal sign agreement 60.8% majority-class baseline 63.6% <- worse than guessing regime partial correlation +0.011 Success requires the magnitude partial correlation above +0.10 AND the sign test beating the majority baseline with a regime partial correlation above +0.10. Also records four patterns that would make a PASS untrustworthy, and the caveat that the reference field is itself a model, not measurements. Adds experiments/leakance/validate_water_table.py, which makes last night's ad-hoc partial correlation reproducible and adds the sign test that only became possible once gaining reaches stopped being clamped to zero. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0143bx7gVz8ug2oc6DTy5jtz
…easured The landscape study refused any arm with `use_leakance: true`, inherited from `InfluenceContext::open` when leakance was declared out of scope in July. The guard turned out to be protecting something real: `landscape/objective.rs` hardcoded `k_d: None, d_gw: None, leakance_factor: None` when building `SpatialParameters`, so simply lifting it would have evaluated a NO-LEAKANCE model and reported the result as the leakance arm's landscape. A silent wrong answer is worse than a refusal. Fixed properly rather than bypassed: - `objective.rs` carries `K_D`/`d_gw`/`leakance_factor` at their trained fields through the existing `carried` mechanism, which already reads trained fields by name from the checkpoint. Carried, not swept: the question is where the roughness parameters sit with everything else held where the model is. - `param_range` learned those three names. - The guard is now opt-in. The ADJOINT study still refuses leakance, and deliberately: its functionals are gradients w.r.t. lateral inflow, and a zeta sink on the RHS is not accounted for in them. ALSO FIXES A THIRD COPY OF THE DENORMALIZE BUG. `objective.rs` had `let log_lo = (lo + 1e-6).ln()`, the same expression PR #46 fixed in `routing/utils.rs::log_space_lower` and `training/forward.rs`. It was harmless while `p_spatial` (lo = 1) was the only log-space axis, where the distortion is 1e-6, and severe for `K_D` (lo = 1e-8), where `lo + 1e-6` is 100x the bound and collapses the box. Two latent hazards had been cancelling: the guard kept K_D away from the bug. Now neither is latent. `log_space_lower` is `pub`. Journal: the two failed attempts are recorded with what each one taught, including that leaving `axes` at its default would have swept a PRESCRIBED p_spatial and reported it as an unidentifiable flat direction. Gates: --lib 383, compare_ddr_sandbox ABSOLUTE MATCH, ddr_sandbox_match, leakance_gamma_gradcheck 21, leakance_reference_match, leakance_off_parity. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0143bx7gVz8ug2oc6DTy5jtz
…s that use them `AXIS_PARAMS` was ["n","p_spatial","q_spatial","gamma"], so the leakance directions were invisible: a landscape of a leakance arm could only report where the ROUGHNESS parameters sat with the exchange parameters pinned, which is not an answer to whether leakance is identifiable. `leakance_factor` is deliberately NOT sweepable. It is the gate input, and the anneal already drives its gradient to zero once a reach commits (d(gate)/d(u) = 3.2e-8 at u = 0.1, tau = 0.1), so a swept curvature along it would report the schedule rather than the physics. Guard: a leakance axis on a non-leakance arm is rejected in the objective, where the arm's config is resolved, rather than in `validate_axes`, which runs before it. Carry loops skip anything being swept, so nothing is both. Three bundles, all reading `epoch_30_mb_9` of the two-way run so they describe one model state: landscape-twoway-gamma-wide alpha_max 4.605, find gamma's optimum landscape-twoway-leakance-axes axes [n, K_D, d_gw], dense 17^2 landscape-twoway-strat400 400-gauge census, axes [n, gamma, q_spatial] RESULTS from the first two, at the TRAINED point (hess0), Juniata pair: gamma: optimum pins at the wall at BOTH alpha_max 1.5 and 4.605, so it has no interior optimum at all. corr(n, gamma) = -0.947 / -0.732: n and gamma are near-perfectly anti-correlated in how they move the loss, which is §31's "one latent direction relabelled as two" measured as a near-singular Hessian. leakance: corr(n, d_gw) = +0.028 / +0.015, i.e. d_gw is essentially ORTHOGONAL to roughness and is not n relabelled. But K_D has NEGATIVE diagonal curvature (-1.28e-3 / -1.62e-3) and its optimum pins at the search ceiling, the same failure mode q_spatial showed in §29. The 18-30% loss gains come from riding K_D to a boundary, not from a determined value. Gates: --lib 383, ddr_sandbox_match, leakance_gamma_gradcheck 21, compare_ddr_sandbox ABSOLUTE MATCH. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0143bx7gVz8ug2oc6DTy5jtz
Records the registered failure (both pre-registered bars, the sign test worse than the one-way arm it was built to improve on), gamma having no optimum at any box width with corr(n,gamma) = -0.947, and the split leakance result: d_gw orthogonal to roughness, K_D with negative curvature and no interior minimum. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0143bx7gVz8ug2oc6DTy5jtz
Moved from /tmp into research/findings/ to match the existing convention (2026-06-06-gpu-device-config-handoff.md and four siblings). Records what was built (nine-parent op, disconnection cap, symmetric mass bound, two-way exchange, leakance axes in the landscape), that the CONUS arm finished clean at NSE 0.7300 / KGE 0.7567, and that BOTH pre-registered tests failed with the sign test worse than the one-way arm it was meant to improve. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0143bx7gVz8ug2oc6DTy5jtz
…s names `MeritGagesDataset::gauge_obs_std_required` computes the per-gauge observation std regardless of the arm's configured loss, sharing the one-shot cache with `gauge_obs_std`. The landscape objective uses it so a synthetic-n student trained with L1 can be scored with nse-batch at a gauge (before this the objective refused such arms). plot3d.py labels its axes from the file's `param_names` attribute and centres the planes at alpha = 0 when `slice_center: trained`, so the leakance-frame (n, K_D, d_gw) landscapes render with the right labels. Gate: cargo test --lib (383 passed), cargo test --test ddr_sandbox_match. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AhTffypSqtbq3BLkW1SQH1
…ero control Four landscape bundles over every gauged location in the Juniata basin: - landscape-jrb-all: axes (n, p, q) on uh-seed42, 8 gauges, 9 to 213 reaches. Stiff eigenvalue rises 30x with network size (Spearman +0.905); stiff direction (+0.8 n, -0.5 p) everywhere; flat direction q. - landscape-jrb-leakance: axes (n, K_D, d_gw) on the two-way arm, 10 gauges, planes through the trained point. K_D at the sweep ceiling at 8 of 10 with negative curvature; d_gw orthogonal to n. - landscape-jrb-zero / -zero-e20: the same cascade on student zero, a head trained on the teacher's own Q' and noise-free synthetic observations (configs synthetic_n_student_zero*.yaml). n sits at its per-gauge optimum but the CONUS level is 1.3x (epoch 5) to 1.6x (epoch 20) the truth, along an (n up, q up, p down) valley; the repeated alpha*_q values are the clamp wall. The pinned-geometry student (zero_pq) recovers the level (ratio 0.99) but against the consensus-geometry teacher. jrb_compare.py and jrb_leakance_compare.py produce the cross-gauge tables and panels. Journal carries the fd-step check (stiff eigenvalue converged to 2 %; flat eigenvalue is h^2 truncation, future bundles use 0.025) and the strat400 read. Outputs under output/jrb_landscapes*/ (gitignored). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AhTffypSqtbq3BLkW1SQH1
…apes experiments/landscape-huc-leakance: ten gauges in each of the 18 HUC2 regions (180, from the 2,365-gauge WY2000 census, HUC02 via the GAGES-II shapefile, 3 to 300 reaches, evenly spaced in network-size rank), axes (n, K_D, d_gw) on the two-way arm through the trained point. Launched 2026-09-22 ~02:00 UTC as 16 detached shards; huc_leakance_summary.py builds the per-region table and figure from the shard outputs. curvature3d.py renders, for one gauge file, the loss surface on each parameter plane and beneath it the local curvature of that surface (largest and smallest eigenvalue of the 2x2 Hessian differenced from the 25x25 loss grid), so where the bowl is sharp, flat, or a saddle is visible across the whole box rather than at one point. On the Juniata leakance files: roughness is 100 to 300x stiffer than K_D or d_gw at the trained point and the loss is a plateau in both leakance parameters over a factor of ten, bending only where both compound into a mass sink. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AhTffypSqtbq3BLkW1SQH1
180 gauges, 16 shards, 4 h. The Juniata "leakance is flat" read holds as a humid-region statement (n 7 to 400x stiffer than K_D or d_gw; 85 %+ of the K_D x d_gw box within 5 % of the trained loss) and is refuted in the plains (Upper Mississippi, Missouri, Souris-Red-Rainy: plateau 0.25 to 0.44, leakance alone reaches +0.03 to +0.15 NSE at well-fit gauges). The arid West is inconclusive: 6 to 9 of 10 gauges per arid region have a negative trained NSE, and their apparent leakance gains track the broken fit (Spearman -0.88), so curvature cannot be read there on this arm. huc_leakance_summary.py gains --min-nse0 to drop broken fits. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AhTffypSqtbq3BLkW1SQH1
`landscape.additive_axes: {name: half_width}` sweeps a slot as
`x0 + alpha * half_width / alpha_max` (physical units) instead of
`x0 * exp(alpha)`. The alpha coordinate, grid, box, Newton search and
Hessian are untouched; only the field mapping in `Objective::fields_at`
and `forward_loss` changes, and the netCDF carries `additive_scale`
(units per alpha, 0 for a log slot) so readers can convert. Needed
because a multiplier can never change the sign of `d - d_gw`: the
regional census (2026-09-22) found every leakance gain at the
maximal-losing corner with the gaining regime untestable by construction.
Bundle `experiments/landscape-plains-dgw-additive/`: the 14 well-fit
plains gauges (HUC 07, 09, 10) of the census with `d_gw` swept +/- 2 m.
`plains_additive_summary.py` reads it against the census.
Gate: cargo test --lib experiment (78 passed, incl.
additive_axes_must_name_an_axis_with_positive_half_width).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AhTffypSqtbq3BLkW1SQH1
7.1 Additive d_gw sweep on the 14 well-fit plains gauges: leakance-only dNSE +0.090 vs +0.099 in the multiplicative census, minimum within 0.2 m of the trained water table at 10 of 14, raise:lower preference 10:4 with near-equal drops on both sides. The plains see the conductance magnitude, not the direction of exchange. 7.2 Arid West: the summed-Q' baseline is as broken as the trained model (median NSE -2.05). Volume-deficit subset (0.3 to 0.7) is the reservoir and diversion gauges; volume-excess subset (2 to 7,860, mostly reference class) is transmission loss the trained term cannot reach. Regulation is a subset explanation, not the population one. 7.3 Leakance leverage tracks the baseline volume error at Spearman +0.54 over 105 well-fit gauges; 36 of 38 "more losing" gauges have inflow above observed. The plains leverage is inflow-volume bias. Journal: entry for the plains run, ledger rows for its 14 shards and the dry run. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AhTffypSqtbq3BLkW1SQH1
…able term Offline fit of dMC-dev's `_first_order_euler_storage` at Raystown Dam with observed inflow (Saxton + local summed Q') and observed release. The law reaches NSE 0.18 train / 0.15 test at the box corner (theta 3, b 3) with an indefinite Hessian and no interior optimum, against pass-through 0.65 / 0.53 and a fitted 0.66-day linear reservoir 0.74 / 0.65. Capacity-as-S0 bounds the law's response time near 45 days while the observed release responds in under a day: model-class misspecification, not the sum-observation problem. Capacity sensitivity x0.5 / x2 does not change the ordering. Verdict: do not port the dMC reservoir operations; a reservoir term would need a release rule, not a fill-fraction law. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AhTffypSqtbq3BLkW1SQH1
Three checks after the flat hydrographs were questioned: the sandbox loop matches dMC-dev's `_first_order_euler_storage` verbatim to 2e-5 m3/s over 5,479 days; the step response e-folds in 93 to 509 days, consistent with the analytic linearised timescale; and a capacity sweep collapses the law onto pass-through (0.754 at 1 MCM vs 0.650) and then down to 0.175 at the real 940 MCM, monotone. The flatness is dMC's capacity-as-S0 design, not the implementation: the response floor S0/(b*theta*q_ref) is 67 days at the box corner, while the dam's daily buffer is ~2 MCM. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AhTffypSqtbq3BLkW1SQH1
…storage law by form Generic above/below dam sandbox plus a candidate search over the training population (67 dam-named arid gauges, 20 with an upstream gauge). At the two arid flood-control dams every model sits at NSE ~ 0 at every capacity: the release is zero for months and then a flat-topped evacuation (Alamo, ~180 m3/s) or a rectangular irrigation block (Santa Rosa), shapes a fill-fraction power law cannot produce. Abiquiu repeats Raystown: a 5.5-day linear reservoir (0.73 / 0.66) beats the law at any capacity. Do not port the law; prescribe observed release at dam rows, and drop the volume-deficit regulated gauges from training. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AhTffypSqtbq3BLkW1SQH1
…ion, literature) The linear reservoir in dam_sandbox.py was driven by the previous day's inflow; same-day it scores 0.800/0.732 at Raystown (was 0.737/0.650). It is exactly a Muskingum row with X = 0, K = T, so ddrs could host it by a coefficient override. Its main error is a release cap: min(S/T, Q_max) adds +0.13 test NSE at Raystown with observed inflow but only +0.02 with the trained model's inflow. Arid scheduled dams stay at NSE 0 under every storage-state rule. Population: 347 of 2,365 eval gauges have DOR > 0.5 (lower bound, NWM reservoir set), median NSE 0.495 vs 0.739 unregulated, 0.12-0.38 below in every area bin, mostly timing not volume. 216 have a gauge between them and their main reservoir. Literature (22 DOIs checked on Crossref): the fill-fraction law and level pool are lake laws; operating rules are not identifiable from downstream Q and the error lands in other parameters (Manning's n included). Options of record in the findings doc and .claude/RESERVOIRS.md: mask DOR > 0.5 gauges first (A/B), then one identity-row mechanism for observed releases and a linear/capped reservoir with prescribed Q_max. Corrects the arid doc's lagged linear column and its Abiquiu 17 % claim; adds §Reservoirs and do-not-use rows to research-status. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…kpoint Phase 1 writes a checkpoint per optimizer step, and the test phase loaded the newest one or failed with "no .mpk checkpoints found after Phase 1". A resume from experiment.checkpoint whose epoch is the last and whose sampler is exhausted (the checkpoint of a run's final step, same epochs) takes no step and writes none, so it could not be evaluated. That run is exactly the resumed checkpoint, which makes it the natural way to route a trained head through a different engine configuration with no training: the in-engine replay of a fixed dam table (next commit). run.rs (TrainAndTest, Phase 2): when Phase 1 wrote no checkpoint and experiment.checkpoint is set, log Phase 1 took no optimizer step and wrote no checkpoint; testing the resumed checkpoint <dir> and evaluate that checkpoint's head.mpk (and, for a learned release, its release files). Without experiment.checkpoint the old error stands. Test: juniata_acceptance's reservoir test resumes its reservoir run at the LAST checkpoint with the same epochs: the log carries the line above and no mb= line, and the gauge series is bitwise the reservoir run's. Docs: traps.md T10. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015TepsZAS2LkFC5EmvuZrz9
…se fits
The v2 rule-curve arms scored ~0 or below at dammed gauges while the
offline per-gauge fits gain +0.049 median NSE. To separate the law from its
training, route the offline fits themselves in the engine, test phase only.
experiments/reservoir/smoke/replay_offline_fits.py builds two
reservoir_release: fixed tables from the offline review's
fits_by_gauge.csv (--fits; the file lives outside the repo), restricted to
the dams on their gauge's own reach (expected_release_fit.csv role == dam,
on_reach; dam COMID from smoke_gauges.csv):
fixed_L2.csv COMID, T_days = L2_p_T0, year_completed, STAID
fixed_L4.csv COMID, T_days = L4_p_T0, c1s, c1c, c2s, c2c
(c_engine = c_offline * Ibar_offline / inflow_mean_m3s),
inflow_mean_m3s, year_completed, provenance columns
214 on-reach gauges map to 202 dams: 12 dams sit on the reach of two
gauges, and the fit of the gauge closest in drainage area to the dam reach
is kept (listed by the script). Ibar_offline / inflow_mean_m3s: median
0.952, range [0.496, 1.041]; engine |c1| + |c2|: median 0.353, max 6.6.
The docstring states the conventions checked against rc.py and
src/routing/release.rs: same flux sign (positive r stored, taken off the
inflow) and basis order; offline daily day-of-year phase vs the engine's
hourly continuous phase (at most ~1.25 d, 0.02-0.04 rad, not corrected);
offline daily implicit-Euler bucket on the routed gauge flow vs the engine's
hourly additive row with the S28 clamp.
config/experiments/dam_release_smoke_replay_{L2,L4}.yaml: the smoke off
arm's config snapshot (2026-09-27T04-29-33Z) verbatim plus
use_reservoirs: true, reservoir_release: fixed, reservoir_dam_row:
additive, the table, and experiment.checkpoint = that run's
epoch_50_mb_3 with epochs 50: zero optimizer steps, the test phase routes
the off arm's routing head with the dams (previous commit). Docs:
config.md (dam_row paragraph).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015TepsZAS2LkFC5EmvuZrz9
…ith feasibility penalty, T0-only)
Copies of dam_release_smoke_frozen_additive_rc.yaml (S3: frozen routing at
the smoke off arm's epoch_50_mb_3, additive dam row, T0-only head, per-dam
rule curve, completion years), with the v3 code (feasibility penalty,
row-sparse Adam, per-step L2, continuous rule-curve phase, clamp account):
S3v3 dam_release_smoke_frozen_additive_rc_v3.yaml
rule_curve_penalty 5.0, rule_curve_alpha 0.9, per_dam_l2 1e-3,
per_dam_lr 0.02, rule_curve_max 0.5 (peak |r| <= 1.41 Ibar)
S4v3 dam_release_smoke_frozen_additive_rc_t0_v3.yaml
S3v3 + per_dam_t0: true
T0v3 dam_release_smoke_frozen_additive_t0_v3.yaml
per-dam T0 multiplier only (rule_curve: false, so no penalty),
per_dam_lr 0.02, per_dam_l2 1e-3: the offline audit puts most of the
dam-specific gain in the per-dam bucket time constant
Penalty scale: P is 0 at theta = 0 (a hinge), so the step-1 value cannot
size it. Resuming v2 S3's end state (rule_curve_max 1.0; 3-17 % of
dam-row steps clamped, hinge active on ~9 % of dam-steps) gave P = 0.4-0.8 %
of the loss at lambda 1.0 (8.3e-4 of 0.204, 3.6e-3 of 0.462); 5.0 puts it
at ~2-4 % there. From theta = 0 it stayed at or below ~1e-6 of the loss
over the first 46 steps at lambda 1.0. Tier D: ddrs plan exits 0 for all
three in this worktree's workspace, no drift.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015TepsZAS2LkFC5EmvuZrz9
The engine reproduces the offline rule-curve gain where the fitted flux stays within the dam's mean inflow (99 of 117 on-reach DOR > 0.5 gauges: +0.028 vs offline +0.038) and fails where the offline fit relied on a carried storage deficit (max|c| > 1). Bounded composite +0.028. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015TepsZAS2LkFC5EmvuZrz9
…inding 1) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015TepsZAS2LkFC5EmvuZrz9
… balance
The dam-row clamp account added Σ max(lb − x, 0)·dt, a one-step rate
deficit, not the water the S28 clamp creates (review v3, finding 1). A dam
row is the trapezoid on S = K·X·I + (K(1 − X) + T)·Q (+ S0); for the
pre-clamp solve x the step balances exactly, and raising Q_{t+1} from x to
lb adds (K(1 − X) + T_{t+1})·δ of forgiven storage plus (dt/2)·δ of
below-floor outflow (the routed series reports lb at the step end while
the solve passed x down), δ = lb − x. Their sum is δ·dt/c4 = δ·D/2, which
is also the lateral volume that would have held the row at lb (A is unit
lower triangular and q'_d enters only row d, so ∂x_d/∂q'_d = c4_d).
- mmc_op::DamStepDiag hands out the dam row's c4 (a saved primitive).
- DamAccount accumulates created = Σ δ·dt/c4 and its storage part
Σ δ·(dt/c4 − dt/2) per dam.
- No pooled created share (finding 6: a dam below another counts the upper
dam's outflow in its own inflow). release_eval::clamp_summary reports the
per-dam (test phase) or per-dam-window (training) share distribution:
median, p90, max, counts >= 0.5 % and >= 5 %.
- release_clamp.csv: COMID,created_m3,storage_m3,inflow_m3,created_share,
clamp_steps,steps. metrics.release_clamp: n_dams, clamp_steps, steps,
created_m3, storage_m3, created_share_{median,p90,max},
n_dams_created_share_ge_{0p5,5}pct (no inflow sum, no pooled share).
- Training logs one "dam clamp, step N: ..." line per optimizer step over
its micro-batches' dam-windows (the engine's per-forward pooled line is
gone); the forward returns training::forward::ForwardRecord.
- tests/reservoir_rule_curve.rs: the self-referential hand recurrence is
replaced by two mass-balance tests that read the created volume off the
routed series alone (inflow + lateral − flux-stored − outflow − ΔS), on
the replace row (headwater) and the additive row (interior dam, with the
channel wedge; K and X held constant so S is a state function). Both
close to 5e-7 relative, with 119-138 of 240 steps clamped; the storage
part is (K(1 − X) + T)/(… + dt/2) of the total.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015TepsZAS2LkFC5EmvuZrz9
…e Adam) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015TepsZAS2LkFC5EmvuZrz9
…ams.reservoir_dam_floor: carry) The S28 clamp holds a dam row at lb and forgives the deficit: the water it creates stays in the river. The offline release fits kept that deficit in storage, and the offline gains depend on it (review v3, finding 2). A new option selects the floor; the default is bitwise today's routing. release_head.dam_floor: forgive (default) | carry (learned release) params.reservoir_dam_floor: forgive (absent) | carry (fixed tables) The params key mirrors params.reservoir_dam_row: rejected with reservoir_release: learned and without use_reservoirs. Config::dam_floor reads the block for a learned release, the params key otherwise; the engine takes it at construction (MuskingumCunge::dam_floor / set_dam_floor). carry: each armed dam carries an owed volume (m3, >= 0), kept on the inner backend next to the clamp account. - after each solve, owed += max(lb - x, 0)*dt/c4 (the created volume); - before each solve, the dam repays paid = min(owed, max(Qin_t - lb, 0)*dt) out of its effective lateral inflow (q'_eff -= paid/dt, owed -= paid), Qin_t = (N Q_t)_d + q'_d the step-start routed inflow plus its own q', before the rule-curve flux (the penalty's Qin; I_t by the op's own SpMV). owed_0 + created - repaid - owed = 0 at every step, so the dam row passes on its inflow less its storage change plus only what it still owes. It covers every dam-row clamp, the additive row's negative c1 at low flow included. What it conserves is the dam row's routed series (what an on-reach gauge reads): the rows below read the pre-clamp x in a clamped step, so over a period they receive the dam's inflow less dS less the below-floor part (created - storage), which is owed and repaid too (7 % of the inflow in the one-year replace-row test). Owing only the storage part would conserve their receipt instead; holding the in-step outflow at lb would conserve both. The owed state is DETACHED: the repayment is a constant cut to q', so training sees neither the debt a flux incurs nor its repayment as a function of theta, T0 or the routing parameters. The gradient through a clamped step stays zero (as with forgive), the repayment acts like a change to the forcing, and the feasibility penalty stays the only restoring gradient. The owed state restarts at 0 every training window (one engine per forward) and runs across the test phase's chunks: training::forward::carry_dam_owed seeds each chunk's dams from DamClampSums::owed_for, and DamClampSums::merge keeps each dam's closing owed volume. A dam not yet completed is not armed and has no owed state. With carry, `created` counts every step at the floor, including the steps spent repaying, so it runs well above the forgive figure; owed is what the floor finally adds. Outputs: release_clamp.csv gains repaid_m3, owed_m3 (COMID,created_m3, storage_m3,repaid_m3,owed_m3,inflow_m3,created_share,clamp_steps,steps); metrics.release_clamp gains dam_floor, repaid_m3, owed_m3, owed_share_max; metrics.release_training.dam_floor; the training "dam clamp" line and the test-phase line gain "dam_floor carry: repaid R m3, owed at end O m3 (max p% of a dam's inflow)". Tests (tests/reservoir_dam_floor.rs): - no clamp: carry == forgive bitwise, outputs and dL/dT0, dL/dc, on option C rows (both rows) and the release path with a rule curve; - one year (one rule-curve period) with forced clamps, replace row (headwater) and additive row (interior dam, channel wedge): created - repaid - owed = 0, the dam-row balance read off the routed series is -owed, outflow = inflow to 1.3e-4 / 2e-5 where forgive creates 11 % / 5 %; - a year in 15-day chunks threaded like the test phase matches one engine bitwise with the owed volume carried; dropping it at each boundary moves the dam's series by up to 8.4 m3/s; - set_dam_owed input checks. Config and summary unit tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015TepsZAS2LkFC5EmvuZrz9
… carry dam floors)
Eight zero-step resumes (experiment.checkpoint = the run's last checkpoint,
epochs unchanged, so no optimizer step is taken and the test phase evaluates
that checkpoint), each the source config plus the lines named in its header:
- dam_release_smoke_replay_{L2,L4}_carry: the in-engine replays of the
offline fits with params.reservoir_dam_floor: carry, the floor that keeps
the deficit in storage as the offline fits did.
- dam_release_smoke_reeval_{S3v3,S4v3,T0v3}_{forgive,carry}: the three v3
arms' final checkpoints (runs 2026-09-27T23-35-58Z, 23-36-04Z, 23-36-12Z;
frozen routing head, release_head.mpk, per-dam release_dams.mpk at full
precision) with release_head.dam_floor: forgive (the law as trained, v4
clamp account) and carry.
ddrs plan (v4 binary) exits 0 for all eight; the only drift is the
reservoirs table switching between configs (relocked).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015TepsZAS2LkFC5EmvuZrz9
The v4 re-evaluations with dam_floor: carry show a failure the unit tests (replace row; additive row with K held constant) cannot: once a dam owes more than its storage, the repayment (all of Qin - lb) pins its outflow at the floor, where the additive row's channel K, X (Cunge, evaluated at the dam's own Q_t) are days and 0.5, so c1 is about -0.5. Every rising step of the upstream inflow then clamps and owes about K*X*dI, while the wedge's release on falling steps (x > lb) passes downstream instead of repaying. Measured: replay L2 carry, COMID 74038104 clamps 25,813 of 131,130 test steps (forgive 2,219), owes 3.7x its 15-year inflow and repays all of it; 21 of 202 dams end owing > 0.5 % of their inflow. S3v3 carry: 25 of 675 dams repay > 50 % of their inflow. Median NSE drops (L2 0.7135 -> 0.7047, L4 0.7165 -> 0.7059, S3v3 0.7184 -> 0.7047). A 30-day synthetic chain reproduces it with a diurnal upstream swing and a debt above the storage. Documented as a known failure (DamAccount docs, config.md dam_floor, RESERVOIRS.md); no code change. A fix has to reclaim the wedge's release while in debt, not only cap the pre-solve repayment. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015TepsZAS2LkFC5EmvuZrz9
…count; journal Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015TepsZAS2LkFC5EmvuZrz9
…vity) The additive dam row keeps its reach's channel wedge K_r·X_r·I in storage, so c1 = (dt − 2·K_r·X_r)/D goes negative wherever K_r·X_r > dt/2. At low outflow the Cunge K_r is days and X_r is 0.5 (c1 about −0.5), so rising inflow drives the pre-clamp solve below the floor: forgive creates water, and carry pumps debt (82351ca; review v3, finding 3). A dam reach is a pool, so on the dam rows only the wedge is capped: release_head.dam_row_positivity: false (default) | true (learned release) params.reservoir_dam_row_positivity: absent (= false) | bool (fixed tables) S19p (forward_chain_inner, after S19'', before the saved X): X_eff = min(X_r, cap), cap = 0.5·(1 − δ)·dt/K_r, δ = POSITIVITY_DELTA = 1e-2 on the additive dam rows, so D and c1..c4 all read X_eff and c1 = (dt − 2·K_r·X_eff)/D >= δ·dt/D > 0. This is S19''s hi_a branch alone, op for op (dam_row_x_cap). δ for S19''s reason: at δ = 0 the cap lands on c1 = 0 and f32 roundoff crosses it. S19''s K floor and hi_b are not needed: the additive row's c3 numerator carries 2·T_t and the cap only raises 2K(1 − X); the cap is positive for any K > 0 (K = 0: infinite, inactive). Skipped under enforce_positivity (S19' already caps every row at hi_a); the key is rejected with it at load, and with the replace row (c1 = dt/D > 0). B19p (timestep_backward_core): where the cap bound (x_eff == cap, recomputed from the saved K by the forward's ops; ties go to the cap), ∂L/∂x_eff = 2K(g_2kx − g_2k1mx) goes into K through ∂cap/∂K = −cap/K, and the Cunge X_r chain (∂X/∂Q, ∂X/∂B, ∂X/∂c) gets nothing. K's total there is 2·g_2k1mx, the derivative of 2K·X_eff = (1 − δ)dt (constant) and 2K(1 − X_eff) = 2K − (1 − δ)dt. Folded at B19''s gk_from_x_cap point; runs under ddr_match too (X's only path). T_{t+1}, T_t gradients unchanged in form (X_eff does not depend on T). Where the cap does not bind the row is bitwise the additive row, outputs and gradients. O(n) extra elementwise work per step; the O(nnz) backward is kept. Diagnostics: DamStepDiag carries c1; DamClampAccount gains c1_min and neg_c1_steps; DamClampRecord / release_clamp.csv (trailing column) / ClampSummary gain neg_c1_steps (metrics.release_clamp.neg_c1_steps, n_dams_neg_c1; "dam-row steps with c1 < 0 k/m (j dams)" in the clamp log lines); metrics.release_training.dam_row_positivity. Tests (tests/reservoir_dam_positivity.rs, 8): - off by default; where the cap never binds (2.5-3 km reach, T0 0-20 d, option C) bitwise the additive row, outputs, dL/dn, dT0, da, db, account; under enforce_positivity the key changes nothing; - low flow (0.5 m3/s upstream, Cunge): without the cap c1 < 0 on 240 of 240 dam-row steps (min −0.60 at T 0.05 d); with it 0 steps, min c1 > 0; the reach above the dam is bitwise untouched; - gradcheck at T0 {0, 0.1, 1.5, 20} d, cap inactive (bitwise precondition per T0) and cap active on every step (20 km): T0, a, b, n_dam, n_up; under ddr_match (constant X); T_t / T_{t+1} at one step, with the one-step c1 measured (−0.58 -> +1.0e-3); - the v4 synthetic debt pump (diurnal upstream ±30 %, 1e7 m3 opening debt, carry, daily chunks): T 0.05 d without the cap: new debt 1.95e7 m3, still owing 1.02e6 at day 30; with it: new debt 0, owed falls every day, cleared on day 11, repaid = opening debt, carried account closes every day; - with the cap binding on every step and K, X constant (ddr_match + velocity floor, 40 km): carried dam-row balance over one rule-curve year off the routed series with S = K·X_eff·I + (K(1 − X_eff) + T)·Q closes to 2.6e-7 of the created volume, outflow = inflow to 2e-5. Config tests for both keys (defaults, parse, the four rejections). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015TepsZAS2LkFC5EmvuZrz9
… training arms)
Test-only replays (zero-step resume of the smoke off arm's last checkpoint,
as dam_release_smoke_replay_L2/_L4) with params.reservoir_dam_row_positivity:
true:
dam_release_smoke_replay_{L2,L4}_pos_forgive.yaml (from replay_{L2,L4}.yaml)
dam_release_smoke_replay_{L2,L4}_pos_carry.yaml (from replay_{L2,L4}_carry.yaml)
Training arms S5carry / S5forgive: S4v3
(dam_release_smoke_frozen_additive_rc_t0_v3.yaml: frozen routing, NID
completion years, additive row, per_dam_t0, rule curve with the feasibility
penalty 5.0 at alpha 0.9, per_dam_l2 1e-3, row-sparse Adam) with
release_head.dam_row_positivity: true, dam_floor carry / forgive,
per_dam_lr 0.05 (v3: 0.02) and rule_curve_max 1.0 (v3: 0.5):
dam_release_smoke_frozen_additive_rc_t0_v5_{carry,forgive}.yaml
Tier D: `ddrs plan` (binary ddrs-v5-90aae43) exits 0 on all six; the only
drift is the reservoirs table switching between configs (relocked).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015TepsZAS2LkFC5EmvuZrz9
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015TepsZAS2LkFC5EmvuZrz9
…it doubles the fitted per-dam T0 gain Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015TepsZAS2LkFC5EmvuZrz9
…ap 1.0) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015TepsZAS2LkFC5EmvuZrz9
…ults page source research/findings/2026-09-28-per-dam-release-calibration-findings.md; research-status §Reservoirs entry; experiments/reservoir/overnight_page/ (prepare.py -> data.json, build.py -> publish/, both gitignored) for https://claude.ai/artifact/YELrGtjwb5qjhq9JEjLs2u; S5 gate output. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015TepsZAS2LkFC5EmvuZrz9
…ion ceiling of any dam model Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015TepsZAS2LkFC5EmvuZrz9
…nst runoff-rescaling nulls A flood-pool law (FA: capture above a per-dam release target, evacuate at the target) is a real, flood-control-specific gain (+0.0073 over the per-dam bucket at 166 flood-control dams, controls 0, DiD +0.0084) and removes about 35 % of the 30-120 d excess at flood-control DOR > 0.5 dams. A seasonal withdrawal at irrigation dams is not supported (they store in the wet season and release in summer). Population median, offline: dam law set +0.0103, global runoff scalar +0.0047, both about +0.014 to +0.017; +0.03 is not reachable through dams and rescaling. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015TepsZAS2LkFC5EmvuZrz9
Adds the laws_v6 flood composite and flood-pool figures (embedded) with their five-part brief and the population-median estimates, and corrects the page's target to the 2,365-gauge population median. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015TepsZAS2LkFC5EmvuZrz9
…s, side-session reports, full-run analysis Brings the review reports (experiments/reservoir/review_2026-09-27/), the synthesis (research/findings/2026-09-27-dam-release-review-synthesis.md), the agent reports and the full-run paired analysis onto the branch the findings cite. Journal conflict resolved by keeping this branch's filled entries; reservoir-options had no entry or ledger row this branch lacks. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015TepsZAS2LkFC5EmvuZrz9
…rvoir_flood_pool) Law FA of experiments/reservoir/laws_v6 (report §2, §5) on top of the dam row. Each pooled dam carries a pool F (m3, >= 0). Before each solve, from the dam's step-start inflow I = (N·Q_t)_d + q'_d (the feasibility penalty's Qin): Vc = min(phi·max(I − Qc, 0)·dt, Fmax − F) captured Ve = min(F, max(Qc − (I − Vc/dt), 0)·dt) evacuated at the release target the dam row's q' gains (Ve − Vc)/dt, and F ← (F + Vc) − Ve after the solve, with Qc = kc·Ibar, Fmax = z·Ibar·86400, Ibar the table's inflow_mean_m3s. Mass is exact by construction (F >= 0 exactly in f32) and the discharge floor is never used; A, the CSR pattern and the hand-written backward are unchanged (src/routing/mmc.rs FloodPool, set_flood_pool, set_flood_pool_state). Gradient: F stays on the autodiff tape across a window (O(n_pool) nodes per step); I is NOT detached (an autodiff gather of Q_t over the pooled rows' upstream edges), so an upstream routing parameter sees the pool's response to its inflow; min/max take Burn's mask_where subgradients. Config: release_head.flood_pool: none | flood_control | all (default none, bitwise today's routing; flood_control pools the dams with the table's raw purpose_flood column). A fixed table: params.reservoir_flood_pool: true with kc, phi, z (days) columns + inflow_mean_m3s (z = 0: no pool). Rejected with the carried dam floor; dataset open refuses a pool without its columns. Per-dam parameters: DamParams.pool [n_dams, 3], raw 0 at init, trained with the row-sparse Adam, per_dam_lr and per_dam_l2 (anchored at the init): kc = clamp(3·exp(r), 0.5, 20), phi = sigmoid(r), z = 120·sigmoid(4·r + logit(0.05/120)) days (0.05 d at init: off in effect, gradient through the cap; rate 4 so ~1.5 raw units of per-dam Adam travel span 0.05 to ~20 d). State: F restarts at 0 every training window and is carried across test-phase chunks (training::forward::carry_dam_state, DamClampSums::merge_pool / pool_f_for). The resolved test table and release_params.csv carry kc, phi, z. Diagnostics: per-step flood pool log line and pool parameter medians; test phase release_pool.csv and metrics.release_pool. Tests: tests/reservoir_flood_pool.rs (z = 0 bitwise, one-year flood mass balance residual 6e-7 of inflow with zero clamps, gradcheck of kc/phi/z/T0 and an upstream n in the capture, cap and evacuation regimes (worst rel 3.1e-3), 15-day chunks = one engine bitwise, input checks, training = resolved table bitwise, dataset-open refusals, frozen-routing training moves kc/phi/z with the routing head bitwise + resume), plus unit tests in config, dam_params, lazy_adam, reservoirs, release_eval. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015TepsZAS2LkFC5EmvuZrz9
…nd without the pool
experiments/reservoir/smoke/replay_flood_pool.py builds fixed_FA.csv from
laws_v6/laws_by_gauge.csv: the 69 flood-control dams on their gauge's own reach
(expected_release_fit.csv role dam, on_reach true; smoke_gauges.csv
dam_purpose Flood Risk Reduction; 6 dams shared by two gauges keep the gauge
closest in area), T_days = FA_p_T0, and the pool's kc, z rescaled from the
gauge's Ibar to the dam's inflow_mean_m3s (Qc in m3/s and Fmax in m3 kept, as
replay_offline_fits.py keeps the rule-curve flux); phi unchanged. Engine kc
median 3.9 (2 dams outside the learned box [0.5, 20]), z median 14.5 d
(10 dams above the learned 120-d bound; the fixed path routes them as fitted).
config/experiments/dam_release_smoke_replay_FA_{pool,nopool}.yaml: the L2 pos
replay recipe (zero-step resume of the smoke off arm, additive row with the
dam-row positivity cap, dam floor forgive) on that table, with
params.reservoir_flood_pool true and false: the arms differ by the pool alone.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015TepsZAS2LkFC5EmvuZrz9
dam_release_smoke_S6base.yaml: the S4v3 recipe (additive row, per-dam T0, rule curve max 0.5, per_dam_lr 0.02, per_dam_l2 1e-3, feasibility penalty 5.0 alpha 0.9, completion years, routing frozen at the smoke off arm) plus release_head.dam_row_positivity: true and experiment.rho 180 / warmup 30 (the pool's ~60-day timescale). testing.warmup: 5 keeps the test metrics on the same days as every earlier arm (the overlay would otherwise inherit 30). dam_release_smoke_S6pool.yaml: S6base plus release_head.flood_pool: flood_control. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015TepsZAS2LkFC5EmvuZrz9
…ins (prepared, not launched)
dam_release_full_{pool,nopool}_s{42,43}.yaml: dam_release_full_learned.yaml
(2,365 gauges of gages_3000, train WY1982-1995, test WY1996-2010) with the S6
release settings (seasonal false, additive row, dam-row positivity, dam floor
forgive, rule curve max 0.5, per-dam T0, per_dam_lr 0.02, per_dam_l2 1e-3,
feasibility penalty 5.0 alpha 0.9), reservoir_T0 box [0.0001, 365], rho 180 /
warmup 30, testing.warmup 5, and joint training: no routing_checkpoint,
freeze_routing false. The pool arms add release_head.flood_pool:
flood_control; the twins differ from them by that key alone. Seeds 42 and 43.
ddrs plan (main workspace) exits 0 on all four.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015TepsZAS2LkFC5EmvuZrz9
… arms Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015TepsZAS2LkFC5EmvuZrz9
Conflicts and resolutions (both features kept, neither changed): - src/config.rs, DataSources: this branch's `reservoirs` table path and master's `pins` map sit side by side (reservoirs first, then pins, ICECHUNK_PINNABLE and DataSources::pin_for). validate_pins (master) and validate_reservoirs (branch) both auto-merged into their call sites. - .claude/skills/ddrs-dev/references/config.md, data_sources: kept the `reservoirs` paragraph and master's `pins:` section with its example. Auto-merged, checked by hand: the baseline cache-key test fixtures carry both `reservoirs: None` and `pins: None`; `reservoirs` is locked through master's fingerprint_path_at (a CSV, so a content hash; pin_for returns None); CLAUDE.md and testing.md carry both sections. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015TepsZAS2LkFC5EmvuZrz9
…n, gate update Review v6 HIGH 1: per_dam_l2 1e-3 anchored at the pool-off init holds the pool off (S6pool z stayed 0.05 d), so S7base/S7pool drop per_dam_l2 and differ only by flood_pool. MEDIUM 2: the replay's no-pool twin now uses the plain-bucket T0 (fixed_FA_L2T0.csv); engine pool minus fair twin +0.0146 vs offline +0.0139 at 69 flood-control gauges, 0 water created. flood_gate uses the fair twin and the S7 pair and reports whether the pools grew. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015TepsZAS2LkFC5EmvuZrz9
Merged
1 of 2 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Learned dam release for ddrs, from the first NID-driven release head to per-dam calibrated parameters, plus the research record behind it. Everything new is off by default: existing configs route bitwise as before.
Engine and training (all gated, default-off):
params.use_reservoirs+reservoir_release: learned | fixed: NID dams routed as linear reservoirs, storage-conserving dam row.release_head.freeze_routing/routing_checkpoint: train only the dam parameters on a fixed routing head (controls bitwise unchanged).release_head.dam_row: additive: reservoir storage adds to the reach's channel storage, so zero residence time is the channel again.release_head.dam_row_positivity(andparams.reservoir_dam_row_positivity): keeps the dam row's inflow coefficient c1 >= 0.per_dam_t0) and a harmonic seasonal rule curve (rule_curve), row-sparse Adam,per_dam_lr,per_dam_l2, feasibility penalty, continuous seasonal phase.release_clamp.csv): created, repaid and owed volume from the dam row's volume balance; optional mass-conserving floor (dam_floor: carry, safe only with the positivity cap).params.reservoir_dam_row,_dam_row_positivity,_dam_floor) for replays of offline fits.Research record: five reviews and their synthesis, offline release-law studies (
experiments/reservoir/rulecurve_offline/,laws_v6/), smoke-set arms and replays (experiments/reservoir/smoke_v2/), findingsresearch/findings/2026-09-28-per-dam-release-calibration-findings.md, journal entries for every run.Results (one seed; details in the findings and on the results page)
Flood pool
Per-dam flood pool (law FA of
experiments/reservoir/laws_v6/) on top of the dam row, default off.release_head.flood_pool: none | flood_control | allfor the learned release (flood_controlpools the dams whose table flags flood control);params.reservoir_flood_pool: truefor fixed tables, withkc,phi,z(days) andinflow_mean_m3scolumns (z = 0: no pool).kc(capture threshold as a multiple of mean inflow),phi(captured fraction of the excess) andz(pool size in days of mean inflow), trained with the row-sparse Adam,per_dam_lrandper_dam_l2.dam_floor: carry.laws_v6/report.txt): median paired dNSE +0.0073 [+0.0034, +0.0124] over the per-dam bucket at all 166 flood-control dams in the test years, matched controls 0.0000.tests/reservoir_flood_pool.rs(z = 0 bitwise, one-year mass balance, gradchecks of kc/phi/z/T0 and upstream n);compare_ddr_sandboxABSOLUTE MATCH (max abs 1.5e-4 m³/s); Tier C on the merged treedbda31c: 1006 passed, 0 failed, 26 ignored.Test plan
cargo test --test ddr_sandbox_matchandcargo run --release --example compare_ddr_sandbox: ABSOLUTE MATCH (max abs 1.5e-4 m³/s) at every engine commitcargo test -j 4 --no-fail-fast: 968 passed, 0 failed at90aae43cargo test --release --test juniata_acceptance -- --nocapture: 3/3dbda31c: conflicts insrc/config.rsand the skill'sconfig.mdresolved by keeping both features; Tier C 1006 passed / 0 failed,icechunk_pins+cli_fingerprint+cli_lockfile21/21, ABSOLUTE MATCH,juniata_acceptance3/3🤖 Generated with Claude Code
https://claude.ai/code/session_015TepsZAS2LkFC5EmvuZrz9