[cheginit]

The Old River Control Structure diversion must assimilate observations and persist the
last one rather than decay it. Two base defects sat under it: the subtraction fed back
through the Muskingum-Cunge state, so the Mississippi lost 1.25 times the observation,
and when the gage record ended the transfer stopped within the hour while the
Atchafalaya decayed over two.

Additions

• diversion_da.diversion_persist_days (whole days, default 11, 0 disables), the same
  idiom as reservoir_rfc_forecast_persist_days. The last observation is held flat on
  the routing grid until an absolute deadline; after it both sides stop at once.
• The diversion reads its own gage from usgs_timeslices_folder (a named consumer in
  both build_da_sets copies); nothing depends on streamflow_nudging. A process with
  no report at or before t0 and no checkpoint seed scans the folder back to the horizon,
  once, and remembers an empty result.
• The amount each donor gave up is kernel state: returned as results element 11 by the
  job that routes the donor, carried to the next window by both drivers and by the BMI
  checkpoint.
• The five production configs carry the block; horizons cover each range plus 28 h of
  data age (12 days for medium range, 32 for long range).
• The NHF driver writes a lite restart (q0 by routing link plus waterbody states) after
  every window when output_parameters.lite_restart names a directory, and the network
  reads it back as it is (a file with a duplicated link is refused); the call had been
  commented out since the driver was created.
  The Old River demonstration spins each case up five days and restarts from that file;
  a restarted run is bit-identical to the uninterrupted one.

Removals

• persist_historical_median and the hardcoded monthly means.

Changes (base fixes)

• Kernel restores the remembered subtraction when loading qdp, as qdpp does for
  bottom-added qlat.
• A diversion gage with no observation keeps its routed flow instead of the lastobs
  decay (keyed on the gage; the receiver's job usually lacks the donor).
• Diversion gages join the nudging roster whenever they have observations, so a
  lastobs_file that predates the diversion cannot silence the receiver.
• The BMI merges the scaling frame into the window's local frame; stored on the DA
  object it handed every later window one column of the run-spanning rows.
• A crosswalk entry whose flowpath or gage is not in the domain is a warning, not a
  failed network build, so one configuration serves every domain.
• A window's missing TimeSlice files are reported in one line, not one per file.
• A configuration with only diversion_da set, which the schema dumps with every other
  DA section as None, initializes and rolls over instead of crashing on those sections.
• A flowpath-level channel restart reaches the kernel in its column order (qu0, qd0,
  h0, ql0); the reader emitted qd0, h0, qu0, ql0, which put the flow in the depth slot.
• The Muskingum-Cunge depth search restarts off its degenerate near-zero-depth root
  (secant residual above a tenth of the Muskingum estimate). A link handed a flow far
  above its previous outflow used to publish a dry channel and pass about 100 cms per
  step, which put five April dips at Simmesport in the demo; pinned by
  test_mc_depth_seed.py, and the three test_mc_nonconvergence.py cases that froze
  depth at bankfull now solve to their normal depth. Ordinary channels are bit-identical
  to development; only collapsed searches differ: cold-start transients, and trickles
  of a few centimeters over very wide beds (under 0.3 cms on a bed wider than 100 m).

Testing

1. pixi run pytest -m "not integration": 920 passed, 4 skipped, 1 xfailed; every
   integration test in the repository 7 passed, 6 skipped (unprepped domains, obsolete modules).
2. pytest test/nhf/old_river -m integration: 3 passed. Record cut 2011-05-31; both
   nodes hold 14,484 cms for 30 days; the donor gives up 1.000 times the observation
   (was 1.251). Runs with nudging on and, against a control with no DA, off. Simmesport moves less than
   10 percent in any hour while the observation is applied or held.
3. BMI chunked run bit-identical to the CLI windowed run over 2,772 hours; kernel toy
   cases identical under every compute method, pools 1 and 2, including a donor at a
   subnetwork cut.
4. Demo notebook re-run on the final kernel, twelve figures embedded. The demonstration
   and the tests now start 2011-04-19 from restarts written by five-day spin-ups of each
   case; the retrospective hot start carries neither the Vicksburg nudge nor the transfer
   and put a two-day dip at Baton Rouge in every earlier figure.
5. test/nhf/old_river/old-river-diversion.ipynb checks each expected behavior on real
   inputs with asserted numbers: the lateral inflow at the structure is at most 0.1 cms
   against 17,000 to 66,000 routed; the amount removed at the donor and added at the
   receiver equals the observation at every written hour; the transfer reaches Baton
   Rouge and Simmesport with the same volume; the last value holds 45 days and the donor
   stops at the horizon; a fourfold observation floors the donor at zero and the run logs
   the mass imbalance while the Atchafalaya still receives the full amount. Run logs go
   to a file, not the cells.

Notes

• The TimeSlice folder is now required wherever diversion_da is set, nudging on or
  off, with station 07381482 in the files; every range's deployment stages it.
• Staging depth: a forecast cycle started without a checkpoint scans the folder back
  diversion_persist_days for a quiet gage, so the routing job stages that many days of
  the per-day usgs_timeslices directories into the folder. A scan that finds nothing
  logs the folder's oldest file.

Checklist

• PR has an informative and human-readable title
• Changes are limited to a single goal (no scope creep)
• Code can be automatically merged (no conflicts)
• Code follows project standards (link if applicable)
• Passes all existing automated tests
• Any change in functionality is tested
• New functions are documented (with a description, list of inputs, and expected output)
• Placeholder code is flagged / future todos are captured in comments
• Project documentation has been updated (including the "Unreleased" section of the CHANGELOG)
• Reviewers requested with the Reviewers tool ➡️

[sclaw]

This looks good, Taher. Before I approve, could you please re-run test/nhf/old_river/run-troute-with-water-transfer.ipynb and paste the output figures? Also please respond to my two comments.

[cheginit]

@sclaw Thanks for the review. I addressed all your concerns and found some new issues with the secant solver that I addressed as well. I created a new notebook that you can find here.