Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion .github/scripts/ruff_check.sh
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,9 @@ if [ $? -ne 0 ]; then
fi

echo "Running ruff format on staged files..."
echo "$STAGED_FILES" | xargs ruff format
# --force-exclude so the [tool.ruff.format] exclude is honored even
# though paths are passed explicitly (notebooks are linted, not formatted)
echo "$STAGED_FILES" | xargs ruff format --force-exclude
if [ $? -ne 0 ]; then
echo "ruff format failed. Please fix errors before committing."
exit 1
Expand Down
16 changes: 5 additions & 11 deletions MAINTENANCE.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@
- [Maintenance ledger](#maintenance-ledger)
- [Open](#open)
- [3.0.1 hotfix release (flopy pin unwind + pyPRMS floor)](#301-hotfix-release-flopy-pin-unwind--pyprms-floor)
- [conda-forge feedstock: pyprms floor replaces packaging pin](#conda-forge-feedstock-pyprms-floor-replaces-packaging-pin)
- [pyPRMS upstream: close PR #64, port its regression test](#pyprms-upstream-close-pr-64-port-its-regression-test)
- [Reconcile check_version.yaml with release_preflight.sh](#reconcile-check_versionyaml-with-release_preflightsh)
- [Deliver CI-usage findings to org admins](#deliver-ci-usage-findings-to-org-admins)
Expand Down Expand Up @@ -51,16 +50,6 @@ check it), **Action** (what to do once unblocked), and optional
released flopy until then; flopy imports lazily so this is not an
import-time problem.

### conda-forge feedstock: pyprms floor replaces packaging pin

- **Blocked on:** feedstock PR #14 (pyprms >=0.10.0 floor, build 1)
MERGED 2026-08-19; awaiting the main-branch CI upload. Check:
`https://api.anaconda.org/package/conda-forge/pywatershed` shows a
3.0.0 build with build number 1.
- **Action:** verify the new build's run deps carry `pyprms >=0.10.0`
and no `packaging <26.3`, then move to Done. This was the last
`packaging <26.3` pin anywhere.

### pyPRMS upstream: close PR #64, port its regression test

- **Blocked on:** nothing — #64 was closed unmerged 2026-08-17; the
Expand Down Expand Up @@ -92,6 +81,11 @@ check it), **Action** (what to do once unblocked), and optional

## Done

- 2026-08-19: conda-forge pywatershed 3.0.0 build 1 published (feedstock
PR #14: `pyprms >=0.10.0` replaces `packaging <26.3`; verified on
anaconda.org, uploaded 19:35 UTC). The last `packaging <26.3` pin
anywhere is gone. Also: PR #407's sagehen ubuntu OOM resolved by
serial pytest (`-n=1`) on Linux.
- 2026-08-19: gh-pages v3.0.0 extended release notes are live upstream
(`doi-usgs.github.io/pywatershed`, 2026-07-13 post) and all 3.0.0
release-body links resolve — step 9 done, the 3.0.0 release is
Expand Down
21 changes: 20 additions & 1 deletion doc/whats-new.rst
Original file line number Diff line number Diff line change
Expand Up @@ -27,11 +27,30 @@ Bug fixes

Internal changes
~~~~~~~~~~~~~~~~
- Lint Jupyter notebooks. ``[tool.ruff] include`` covered only ``*.py``, so
``ruff check .`` and CI never saw notebooks, while the pre-commit hook
passed ``*.ipynb`` paths explicitly (which overrides ``include``) and did.
Notebook problems were therefore invisible until someone edited a notebook
and was met with errors they had not caused. Notebooks are now in
``include``, with ``E501`` and ``I001`` exempted: they are narrative, and
several import a module purely for its side effect (``hvplot.xarray``
registers a ``.hvplot`` accessor) and must do so after the module they
extend, which sorting undoes. Notebooks are linted but not auto-formatted,
for the same line-length reason. This surfaced four broken cells that no
test covered: two stray ``)``, a string opened with ``"`` and closed with
``'``, and a use of ``pl.Path`` with no ``import pathlib``. Three notebooks
needed further cleanup once the syntax errors stopped masking it: unused
imports, unused assignments, a semicolon-joined statement, and bare
``except`` clauses narrowed to ``except AssertionError``. Also repairs
four malformed ``# noqa`` directives (including a ``# noaq`` typo) that
ruff silently ignored, which meant the unused-import fixer would have
deleted the ``hvplot`` imports they were meant to protect.
(:pull:`412`) By `James McCreight <https://github.com/jmccreight>`_.
- Add ``MAINTENANCE.md``, a ledger of maintenance todos blocked on external
events (dependency releases, cross-repo work), each with a mechanically
checkable unblock condition; the ``/maintenance`` Claude skill checks them
live and reports what is actionable.
(:pull:`XXX`) By `James McCreight <https://github.com/jmccreight>`_.
(:pull:`409`) By `James McCreight <https://github.com/jmccreight>`_.
- Reduce CI footprint with a skeleton/full split: pushes to any branch (in
this repository or on forks) run a skeleton — installs, linting, domainless
tests, the docs build, and example notebooks on ubuntu only — while the
Expand Down
1 change: 0 additions & 1 deletion evaluation/performance/input_performance.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,6 @@
"metadata": {},
"outputs": [],
"source": [
"import pathlib as pl\n",
"import pywatershed\n",
"\n",
"repo_root = pywatershed.constants.__pywatershed_root__.parent"
Expand Down
1 change: 0 additions & 1 deletion evaluation/performance/nhm_pynhm_vs_prms_performance.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,6 @@
"metadata": {},
"outputs": [],
"source": [
"import pandas\n",
"import pandas as pd\n",
"pd.options.plotting.backend = 'holoviews'\n",
"results_df = pd.DataFrame(results_post).T\n",
Expand Down
2 changes: 2 additions & 0 deletions evaluation/performance/plot_4_performance_domains.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,8 @@
"source": [
"%matplotlib inline\n",
"\n",
"import pathlib as pl\n",
"\n",
"import matplotlib.pyplot as plt\n",
"import geopandas as gpd\n",
"import contextily as cx"
Expand Down
4 changes: 2 additions & 2 deletions evaluation/performance/prms_5.2.1_performance.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@
" else:\n",
" raise FileExistsError(f\"working_dir exists and overwrite=False: {working_dir}\")\n",
"\n",
" control_dir = data_dir / 'pynhm/performance_runs/PRMS/control_files')\n",
" control_dir = data_dir / 'pynhm/performance_runs/PRMS/control_files'\n",
" n_hru_desc = 'multi'\n",
" if domain_dir.name == 'hru_1':\n",
" n_hru_desc = 'single'\n",
Expand Down Expand Up @@ -173,7 +173,7 @@
"metadata": {},
"outputs": [],
"source": [
"import os; os.getcwd()"
"os.getcwd()"
]
},
{
Expand Down
3 changes: 0 additions & 3 deletions evaluation/prms/panel_errors.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -17,11 +17,8 @@
"metadata": {},
"outputs": [],
"source": [
"import pathlib as pl\n",
"import numpy as np\n",
"import param\n",
"import pywatershed\n",
"import xarray as xr\n",
"\n",
"import panel_errors_helpers"
]
Expand Down
3 changes: 0 additions & 3 deletions examples/01_multi-process_models.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -28,9 +28,7 @@
"source": [
"from copy import deepcopy\n",
"import pathlib as pl\n",
"from platform import processor\n",
"from pprint import pprint\n",
"from sys import platform\n",
"import yaml\n",
"\n",
"import pydoc\n",
Expand All @@ -39,7 +37,6 @@
"import jupyter_black\n",
"import numpy as np\n",
"import pywatershed as pws\n",
"from pywatershed.utils import gis_files\n",
"from pywatershed.utils.path import dict_pl_to_str\n",
"\n",
"import xarray as xr\n",
Expand Down
4 changes: 0 additions & 4 deletions examples/02_prms_legacy_models.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -32,11 +32,7 @@
"outputs": [],
"source": [
"import pathlib as pl\n",
"from platform import processor\n",
"from pprint import pprint\n",
"from shutil import rmtree\n",
"from sys import platform\n",
"import warnings\n",
"\n",
"import pydoc\n",
"\n",
Expand Down
4 changes: 0 additions & 4 deletions examples/03_compare_pws_prms.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -45,18 +45,14 @@
"outputs": [],
"source": [
"import pathlib as pl\n",
"from platform import processor\n",
"from pprint import pprint\n",
"from shutil import rmtree\n",
"import subprocess\n",
"from sys import platform\n",
"import warnings\n",
"\n",
"import hvplot.pandas # noqa\n",
"import hvplot.xarray # noqa\n",
"import jupyter_black\n",
"import numpy as np\n",
"import pandas as pd\n",
"import pywatershed as pws\n",
"import xarray as xr\n",
"\n",
Expand Down
5 changes: 1 addition & 4 deletions examples/04_preprocess_atm.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -23,13 +23,10 @@
"metadata": {},
"outputs": [],
"source": [
"from copy import deepcopy\n",
"import pathlib as pl\n",
"from pprint import pprint\n",
"import shutil\n",
"\n",
"import jupyter_black\n",
"import numpy as np\n",
"import pywatershed as pws\n",
"import xarray as xr\n",
"\n",
Expand Down Expand Up @@ -103,7 +100,7 @@
" \"tmin\": dom_dir / \"tmin.cbh\",\n",
"}\n",
"\n",
"cbh_dir = nb_output_dir / f\"cbh\"\n",
"cbh_dir = nb_output_dir / \"cbh\"\n",
"cbh_dir.mkdir(exist_ok=True)\n",
"\n",
"for kk, vv in cbh_files.items():\n",
Expand Down
10 changes: 6 additions & 4 deletions examples/06_flow_graph_starfit.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -36,17 +36,19 @@
"import geopandas as gpd\n",
"import jupyter_black\n",
"import numpy as np\n",
"import pandas as pd\n",
"from tqdm.auto import tqdm\n",
"import xarray as xr\n",
"\n",
"import hvplot.xarray # noqa, after xr\n",
"import hvplot.pandas # noqa, after pandas\n",
"# hvplot.xarray and hvplot.pandas are imported for their side effect:\n",
"# each registers a .hvplot accessor, on xarray and pandas objects\n",
"# respectively. Neither is referenced by name, so ruff sees them as\n",
"# unused. Import them after xarray and pandas.\n",
"import hvplot.xarray # noqa: F401\n",
"import hvplot.pandas # noqa: F401\n",
"\n",
"import pyPRMS\n",
"import pywatershed as pws\n",
"from pywatershed.plot import DomainPlot\n",
"from pywatershed.constants import zero\n",
"\n",
"ndays_run = 365 * 2\n",
"plot_height = 600\n",
Expand Down
2 changes: 0 additions & 2 deletions examples/07_mmr_to_mf6_chf_dfw.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -86,14 +86,12 @@
"import geopandas as gpd\n",
"import jupyter_black\n",
"import numpy as np\n",
"import pandas as pd\n",
"import pint\n",
"import pywatershed as pws\n",
"import xarray as xr\n",
"\n",
"import hvplot.xarray # noqa\n",
"\n",
"import pywatershed as pws\n",
"from pywatershed.utils.mmr_to_mf6_dfw import MmrToMf6Dfw\n",
"\n",
"jupyter_black.load()\n",
Expand Down
15 changes: 9 additions & 6 deletions examples/09_model_output.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,6 @@
"source": [
"import inspect\n",
"import pathlib as pl\n",
"from pprint import pprint\n",
"\n",
"import jupyter_black\n",
"import numpy as np\n",
Expand Down Expand Up @@ -682,6 +681,9 @@
"metadata": {},
"outputs": [],
"source": [
"# some excessive hvplot internal warnings to avoid here\n",
"import warnings\n",
"\n",
"plot_data = xr.merge(\n",
" [\n",
" output2.noi_arrays[\"node_outflows\"]\n",
Expand All @@ -699,14 +701,15 @@
" }\n",
")\n",
"\n",
"# some excessive hvplot internal warnings to avoid here\n",
"import warnings\n",
"\n",
"with warnings.catch_warnings():\n",
" warnings.simplefilter(\"ignore\") # Temporarily suppress all warnings\n",
" import pandas as pd\n",
" import hvplot.xarray # noqa, after xr\n",
" import hvplot.pandas # noaq, after pandas\n",
" # imported for their side effect: each registers a .hvplot\n",
" # accessor, on xarray and pandas objects respectively. Neither is\n",
" # referenced by name, so ruff sees them as unused. Import them\n",
" # after xarray and pandas.\n",
" import hvplot.xarray # noqa: F401\n",
" import hvplot.pandas # noqa: F401\n",
"\n",
" display(\n",
" plot_data.drop_vars(\n",
Expand Down
1 change: 0 additions & 1 deletion examples/developer/01_automated_testing.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,6 @@
"outputs": [],
"source": [
"import pywatershed\n",
"import os\n",
"import shutil\n",
"import subprocess\n",
"import pathlib as pl\n",
Expand Down
4 changes: 2 additions & 2 deletions examples/developer/02_process_models.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -283,7 +283,7 @@
" pywatershed.analysis.ModelGraph(\n",
" multi_proc_model, process_colors=palette, hide_variables=False\n",
" ).SVG(verbose=True)\n",
"except:\n",
"except Exception:\n",
" print(\"Sorry, this fails for no good reason on Mac ARM architecture\")"
]
},
Expand Down Expand Up @@ -412,7 +412,7 @@
" pywatershed.analysis.ModelGraph(\n",
" nhm, process_colors=palette, hide_variables=False\n",
" ).SVG(verbose=True)\n",
"except:\n",
"except Exception:\n",
" print(\"Sorry, this fails for no good reason on Mac ARM architecture\")"
]
},
Expand Down
2 changes: 0 additions & 2 deletions examples/model_loop_custom_output.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -30,14 +30,12 @@
"outputs": [],
"source": [
"from itertools import product\n",
"import math\n",
"import pathlib as pl\n",
"import shutil\n",
"import time\n",
"\n",
"import numpy as np\n",
"import pywatershed as pws\n",
"from tqdm import tqdm\n",
"import xarray as xr"
]
},
Expand Down
11 changes: 4 additions & 7 deletions examples/runoff_errors.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -29,8 +29,7 @@
"outputs": [],
"source": [
"import pathlib as pl\n",
"from pprint import pprint\n",
"from shutil import rmtree, copy2\n",
"from shutil import copy2\n",
"\n",
"import hvplot.xarray # noqa\n",
"from IPython.display import display\n",
Expand Down Expand Up @@ -99,11 +98,9 @@
"outputs": [],
"source": [
"def run_prms(binary: pl.Path, run_dir: pl.Path, skip_if_exists=False):\n",
" import shlex\n",
" import subprocess\n",
"\n",
" from pywatershed import CsvFile, Soltab\n",
" from pywatershed.parameters import PrmsParameters\n",
" from pywatershed import CsvFile\n",
"\n",
" if skip_if_exists and run_dir.exists():\n",
" print(\n",
Expand All @@ -127,7 +124,7 @@
" output_dir.mkdir()\n",
"\n",
" exe_command = f\"time ./{binary.name} nhm.control -MAXDATALNLEN 60000 2>&1 | tee run.log\"\n",
" result = subprocess.run(\n",
" subprocess.run(\n",
" exe_command,\n",
" shell=True,\n",
" # stdout = subprocess.PIPE,\n",
Expand Down Expand Up @@ -373,7 +370,7 @@
" assert (output_dir / f\"{vv}.nc\").exists()\n",
" try:\n",
" assert (input_dir_cp / f\"{vv}.nc\").exists()\n",
" except:\n",
" except AssertionError:\n",
" print(f\"********** {vv} not in input_dir_cp\")"
]
},
Expand Down
Loading
Loading