Locate network observations with a MIKE+ database

[jpalm3r]

Building a NodeObservation today means knowing which node each sensor sits on, which usually ends up as a hand-written SQL query in a notebook. A MIKE+ project already records that: m_Measurement says which file and item each measured timeseries lives in, m_Station says where in the network it belongs.

This adds a db argument to NodeObservation.from_multiple and ReachObservation.from_multiple that does the lookup.

quantity = "Pressure"

network = Network.from_epanet("model.res", quantities=quantity)
network_model = ms.NetworkModelResult(network, item=quantity)
obs = ms.NodeObservation.from_multiple(
    data="calibration.dfs0", db="model.sqlite", quantity=quantity
)
cc = ms.match(obs, network_model)

The mapping is keyed by item, not by location

One dfs0 item maps to exactly one location, but a location can carry several items — two pressure transmitters either side of a check valve, three flow meters on one pipe. So the lookup returns one row per item and one observation is created per row. The existing nodes={location: item} dict cannot express this, since a location can only appear once as a key; it still works, and its docstring now says what it cannot do.

Which class you get is decided by the database

m_Station.locationtype says whether a station sits on a node or a link, and it is load-bearing: some locationid values match both a node alias and a reach id, so trying node-first-then-reach would silently resolve the wrong one. Junctions and tanks become NodeObservation, links become ReachObservation, and a link carrying a chainage becomes a NodeObservation at that breakpoint. Asking one class for a quantity the database places on the other raises and names the class you want.

Contained behind a fixed contract

network/_mikeplus.py returns a fixed set of columns — item_name, name, location, kind, quantity. Table names, the join, the locationtype codes and the resitemname encoding stay inside that module, so a change to the database layout is a change to one file. obs.py never sees a MIKE+ concept.

network.py becomes network/__init__.py to make room for it. Pure move — every import in the repo is from modelskill.network import ... and is unaffected.

Other changes

• ReachObservation.from_multiple, mirroring the NodeObservation classmethod, which did not exist before.
• from_multiple(nodes=...) keys now accept aliases and (reach_id, distance) breakpoints. NodeObservation.at already took all three; only from_multiple advertised integers.
• Bug fix: Network.to_dataset() keyed its data variables by quantity name but never wrote it to the DataArray attrs, so NetworkModelResult reported Quantity.undefined() and skill tables labelled the model column with the observation's quantity. res1d and EPANET files carry a name without a unit, and Quantity.from_cf_attrs needs both, so it falls back to the name alone.

Notes for review

• Quantity comes from the database, not the data. Calibration dfs0 files routinely carry EUM type Undefined for every item, so the database is the only reliable source for the name; the unit still comes from the data.
• Items the database cannot place raise by default, separating the two causes — a station that exists but has no measurement registered for the file, and an item not in the database at all. on_missing="skip" builds from the rest.
• Tests run against a synthetic MIKE+ database built in a fixture, so nothing depends on a real project file.
• match() still does not compare an observation's quantity with the model's; that is a wider change, tracked in Quantity.is_compatible reports undefined quantities as incompatible #697.

Verified end to end against a real EPANET model and its MIKE+ database: 30 observations, 30 comparers.

[jpalm3r]

Closing unmerged. This branch splits in two, and both halves survive.

The MIKE+ lookup is on branch mikeplus-station-lookup, which is #702 plus this one feature. It is not part of the mikeio1d move and is not ready to ship on its own terms — #706 lists what needs more thought. The tests came along and grew, 453 lines here to 558 there, and source= was added on top.

One design decision from this PR did not survive the extraction, and is recorded on #706: here the lookup is network/_mikeplus.py, a module returning a fixed five-column contract so obs.py never sees a MIKE+ concept. On that branch it is a _MikePlusStationResolver class inside obs.py. The class docstring still claims the containment; the module boundary that enforced it is gone.

Everything else is in #702:

• The to_dataset() quantity bug. mikeio1d writes attrs={"long_name": str(q)} (Add a network module mikeio1d#247), and NetworkModelResult reads the attributes directly rather than through Quantity.from_cf_attrs, which needs a unit as well as a name and reports neither without both.
• ReachObservation.from_multiple.
• from_multiple(nodes=...) keys accepting aliases and (reach, distance) break points, with the docstring saying what the dict form cannot express.

network.py becoming network/__init__.py is moot: #702 deletes it.

The three review comments here are lint nits about ... overload bodies. They apply to mikeplus-station-lookup and can be taken in its own review.

#697 stays open; it was only referenced, not closed, here.