From 49b4bd0efbccf23a3cd296d8cc6edf1d695393a2 Mon Sep 17 00:00:00 2001 From: Josh Poole Date: Wed, 30 Sep 2026 14:05:34 +0100 Subject: [PATCH] 20260930 - Derive the RSPduo notch filters from the capture frequency config-merger now sets capture.device.rfNotch and dabNotch from the final capture.fc and fs, after forced.yml. A notch is on unless the capture band overlaps its stopband (RF 77-115 MHz, DAB 155-235 MHz). The derived value overwrites every overlay, so the true/true each node persisted in its first-boot user.yml needs no migration. At the shipped 503 MHz both stay on. default.yml keeps both keys: blah2 aborts on a missing key, and standalone and staging build config.yml from default.yml without config-merger. Co-Authored-By: Claude Opus 5.5 (1M context) --- README.md | 47 +++++++++++++- STANDALONE.md | 14 +++++ config-merger/script/merge_config.py | 49 +++++++++++++++ config-merger/test/test_merge_config.py | 82 +++++++++++++++++++++++++ config/default.yml | 3 + staging/README.md | 6 ++ staging/merge-config.sh | 1 + 7 files changed, 201 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 35c41a6..0b7e84f 100644 --- a/README.md +++ b/README.md @@ -87,7 +87,52 @@ Config-merger generates config files at startup by merging: default.yml → user.yml → forced.yml ``` -Later files override earlier ones. +Later files override earlier ones. A few values are then derived from the +merged result rather than taken from any file. The notch filters below are one. + +### RF notch filters + +The RSPduo has two hardware notch filters, set by `capture.device.rfNotch` and +`capture.device.dabNotch`. Config-merger derives both from the final +`capture.fc` and `capture.fs` on every merge, after `forced.yml` is applied, so +any value in `default.yml`, `user.yml` or `forced.yml` is overwritten. + +| Key | Stopband (SDRplay spec) | Rejection | +| --- | --- | --- | +| `rfNotch` (FM) | 77-115 MHz | 30 dB | +| `dabNotch` | 155-235 MHz | at least 20 dB | + +A notch is on unless the capture band `[fc - fs/2, fc + fs/2]` overlaps its +stopband (touching an edge counts as overlap). With the notch on, strong +broadcast signals in that band are kept out of the ADC. With the capture band +inside the stopband, the notch would reject the very signal being received, so +it is turned off. At the shipped 503 MHz both notches are on. + +Why the derived value always wins: on first boot config-merger seeds `user.yml` +with a full copy of `default.yml`, so every node already persists +`rfNotch: true` and `dabNotch: true`. Overwriting on every merge means those +copies need no migration, and a node moved to an FM or DAB illuminator gets the +right notches with no hand edit. To change a notch, change `fc`. + +A node whose `fc` sits inside a stopband has that notch turned off by this rule, +so its illuminator arrives 20-30 dB stronger than before. Gain set while the +notch was on may now overload the front end: lower it, or run Auto-Calibrate +again, after the change lands. + +If `fc` or `fs` is missing or not a number, the merged values are left as they +are (`default.yml` ships both keys as `true`, and blah2 aborts on a missing +key). blah2 cannot tune without a valid `fc` in any case. + +**Known limitation: live retune.** blah2 applies the notches only at +`sdrplay_api_Init()`. blah2-api's `POST /capture/retune` changes `fc` without +restarting blah2 and never touches the notches. retina-gui's Auto-Calibrate uses +it to probe candidate towers, so a candidate in a different notch band from the +starting `fc` is probed with the wrong notches (for example an FM tower probed +with the FM notch still on, which costs it up to 30 dB). The persisted result +is correct, because `/calibrate/apply` runs config-merger and restarts blah2. +Standalone and staging deployments do not run config-merger: see +[STANDALONE.md](STANDALONE.md#rf-notch-filters) and +[staging/README.md](staging/README.md). ### Editing Config on a Node diff --git a/STANDALONE.md b/STANDALONE.md index 6fb8b98..e21bf1f 100644 --- a/STANDALONE.md +++ b/STANDALONE.md @@ -142,6 +142,20 @@ For development, uncomment and edit the image versions in `.env` to quickly swit Edit `config/config.yml` to set your location, frequency, and SDR settings. See [blah2-arm documentation](https://github.com/offworldlabs/blah2-arm) for full config options. +### RF notch filters + +Standalone skips config-merger (`SKIP_CONFIG_MERGER=true`), so nothing derives +`capture.device.rfNotch` and `capture.device.dabNotch` from your frequency. The +copy of `default.yml` ships both as `true`, which suits UHF. If you change +`capture.fc` so the capture band `[fc - fs/2, fc + fs/2]` overlaps a stopband, +set that notch to `false` yourself, or it rejects the signal you are receiving: + +- `rfNotch`: 77-115 MHz (FM band) +- `dabNotch`: 155-235 MHz (DAB band) + +Keep both keys present: blah2 aborts if either is missing. See the +[README](README.md#rf-notch-filters) for the full rule. + ## Commands From the install location (`/opt/retina-standalone`): diff --git a/config-merger/script/merge_config.py b/config-merger/script/merge_config.py index 3ae2f7a..9cf60d7 100755 --- a/config-merger/script/merge_config.py +++ b/config-merger/script/merge_config.py @@ -8,6 +8,8 @@ Generates tar1090.env for tar1090-node if the tar1090 section exists in config. Generates retina-tracker.yaml for the retina-tracker sidecar if the retina_tracker section exists in config. +Derives the RSPduo DAB and RF notch filters from the final capture.fc and +capture.fs, overwriting whatever the merged files held (see derive_notches()). Takes as inputs: 1. Defaults directory - holds baked-in default.yml and forced.yml @@ -51,6 +53,14 @@ # first-boot copy rather than anyone's choice. See migrate_adsb_truth_server(). LEGACY_ADSB_TAR1090 = 'sfo1.retnode.com' +# RSPduo notch filter stopbands in Hz, from the SDRplay spec. The RF (FM) notch +# rejects 30 dB over 77-115 MHz, the DAB notch at least 20 dB over 155-235 MHz. +# See derive_notches(). +RF_NOTCH_MIN_HZ = 77_000_000 +RF_NOTCH_MAX_HZ = 115_000_000 +DAB_NOTCH_MIN_HZ = 155_000_000 +DAB_NOTCH_MAX_HZ = 235_000_000 + def get_node_id_from_mender(): """Read node_id from Mender device identity file (generated by mender-device-identity)""" @@ -203,6 +213,42 @@ def migrate_gain_reduction(config): config['capture']['device']['gainReduction'] = [gain, gain] +def derive_notches(config): + """Set capture.device.dabNotch and rfNotch from the final capture.fc and fs. + + A notch is on unless the capture band [fc - fs/2, fc + fs/2] overlaps its + stopband (touching an edge counts). Runs on the merged config, so the result + overwrites default.yml, user.yml and forced.yml alike. That is why the + true/true every node persisted in its first-boot user.yml needs no LEGACY_* + migration. Rationale and the live-retune caveat: README.md, "RF notch filters". + """ + try: + capture = config['capture'] + fc = capture['fc'] + fs = capture['fs'] + except (KeyError, TypeError): + return + + # With no usable fc or fs, leave the merged values: blah2 cannot tune anyway, + # and default.yml guarantees both keys exist. + if (not all(isinstance(v, (int, float)) and not isinstance(v, bool) for v in (fc, fs)) + or fc <= 0 or fs < 0): + print(f"Cannot derive notch filters from capture.fc={fc!r}, fs={fs!r}; leaving them as merged") + return + + device = capture.setdefault('device', {}) + if not isinstance(device, dict): + return + + low, high = fc - fs / 2, fc + fs / 2 + rf = not (low <= RF_NOTCH_MAX_HZ and high >= RF_NOTCH_MIN_HZ) + dab = not (low <= DAB_NOTCH_MAX_HZ and high >= DAB_NOTCH_MIN_HZ) + device['rfNotch'] = rf + device['dabNotch'] = dab + print(f"Derived notch filters for {low / 1e6:g}-{high / 1e6:g} MHz: " + f"rfNotch={str(rf).lower()}, dabNotch={str(dab).lower()}") + + def migrate_doppler_span(user): """Drop a Doppler span the user never chose, so default.yml can change it. @@ -463,6 +509,9 @@ def main(): # Migrate legacy field formats forward (e.g. scalar -> per-tuner gainReduction) migrate_gain_reduction(config) + # Notches follow the final frequency, whatever any overlay said + derive_notches(config) + # Write merged config to output print(f"Writing merged config to {output_config_path}") os.makedirs(os.path.dirname(output_config_path), exist_ok=True) diff --git a/config-merger/test/test_merge_config.py b/config-merger/test/test_merge_config.py index d87d02c..a4ac819 100755 --- a/config-merger/test/test_merge_config.py +++ b/config-merger/test/test_merge_config.py @@ -1048,6 +1048,88 @@ def test_upstream_chain_falls_back_to_adsblol_when_unset(self): self.assertIn('ADSB_UPSTREAMS=https://api.adsb.lol\n', env) + def merge_notches(self, capture, user_capture=None): + """Merge the given capture section and return the resulting device block""" + self.write_yaml(os.path.join(self.defaults_dir, 'default.yml'), {'capture': capture}) + self.write_yaml(os.path.join(self.defaults_dir, 'forced.yml'), {}) + user_yml = os.path.join(self.config_dir, 'user.yml') + if user_capture is not None: + self.write_yaml(user_yml, {'capture': user_capture}) + elif os.path.exists(user_yml): + os.remove(user_yml) # else the first-boot copy of a previous call wins + output = self.run_merge() + return self.read_yaml(output)['capture']['device'] + + def test_notches_on_for_uhf(self): + """The shipped 503 MHz leaves both notches on, as they were before derivation""" + with open(os.path.join(os.path.dirname(os.path.dirname(os.path.dirname(__file__))), + 'config', 'default.yml')) as f: + capture = yaml.safe_load(f)['capture'] + device = self.merge_notches(capture) + self.assertEqual(capture['fc'], 503000000) + self.assertIs(device['rfNotch'], True) + self.assertIs(device['dabNotch'], True) + + def test_rf_notch_off_in_fm_band(self): + device = self.merge_notches({'fc': 98_000_000, 'fs': 2_000_000, + 'device': {'rfNotch': True, 'dabNotch': True}}) + self.assertIs(device['rfNotch'], False) + self.assertIs(device['dabNotch'], True) + + def test_dab_notch_off_in_dab_band(self): + device = self.merge_notches({'fc': 204_640_000, 'fs': 2_000_000, + 'device': {'rfNotch': True, 'dabNotch': True}}) + self.assertIs(device['rfNotch'], True) + self.assertIs(device['dabNotch'], False) + + def test_notch_band_edges_use_fs(self): + """A capture band that reaches into a stopband turns that notch off""" + # 116 MHz +/- 1 MHz touches the RF stopband's 115 MHz top edge + device = self.merge_notches({'fc': 116_000_000, 'fs': 2_000_000, 'device': {}}) + self.assertIs(device['rfNotch'], False) + # the same fc with a narrower fs clears it + device = self.merge_notches({'fc': 116_000_000, 'fs': 1_000_000, 'device': {}}) + self.assertIs(device['rfNotch'], True) + # 154 MHz +/- 1 MHz reaches the DAB stopband's 155 MHz bottom edge + device = self.merge_notches({'fc': 154_000_000, 'fs': 2_000_000, 'device': {}}) + self.assertIs(device['dabNotch'], False) + self.assertIs(device['rfNotch'], True) + device = self.merge_notches({'fc': 154_000_000, 'fs': 1_000_000, 'device': {}}) + self.assertIs(device['dabNotch'], True) + + def test_notches_overwrite_user_yml(self): + """The persisted first-boot true/true and any hand-set value lose to fc""" + device = self.merge_notches( + {'fc': 503_000_000, 'fs': 2_000_000, 'device': {'rfNotch': True, 'dabNotch': True}}, + user_capture={'fc': 98_000_000, 'device': {'rfNotch': True, 'dabNotch': False}}) + self.assertIs(device['rfNotch'], False) + self.assertIs(device['dabNotch'], True) + + def test_notches_follow_forced_fc(self): + """Derivation runs after forced.yml, so a forced fc decides""" + self.write_yaml(os.path.join(self.defaults_dir, 'default.yml'), + {'capture': {'fc': 503_000_000, 'fs': 2_000_000, 'device': {}}}) + self.write_yaml(os.path.join(self.defaults_dir, 'forced.yml'), {'capture': {'fc': 204_640_000}}) + device = self.read_yaml(self.run_merge())['capture']['device'] + self.assertIs(device['dabNotch'], False) + self.assertIs(device['rfNotch'], True) + + def test_notches_left_alone_without_fc(self): + """No usable fc: the merged values stand and the merge still succeeds""" + device = self.merge_notches({'fs': 2_000_000, 'device': {'rfNotch': True, 'dabNotch': False}}) + self.assertIs(device['rfNotch'], True) + self.assertIs(device['dabNotch'], False) + + device = self.merge_notches({'fc': 'abc', 'fs': 2_000_000, + 'device': {'rfNotch': False, 'dabNotch': True}}) + self.assertIs(device['rfNotch'], False) + self.assertIs(device['dabNotch'], True) + + def test_no_capture_section_adds_no_notches(self): + self.write_yaml(os.path.join(self.defaults_dir, 'default.yml'), {'network': {'ip': '0.0.0.0'}}) + self.assertNotIn('capture', self.read_yaml(self.run_merge())) + + class TestAtomicWrites(unittest.TestCase): """The compose .env must be replaced whole, never rewritten in place. diff --git a/config/default.yml b/config/default.yml index e31ec00..103c725 100644 --- a/config/default.yml +++ b/config/default.yml @@ -7,6 +7,9 @@ capture: bandwidthNumber: 0 gainReduction: [40, 40] #20-59 dB each, [reference, surveillance], higher is less gain lnaState: 4 #1-9, higher is more gain + # Derived by config-merger from fc and fs on every merge, overwriting any + # value set here or in user.yml. Kept so config.yml always has both keys + # (blah2 aborts without them). See README.md, "RF notch filters". dabNotch: true rfNotch: true replay: diff --git a/staging/README.md b/staging/README.md index c3d769c..267271f 100644 --- a/staging/README.md +++ b/staging/README.md @@ -123,6 +123,12 @@ This creates `config/config.yml` by merging: **Optional**: Edit `config/user.yml` to override specific settings. +`merge-config.sh` is a plain merge, not config-merger, so it does not derive the +RF and DAB notch filters from `capture.fc`. The `true`/`true` from `default.yml` +passes through, which is right for UHF. If you override `fc` into 77-115 MHz or +155-235 MHz, set `rfNotch` or `dabNotch` to `false` in `config/user.yml` as well. +See the main [README](../README.md#rf-notch-filters). + ### Step 4: Start Services ```bash diff --git a/staging/merge-config.sh b/staging/merge-config.sh index f09350d..d67dc71 100755 --- a/staging/merge-config.sh +++ b/staging/merge-config.sh @@ -1,6 +1,7 @@ #!/usr/bin/env bash # Manually merge config files for staging # Combines: default.yml → user.yml → forced.yml = config.yml +# Unlike config-merger, this does not derive the RF/DAB notch filters from fc. set -euo pipefail SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"