feat(agent): make the CGNAT range collision visible (metrics + startup detection) - #17
Open
wighawag wants to merge 3 commits into
Open
feat(agent): make the CGNAT range collision visible (metrics + startup detection)#17wighawag wants to merge 3 commits into
wighawag wants to merge 3 commits into
Conversation
Contributor
Author
Multicast and broadcast are unroutable on the mesh by design, but they were counted as `no_route`, the same label used when a genuine unicast mesh destination has no peer. LAN discovery apps beacon on every interface including ours, forever, at a steady rate. On one host `ableton-linkd` (224.76.78.75:20808) accounted for 1.27M `no_route` drops over 1d20h at ~8 packets/s, sourced from the mesh interface's own addresses. A packet capture showed zero unicast mesh destinations among them. That benign noise was read as evidence of a routing leak, which cost a full investigation to disprove. Classify multicast/broadcast as `reason="multicast"` at the drop site so `no_route` means what a reader assumes it means.
Direct mode puts a connected route for the whole mesh range on the tunnel
interface. Tailscale uses the same 100.64.0.0/10 and installs a
source-based anti-spoof rule:
-A ts-input -s 100.64.0.0/10 ! -i tailscale0 -j DROP
Because it matches on SOURCE, it also drops the host's own traffic to our
addresses, including over loopback. Measured on an affected host: 50 of 50
packets to the mesh IP and 50 of 50 to the MagicDNS resolver were dropped,
while a 127.0.0.1 control was untouched. The resolver stays listening and
never receives a query, so MagicDNS fails with no error and no log line.
Neither product reports any of this. The user sees "connected" on both and
traffic that does not work.
Detect at agent start that another interface holds an address overlapping
our mesh CIDR, name the likely owner, and say which of our own addresses
are consequently unreachable. Export it as a gauge so the condition is
visible to monitoring rather than only in a log line.
Detection is a pure function over plain interface data, so it is tested
without a host; the netdev path was verified separately against a machine
running both products. Observation only: nothing here changes routing, and
a failed check never blocks startup.
Captures the reproduction and what the evidence did and did not support, including that the originally reported no_route figure was disproven (it was multicast discovery noise) and that the anti-spoof drops were later confirmed causally with a TCP probe against a live peer. Notes that tunnetio#24 supersedes the proposal section, and which two defects it does not address.
wighawag
force-pushed
the
fix/mesh-cidr-collision-diagnostics
branch
from
September 6, 2026 15:14
80cd2b9 to
a776d25
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Diagnostics only. No wire format, no protocol change, no behaviour change to the datapath. Splits into four commits that can be dropped independently.
Fixes the reason the 100.64.0.0/10 collision with Tailscale was invisible, and then easy to misdiagnose once noticed.
Every drop reason gets its own counter. Of the 21 reasons the agent can report, only 6 were distinguishable; no_route, ipv6_unsupported, antispoof and 12 others all reported as
reason="other". A test scans the crate source for dropped_inc("...") call sites and fails if any reason is unregistered, so this cannot silently regress.
Multicast is counted separately from no_route. LAN discovery apps beacon on every interface forever; on one host that was 1.27M no_route drops from ableton-linkd alone, which was read
as evidence of a routing leak and took a packet capture to disprove.
Startup detection of another interface claiming our mesh CIDR, naming the likely owner and which of our own addresses are consequently unreachable, exported as a gauge.
Design doc recording the investigation, including what the evidence did and did not support.
Related: #15 fixes the MagicDNS symptom, #16 fixes an unrelated multicast leak found while investigating.
Verified with cargo fmt --all -- --check, cargo clippy --workspace --exclude tunnet-desktop --all-targets --all-features -- -D warnings, and cargo test --workspace --exclude tunnet-desktop
--all-features (433 passed, 0 failed). Each commit builds and tests green individually.