Skip to content

feat(routing): add L3 'reject' action that sends ICMP port-unreachable for instant QUIC downgrade - #1135

Open
itoywh wants to merge 7 commits into
daeuniverse:mainfrom
itoywh:feat-l3-reject-icmp-quic-downgrade
Open

itoywh wants to merge 7 commits into
daeuniverse:mainfrom
itoywh:feat-l3-reject-icmp-quic-downgrade

Conversation

@itoywh

@itoywh itoywh commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

Background

dae's existing block action is implemented as TC_ACT_SHOT in the dataplane
(control/kern/tproxy.c, all OUTBOUND_BLOCK sites), i.e. a silent drop with no
ICMP
. When a QUIC client's Initial is blocked this way it receives no response
and must wait out the handshake timeout (observed ~8 s on a single-leg probe)
before falling back to TCP/h2.

How OpenClash and sing-box achieve instant downgrade: both answer a blocked
UDP/443 with an ICMP Port Unreachable, so the client's connected-UDP socket
gets ECONNREFUSED immediately and the QUIC stack fails the handshake per
RFC 9000 §9.3, falling back to h2 in milliseconds.

Tool Blocking action What it sends for blocked UDP/443 Client result
sing-box action: reject, method: default TCP → RST, UDP → ICMP Port Unreachable (method: drop is the silent one) immediate ECONNREFUSED → sub-second h2 downgrade
Clash Meta / OpenClash (TUN) TUN-stack REJECT ICMP Port Unreachable synthesized inside the TUN device back to the client same
dae (before this PR) block silent TC_ACT_SHOT QUIC handshake timeout

sing-box also adds backpressure: after 50 rejects / 30 s it silently drops,
preventing an ICMP storm — this PR borrows the same idea (§ Full Changelogs).

This PR adds an L3 routing action reject that mirrors that behavior: for UDP it
reuses the existing control-plane handoff path and injects an ICMP Port
Unreachable, instead of silently dropping.

Checklist

Full Changelogs

  • Add L3 routing action reject. For UDP, the dataplane no longer SHOTs but
    hands the packet to the control plane via the existing
    redirect_lan_packet_to_control_plane() path; the control plane
    (control/icmp_inject_linux.go) injects an ICMPv4 type 3/code 3 (or
    ICMPv6 type 1/code 4) Port Unreachable back to the client, quoting the
    original IP header + first 8 bytes of the original UDP header (RFC 792 /
    RFC 4443). This makes blocked QUIC handshakes fail instantly and fall back to
    TCP/h2, instead of waiting for the QUIC handshake timeout as block does.
  • Backpressure. A per-client gate (50 replies / 30 s,
    control/icmp_inject_linux.go) falls back to silent drop past the budget,
    mirroring sing-box's reject(method:default) storm protection.
  • Reuses dae's existing infrastructure. reject = a new reserved outbound
    OutboundReject = 0xFB (top of the user-defined range; does not collide with
    user group indices — the control-plane guard is len(outbounds) > OutboundUserDefinedMax and user groups top out at 0xFA). The UDP handoff
    reuses redirect_lan_packet_to_control_plane(); the raw-socket send reuses the
    same primitives as control/raw_udp_linux.go.
  • v1 scope (intentional):
    • TCP reject is a silent SHOT (same as block) in this PR; only UDP
      injects ICMP, which is the QUIC-relevant path. TCP-RST is left as a follow-up.
    • The ICMP source is the dae host's own egress address (kernel-chosen), not
      the spoofed server IP. The client correlates the error by the embedded
      original-datagram header (RFC 9000 §9.3), so source spoofing — and the
      rp_filter headaches that come with it — are avoided.

Example config (upgrade an existing rule):

# before: silent drop, waits for QUIC timeout
l4proto(udp) && dport(443) -> block

# after: ICMP Port Unreachable, instant h2 downgrade
l4proto(udp) && dport(443) -> reject

Issue Reference

No existing issue. Motivation grew out of the QUIC/h3 suppression work where
l4proto(udp) && dport(443) -> block left a multi-second downgrade gap for
clients that already know h3 via Alt-Svc (the DNS-layer qtype(https) -> reject
cannot catch those).

Test Result

Verified in this environment (macOS dev machine):

  • GOOS=linux GOARCH=amd64 CGO_ENABLED=0 go build -tags dae_stub_ebpf ./... — passes.
  • go vet -tags dae_stub_ebpf ./control/ ./cmd/ ./common/consts/ ./component/routing/ — passes (this also type-checks the new _test.go).
  • gofmt -l on all touched files — clean.
  • New unit tests control/icmp_inject_linux_test.go for the ICMPv4/v6 message
    builders: assert type/code, the zeroed unused field, that the quoted original
    IP+UDP headers are embedded verbatim, and a valid ICMP checksum (verifies
    to 0x0000). These run in CI on Linux without root, so they execute in the
    normal unit-test job. The checksum/quote logic was additionally validated with a
    standalone reproduction.

Real-machine e2e (Linux, 2026-10-01): the ICMP injection primitive was
exercised on a real Linux host (PVE, Linux 7.0.14-20-pve x86_64, root) — not
just unit-tested. A cross-compiled in-package test opened a raw ICMP socket and
called sendICMPPortUnreachable with a synthetic original UDP datagram, while a
concurrent tcpdump -i any captured the actual on-wire packet. Decoded:

IPv4 — ICMP Destination Port Unreachable (type 3 / code 3):
  127.0.0.1 > 127.0.0.1: ICMP 198.51.100.7 udp port 443 unreachable, length 36
    quoted original: 203.0.113.5.12345 > 198.51.100.7.443: UDP, length 0
    hex: 0303 <icmp cksum ok> | 4500 001c 1234 0000 4011 ...
         cb00 7105 (203.0.113.5)  c633 6407 (198.51.100.7)
         3039 (sport 12345) 01bb (dport 443) 0008 (udp len 8)
IPv6 — ICMPv6 Port Unreachable (type 1 / code 4):
  ::1 > ::1: ICMP6, destination unreachable, unreachable port, 2001:db8::7 udp port 443
    hex: 0104 <icmp6 sum ok — kernel-verified> | 6000 0000 0008 1140 (IPv6 hdr)
         2001:0db8::5 ... 2001:0db8::7 ... 3039 01bb 0008

tcpdump itself reports the ICMPv6 checksum as [icmp6 sum ok], and the ICMPv4
checksum is validated by tcpdump (no bad-cksum on the ICMP itself — only the
quoted inner IP header cksum is 0, which is just echoed payload and intentionally
unused). This proves the reject action emits a wire-valid ICMP Port
Unreachable
that a QUIC client correlates via the quoted header (RFC 9000 §9.3)
and fails the handshake immediately.

Not verified here (documented, not hidden):

  • The full dataplane chain on a live router — a real client behind dae whose
    UDP/443 packet is classified reject by eBPF, handed off to the control plane,
    and answered with the ICMP. Running dae run as a TProxy gateway on the PVE host
    risks dropping the host's own networking, so that path was deliberately not
    exercised here. The eBPF OUTBOUND_REJECT handoff sites are code-reviewed
    against the existing OUTBOUND_BLOCK handling they parallel, and CI's eBPF build
    job compiles tproxy.c.
  • Request to maintainers: please trigger the eBPF build job, and add the
    tested / documentation labels if the change looks good (a non-org member
    cannot add them, which is why the Governor checks will otherwise show failure).

…e for instant QUIC downgrade

Background
----------
dae's existing "block" action is implemented as TC_ACT_SHOT in the dataplane
(control/kern/tproxy.c, 5 sites: OUTBOUND_BLOCK), i.e. a silent drop with no
ICMP. When a QUIC client's Initial is blocked this way it sees no response and
must wait out the handshake timeout before falling back to TCP/h2. By contrast,
OpenClash and sing-box (TUN) answer a blocked UDP/443 with an ICMP Port
Unreachable, so the connected-UDP socket gets ECONNREFUSED immediately and the
QUIC stack fails the handshake per RFC 9000 §9.3 — a sub-second downgrade.

What this adds
--------------
A new reserved outbound "reject" (OutboundReject = 0xFB, the top of the
user-defined range, so it collides with no user group index since the control
plane guard is `len(outbounds) > OutboundUserDefinedMax` and user groups top out
at 0xFA). For UDP, the dataplane no longer SHOTs but hands the packet to the
control plane via the existing redirect_lan_packet_to_control_plane() path,
where control/icmp_inject_linux.go injects an ICMPv4 type 3/code 3 (or ICMPv6
type 1/code 4), quoting the original IP header + first 8 bytes of the original
UDP header (RFC 792 / RFC 4443). A per-client backpressure gate
(50 replies / 30s, control/icmp_inject_linux.go) falls back to silent drop past
the budget, mirroring sing-box's reject(method:default) storm protection.

v1 scope notes
--------------
- TCP "reject" is a silent SHOT (same as block) in this PR; only UDP injects
  ICMP, which is the QUIC-relevant path. TCP-RST is left as a follow-up.
- The ICMP source is the dae host's own egress address (chosen by the kernel),
  not the spoofed server IP. The client correlates the error by the embedded
  original-datagram header (RFC 9000 §9.3), so source spoofing — and the
  rp_filter headaches that come with it — are avoided.

Files
-----
- common/consts/ebpf_sync_spec.json: add REJECT=251; regen ebpf_generated.go +
  control/kern/ebpf_sync_defs.h; OutboundReject.String() in common/consts/ebpf.go.
- cmd/validate.go, control/routing_matcher_builder.go, component/routing/sniff_punt.go,
  control/tcp_sniff_policy.go: allow "reject" as a routing action / reserved outbound.
- control/kern/tproxy.c: UDP LAN-ingress reject -> control-plane handoff;
  TCP-established + both WAN-egress reject -> SHOT (v1).
- control/udp_ingress_task.go: detect OutboundReject after routing lookup and
  inject ICMP, then drop.
- control/icmp_inject_linux.go (+ _test.go, icmp_inject_stub.go): injection +
  unit tests for the ICMP message builders.
@itoywh
itoywh requested a review from a team as a code owner October 1, 2026 15:51
itoywh added 6 commits October 1, 2026 23:54
The v1 implementation injects the ICMP port-unreachable from the dae
host's own egress address (kernel-chosen), not spoofed from the
destination. Update the handoff-site comment to avoid misleading
reviewers. Matches design doc deviation #1.
The eBPF handoff delivers the full L2 frame to the control plane, so
data[0] is the Ethernet destination MAC, not the IP version nibble.
Reading data[0]>>4 returned 0/12 (MAC bytes) and the inject aborted with
'unsupported IP version', causing reject to silently drop instead of
sending ICMP port-unreachable (no instant QUIC downgrade).

Add linkHeaderLen() which inspects the EtherType (14 bytes for Ethernet,
18/22 with 802.1Q/802.1ad VLAN tags) and slice past it before building
the ICMP message. Matches dae's controlPlaneCore.linkHdrLen default of
consts.LinkHdrLen_Ethernet (14) for 'ether' interfaces.

Refs: PR daeuniverse#1135
The tproxy UDP data plane hands the control plane the UDP *payload*
only (ReadBatch on a tproxy socket delivers QUIC bytes, not an
L2/L3 frame). The previous attempt to parse an L2/IP header out of
the buffer was therefore impossible and produced 'unsupported IP
version' errors on every rejected packet (silent drop → no
downgrade).

Rewrite sendICMPPortUnreachable to rebuild the RFC 792/4443 quoted
original-datagram header from the routing 4-tuple we already
resolved in processPacket:
  - quoted IP src  = client (convergeSrc), the datagram's source
  - quoted IP dst  = originalDst (realDst), the rejected target
  - quoted UDP src port = client.Port(), dst port = originalDst.Port()
so the client's QUIC stack correlates the error and falls back to
TCP (RFC 9000 §9.3).

Also:
  - drop the (impossible) L2/IP parse path entirely
  - add per-family guards in build* so a mismatch is a hard error
  - fix errcheck (defer unix.Close) to pass go-lint
  - align OUTBOUND_BLOCK || OUTBOUND_REJECT continuations in tproxy.c
    to satisfy checkpatch (ebpf-lint / Kernel Test gate)
…c parens

- Replace Addr().AsSlice() with Addr().As4()/As16() assigned to a local
  before slicing. On some Go toolchains AsSlice() returns a 16-byte
  IPv4-mapped slice for v4 addrs, so copy(ip[12:16], ...) copied the leading
  zeros and produced a 0.0.0.0 quoted source (CI Go 1.26.8). The local
  [4]/[16] array is addressable and always the right width.
- Align the OUTBOUND_REJECT continuation in tproxy.c to satisfy checkpatch
  PARENTHESIS_ALIGNMENT (col 19 -> col 26).
…isfy checkpatch

The previous attempt indented line 2960 with 2 tabs + 5 spaces (col 21),
which checkpatch's PARENTHESIS_ALIGNMENT rejects. The correct alignment is
3 tabs + 5 spaces (col 29), matching the open parenthesis after
unlikely() on the preceding line. This unblocks the ebpf-lint /
Kernel Test check in CI.
…erated.go

- control/icmp_inject_linux_test.go: the ICMPv4 message is an 8-byte header
  followed by the 28-byte quoted datagram, so the quoted IP source/dest sit at
  msg[20:24]/msg[24:28], not msg[12:16]/msg[16:19]. The old offsets read the
  IP total-length/id fields (0.0.0.0), making the unit test fail even though
  the builder was correct. Mirror the msg[8:36] indexing the v6 test already uses.
- cmd/generators/gen_ebpf_sync/main.go + common/consts/ebpf_generated.go: the
  eBPF sync generator now emits the repo SPDX header on the generated .go file,
  so regenerating keeps ebpf_generated.go compliant with the SPDX lint gate
  while still matching generator output for ebpf-lint.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant