From 3212a86d560cbd2365cdacda816823cb4cb24dc8 Mon Sep 17 00:00:00 2001 From: EvaTheSalmon <18466855+EvaTheSalmon@users.noreply.github.com> Date: Wed, 14 Oct 2026 17:37:14 +0300 Subject: [PATCH 01/32] docs(plans): add xray config backend plan Co-Authored-By: Claude Opus 5.5 (1M context) --- docs/plans/20261014-xray-config-backend.md | 511 +++++++++++++++++++++ 1 file changed, 511 insertions(+) create mode 100644 docs/plans/20261014-xray-config-backend.md diff --git a/docs/plans/20261014-xray-config-backend.md b/docs/plans/20261014-xray-config-backend.md new file mode 100644 index 0000000..26ed3d8 --- /dev/null +++ b/docs/plans/20261014-xray-config-backend.md @@ -0,0 +1,511 @@ +# Xray backend from the operator's config + +## Overview + +- usher gains an Xray backend whose inbounds are whatever the operator defines in Xray's own + server config. usher reads that config, pushes users into the inbounds through Xray's + `HandlerService`, and writes per user and inbound a full client config (`.json`) and a share + link (`.txt`). +- Supported protocols: VLESS and Hysteria2; VMess and Trojan as table entries covered by unit and + golden tests only. Shadowsocks and WireGuard are out of scope. +- Supported shapes: transports raw/tcp (header none), ws, grpc, httpupgrade, xhttp, hysteria; + security none, tls, reality; finalmask salamander (Hysteria2) and a synthesized udphop for port + hopping. Anything else fails that inbound with an error that names the field. +- TLS cover is Reality (no own certificate) or a certificate a public CA issued for the operator's + domain. A pinned self-signed certificate is an explicit opt-in. +- Xray-core generates no client configs or share links (verified 2026-10-14 against v26.9.9 and + the 26.10.10 commit: no `vless://`, `hysteria2://` or similar in the source; the CLI commands in + `main/commands/all/` generate keys and talk to the API). The client side is usher's job, as it + is a panel's job in 3x-ui and Marzban. +- Breaking changes: `USHER_AWG_SOCKET` becomes required for the awg backend, `users.json` moves to + version 2. usher is below 1.0, so they ship in a minor release with no breaking marker in commit + headers. + +## Context (from discovery) + +usher on `master` (`876b208`): +- `cmd/usher/main.go:22` hardcodes the awg socket. +- `internal/config/config.go:13-14`: user names `{0,31}`, interface names + `^[A-Za-z0-9_][A-Za-z0-9_=+.-]{0,14}$`. +- `internal/state/state.go:107` rejects every version other than 1. +- `internal/clients/mirror.go:22-24` hardcodes `.conf` in `Path`. +- `internal/watch` polls `usher.yml` only; any other change is seen at the refill tick. + +Branch `xray-driver` (parked, draft PR #2) holds reusable commits: +- seam and awg: `bd02bac` (`USHER_AWG_SOCKET`), `00113e4` (seam; moves `internal/clients/render.go`, + `internal/pass/enroll.go`, `fake_test.go`, `testdata/phone.conf` into `internal/awg`, edits + `internal/pass/settings.go`), `f409cac` (shared Open adapter, AWG dialer), `7100137` (unexport + render). +- state: `64dd6b0` (xray group, version 2), `69a3ba5` (validator renames). +- e2e: `c5a9f2c` (Xray e2e suite), `e47b4cc` (freedom outbound reaches the target), `0993e07` + (`test/e2e/e2eutil`, also rewrites `test/e2e/e2e_test.go`). +- `internal/pass/pass.go` there marks an interface that fails in `Prepare` as failed, skips its + `Apply` and keeps its files (lines 71-117). + +Xray-core at the commit libxray requires (`v1.260327.1-0.20261010092107-701af60772cd`, release +26.10.10 per `core/core.go:20-24`), libxray `v1.261010.0` and reality +`v0.0.0-20260921001439-3c98159dee38`: +- Eight inbound protocols implement `proxy.UserManager`: vless, vmess, trojan, shadowsocks, + shadowsocks_2022, hysteria, wireguard, masque (`proxy/masque/server.go:225`). +- Accounts: `vless.Account{id, flow, encryption}`, `vmess.Account{id}`, `trojan.Account{password}`, + `hysteria/account.Account{auth}`. `vmess.MemoryAccount.ToProto` always adds `SecuritySettings` + (`proxy/vmess/account.go:33-45`); the others echo what was sent. +- `app/proxyman/command/command.go:138`: `GetInboundUsers` with an email answers `[nil]` for a + user that does not exist; without an email it lists all users. +- Hysteria server: `infra/conf/hysteria.go:39-75` (`version` must be 2; `clients`, or its alias + `users`, of `{auth, email, level}`). Salamander is a `finalmask.udp` entry + (`infra/conf/transport_finalmask.go:83`); with `packetSize` it builds a different framing + (`transport_finalmask.go:733-740`). `transport/internet/finalmask/udphop/config.go:15`: udphop is + client only; a hop range lives in host DNAT, outside Xray. +- `transport/internet/hysteria/hub.go:62-68`: the hub asks the user validator per request; with + empty `clients` and empty server `auth` it rejects everyone. +- `infra/conf/serial/loader.go:55-57`: a plain `json.Decoder`, so the `x-usher` key is ignored; + `infra/conf/json/reader.go` accepts `//`, `/* */` and `#` comments. +- `infra/conf/transport_internet.go:16-45,80-82`: aliases `method` for `network`, `splithttp` for + `xhttp`, `websocket` for `ws`, `tcp` for `raw`, `splithttpSettings` and `tcpSettings` for the + settings blocks. +- `infra/conf/transport_security.go:258-278`: `TLSCertConfig.Build()` reads `certificateFile` and + `keyFile` from disk. `transport_security.go:361-363`: `allowInsecure` is a removed feature and + fails config load. `infra/conf/trojan.go:73-74`: a Trojan `flow` fails the build. +- `transport/internet/tls/config.go:609-620`: the Xray client verifies against its bundled Mozilla + roots unless `useSystemCA` is set. `tls/config.go:97-130`: certificate files are re-read every + 3600 s and swapped in when they change. +- `transport/internet/splithttp/hub.go:143-154`: the xhttp server rejects a request whose padding, + session id or seq placement differs from its config. `infra/conf/transport_method.go:317-326`: + `extra` replaces every xhttp field except host, path and mode. +- `transport/internet/system_listener.go:128-165`: a unix listener never unlinks an existing + socket path and holds `flock(LOCK_EX)` on `.lock` while it listens; + `app/commander/commander.go:91-95` fails core start on the bind error. +- reality `tls.go:212`: `serverNames` is an exact-match map. `tls.go:216-235`: a ClientHello + without an `X25519MLKEM768` key share is not authenticated and goes to the target. +- libxray `share/generate_share.go:30-41`: a failing outbound is skipped and the only error is the + constant `no valid outbounds`. `share/hysteria2.go:235-255`: refuses salamander `packetSize`; + accepts udphop only as `mode: "intervalLocal,intervalRemote"`, `interval` with From equal to To + and at least 5, no `remoteIPs`, no `sockopt`, udphop last in `udp` and salamander first. + +Spike, measured 2026-10-14 on a Linux host with `ghcr.io/xtls/xray-core:26.10.10`, a throwaway +Go tool and docker-network clients (Xray 26.10.10 and sing-box 1.14.3): +- Xray starts with `x-usher` blocks, empty `clients` and a comment in the file. +- A user added through `AlterInbound` works at once on every inbound: VLESS raw+Reality+vision, + five xhttp+TLS variants, ws+TLS, Hysteria2+TLS+salamander, VMess ws, Trojan TLS. +- After `RemoveUser`, new connections fail on every inbound. An open Hysteria2 QUIC connection + keeps working until the client reconnects. +- The plain-copy xhttp mirror (path, host, mode) fails against a server with + `sessionIDPlacement: header`, `xPaddingObfsMode` or `xPaddingBytes`; the full block, or the full + block packed into `extra` in both `.json` and `.txt`, succeeds. +- With a CA the client trusts, `.json` and `.txt` work for every TLS inbound without a pin. +- The Xray client connects to Reality with fingerprints `chrome`, `firefox` and `safari`; `ios`, + `android`, `edge`, `360`, `qq`, `random` and `randomized` fail (three runs each for the random + ones). +- sing-box 1.14.3 works with VLESS ws, Trojan, VMess and Hysteria2+salamander. It fails Reality + against this server on every uTLS fingerprint (the server debug output shows no auth key) and + has no xhttp transport. +- `GetInboundUsers` echo equals the sent account for VLESS, Trojan and Hysteria; VMess returns + `security_settings:{type:AES128_GCM}` added. +- `docker kill -s KILL` on Xray, then start: `Failed to start: listen unix /run/xray/api.sock: + bind: address already in use`. The image has no shell. The uid that owns the socket directory + can unlink the stale socket, after which Xray starts. +- With inline certificates, `Build()` of every inbound in the file is `proto.Equal` to + `ListInbounds` (receiver and proxy settings; runtime users do not appear in the listing). + Editing `shortIds` or a port in the file makes the receiver differ. File certificates were not + measured. +- After `docker compose restart xray` with a 30 s refill tick, Reality was back after 28 s and + Hysteria2 after 32 s. +- `go list -deps` gives 711 packages; the static stripped tool binary is 34 MB. + +## Development Approach + +- **testing approach**: Regular (code first, then tests) +- complete each task fully before moving to the next +- make small, focused changes +- **CRITICAL: every task MUST include new/updated tests** for code changes in that task + - tests are not optional - they are a required part of the checklist + - write unit tests for new functions/methods + - write unit tests for modified functions/methods + - add new test cases for new code paths + - update existing test cases if behavior changes + - tests cover both success and error scenarios +- **CRITICAL: all tests must pass before starting next task** - no exceptions +- **CRITICAL: update this plan file when scope changes during implementation** +- run tests after each change +- backward compatibility: kept for `usher.yml` and for reading `users.json` version 1; broken for + the environment and for rolling back to an image that only knows version 1 + +## Testing Strategy + +- **unit tests**: table-driven, beside the source; gRPC fakes as structs, as `AGENTS.md` describes. +- **golden tests**: one input inbound (server JSON plus `x-usher`) and its expected client `.json` + and `.txt` per supported shape. +- **leak test**: no file under `clients/`, no log line and no drift reason contains `privateKey`, + `mldsa65Seed`, `certificateFile`, `keyFile`, a `key` PEM, `sockopt` or a server `fallbacks` + entry. +- **e2e**: tag `e2e`, package `test/e2e/xray`, Docker only. The server certificate comes from a + test CA the test generates; the Xray client container trusts it through `SSL_CERT_FILE` plus + `useSystemCA`, which the test adds to the client config (the Xray client ignores the system + store by default). Each case connects with the rendered `.json` and with the `.txt` turned back + into a config by libxray `ConvertShareLinksToXrayJson`. + +## Acceptance Evidence + +Today `master` has no Xray backend: naming an Xray tag in `usher.yml` logs "interface is not +reported by awg-grpc" and nothing reaches Xray. + +1. `go test -race -count=1 ./...` and `scripts/format.sh` pass. +2. Golden tests: `go test ./internal/xray/ -run Golden` passes for VLESS+raw+Reality, + VLESS+xhttp+TLS with non-default fields, VLESS+xhttp+TLS with a server `extra`, VLESS+ws+TLS, + Hysteria2+TLS+salamander, Hysteria2 with `hop_ports`, Hysteria2 with `trust: pin`, VMess+ws+TLS + and Trojan+raw+TLS. Every golden `.json` outbound builds with `conf.OutboundDetourConfig.Build()` + and every golden `.txt` parses back through `ConvertShareLinksToXrayJson`. +3. Xray e2e: `go test -tags e2e -count=1 -v ./test/e2e/xray/` passes with four inbounds + (VLESS+raw+Reality; VLESS+xhttp+TLS with `sessionIDPlacement: header` and `xPaddingBytes`; + VLESS+ws+TLS with `certificateFile`; Hysteria2+TLS+salamander with an inline certificate) and + two users: + - an HTTP request through each user's `.json` and `.txt` returns 200; + - after one user leaves `usher.yml`, new connections through that user fail and the other user + still works; + - after `docker compose restart xray`, connections work again within 45 s; + - after `docker kill -s KILL` of Xray, the restart policy restarts it, it fails on the stale + socket until usher unlinks it at its next tick, and connections work again within 90 s; + - changing `shortIds` in the mounted config without restarting Xray fails that inbound with a + drift reason at the next tick, and a user removed from `usher.yml` while that drift is present + is still removed from Xray; + - rewriting the `certificateFile` with a new leaf from the same CA causes no drift. +4. awg e2e: `go test -tags e2e -count=1 -v ./test/e2e/` still passes on a host with the module. +5. Manual smoke on a public host: import one VLESS+Reality link and one Hysteria2 link into + v2rayNG and open a page through each. + +## Progress Tracking + +- mark completed items with `[x]` immediately when done +- add newly discovered tasks with ➕ prefix +- document issues/blockers with ⚠️ prefix +- update plan if implementation deviates from original scope +- keep plan in sync with actual work done + +## Solution Overview + +- **Ownership of Xray's JSON.** The operator (delivery) owns one server `config.json`: inbounds, + outbounds, routing, `api`, certificates, Reality keys, plus a per-inbound `x-usher` block with + client hints. Xray reads it at start; usher reads the same file read-only. usher owns the users + inside the inbounds at runtime (through the API) and everything under `clients/`. Xray keeps the + users in memory only, so usher refills them after a restart. +- **Backend seam.** The `Session`/`Backend` seam from `xray-driver` carries over, with + `Backend.Suffixes []string`, because the Xray backend owns two file kinds. A down backend keeps + all files with any of its suffixes. The Xray session reports every managed tag of the file, so a + tag that fails validation or drift keeps its files instead of having them deleted. +- **Reading the config.** The file is read with Xray's comment-aware reader into `infra/conf` + types plus raw maps. Config aliases (`method`, `splithttp`, `websocket`, `tcp`, + `splithttpSettings`, `tcpSettings`, Hysteria `users`) are normalized before validation and + mirroring. The backend registers a discard handler for Xray's global logger once at + construction, because `Build()` prints deprecation warnings and usher runs no Xray instance. +- **Credentials in final form.** The `xray` group of `users.json` stores what the protocol uses: + `id` (UUID v4, lowercase) for VLESS and VMess, `password` (base64url of 32 random bytes) for + Trojan and Hysteria2. An operator can still seed or re-key an entry by hand. +- **Users.** + - Email is `tag/user`, level 0. Every managed inbound starts with an empty `clients` list. + - Reconcile lists all users of an inbound (`GetInboundUsers` without an email) and compares per + protocol on the credential only: VLESS `id` and `flow`, VMess `id`, Trojan `password`, + Hysteria `auth`. It removes foreign and changed emails and adds missing and changed ones. + - Removal is effective for new connections. A Hysteria2 client with an open QUIC connection + keeps it until it reconnects; the README states this. +- **API socket.** When the socket path exists and `flock(LOCK_EX|LOCK_NB)` on `.lock` + succeeds, no Xray listens on it: usher unlinks the socket and reports Xray down for this pass. + Xray's restart policy brings it up on the next attempt. The socket directory is writable by + usher. +- **Drift guard.** `Open` calls `ListInbounds` once and compares each managed inbound with + `Build()` of its file entry: proxy type, proxy settings and receiver settings. Certificate + entries with a `certificateFile` are removed from the file entry before `Build()` (usher has no + access to the key files) and entries with a `CertificatePath` from the runtime side; Xray + reloads those files itself. A tag in the file but not at runtime also counts as drift. `Apply` + reconciles users regardless and then returns the drift error, so revocation keeps working while + the old client files stay. The drift reason names the differing part (port, proxy type, proxy + settings, stream settings), never a value. +- **Client mirror.** + - Taken from the server: `network`, `security`; from `wsSettings`, `grpcSettings` and + `httpupgradeSettings` only path, host, serviceName and authority; `rawSettings` only with + header type none. + - xhttp: path, host and mode at the top. If the server has `extra`, it is mirrored verbatim and + the other top-level fields are dropped, as the server itself ignores them; otherwise every other + field goes into `extra`. The same form serves `.json` and `.txt`. An inbound with + `downloadSettings` fails. + - Rebuilt: `tlsSettings` and `realitySettings`. Dropped: everything else. + - VLESS only: `flow` is `xtls-rprx-vision` iff network is raw and security is tls or reality. + - The rendered outbound passes `conf.OutboundDetourConfig.Build()` before any file is written; + an error fails the inbound. +- **TLS trust**, one rule for every protocol, set by `x-usher.trust`: + - `ca` (default): the client gets `serverName`, `alpn` and the fingerprint, nothing else. + - `pin`: the client also gets `pinnedPeerCertSha256`, from `x-usher.pin_sha256` or computed from + an inline `certificates[].certificate`. A pin inbound without either fails. A Hysteria2 pin + inbound gets no `.txt`, because v2rayNG and Hiddify do not read `pcs` for `hysteria2://`. +- **Reality.** + - `serverName` is the first `serverNames` entry that is non-empty and has no `*`; none such fails + the inbound. + - `publicKey` comes from `privateKey` (`crypto/ecdh`); `shortId` is `shortIds[0]` verbatim; + `spiderX` is `/`; `mldsa65Verify` is derived from `mldsa65Seed` when the seed is set. + - `fingerprint` is `chrome` by default; an override is limited to `chrome`, `firefox` and + `safari`, the ones that carry `X25519MLKEM768`. + - Reality links work in Xray-core clients only; sing-box based clients fail the server's + `X25519MLKEM768` requirement. The README states this. +- **Hysteria2.** + - The server `hysteriaSettings.auth` must be empty; the client gets its `auth` from the user's + password. + - From salamander only `password` is mirrored; a salamander with `packetSize` fails the inbound. + `finalmask.quicParams` is allowed and stays on the server, with `udpIdleTimeout` and + `masquerade`. + - `x-usher.hop_ports` synthesizes a client udphop mask in libxray's shape (Context). The hop + range needs a host DNAT to the inbound port, which the README states as a prerequisite. +- **Outputs.** + - `clients//.json` is a complete client config: one outbound and a local socks/http + inbound. + - `clients//.txt` comes from libxray, called once per outbound. usher builds every + outbound from shapes it knows, so a libxray refusal is a usher defect: it is logged with the + tag, and no `.txt` is written. +- **Dependency.** usher imports `xray-core` (`infra/conf`, `app/proxyman/command`) and + `libxray/share`. The vendored protos and buf tooling of `xray-driver` are not used. + +## Technical Details + +- **Environment.** + - `USHER_AWG_SOCKET` and `USHER_XRAY_SOCKET` are absolute socket paths; at least one is set. + - `USHER_XRAY_CONFIG` is the read-only path of the Xray server config and is required with + `USHER_XRAY_SOCKET`. + - `USHER_HOST` stays required. `USHER_DNS` is required only with awg. +- **`x-usher` block** on an inbound (Xray ignores unknown keys): + ```json + "x-usher": { + "address": "203.0.113.10", + "port": 443, + "sni": "example.com", + "fingerprint": "chrome", + "trust": "ca", + "pin_sha256": "…", + "hop_ports": "20000-30000" + } + ``` + - `address` defaults to `USHER_HOST`, `port` to the inbound port, `sni` to the server + `tlsSettings.serverName`, `fingerprint` to `chrome`, `trust` to `ca`. + - A TLS inbound with neither `sni` nor `tlsSettings.serverName` fails. + - `pin_sha256` is valid only with `trust: pin`. +- **Delivery contract.** + - One JSON file, no `-confdir`; comments allowed. Both containers mount the directory that holds + it, so a replaced file is seen; usher sees an edit at its next refill tick. + - `api.services: ["HandlerService"]` and `api.listen` on a unix path with `,0660` in a directory + usher can write; the Xray process gid equals usher's group. + - Xray runs with `restart: unless-stopped`. + - Every managed inbound has empty `clients`, a single integer `port`, no unix `listen`, and a tag + that matches the usher interface pattern. + - Hysteria2 inbounds have an empty `hysteriaSettings.auth`. + - TLS inbounds use a public-CA certificate for the operator's domain unless `trust: pin`. + - The Xray image is `ghcr.io/xtls/xray-core:26.10.10`, the release of the `xray-core` commit in + `go.mod`. A different image can change `Build()` output and put every inbound in drift. +- **Interface name pattern** widens from 15 to 32 characters. awg-grpc still rejects names the + kernel cannot hold. +- **State.** Version 2 adds the `xray` group: `{"vless-reality/alice": {"id": "…"}, + "hy2/alice": {"password": "…"}}`. Before the first save of a version 1 file usher writes + `users.json.v1` next to it, so a rollback is a file rename. +- **Rendering per inbound shape**: the golden tests in Task 8 are the authoritative table. + +## What Goes Where + +- **Implementation Steps** (`[ ]` checkboxes): code, tests, deploy examples and docs in this + repository. +- **Post-Completion** (no checkboxes): the public-host smoke with phone clients, the release. + +## Implementation Steps + +### Task 1: Port the backend seam and the awg socket variable + +**Files:** +- Modify: `cmd/usher/env.go`, `cmd/usher/main.go`, `cmd/usher/*_test.go` +- Create: `internal/pass/backend.go`; Modify: `internal/pass/pass.go`, `internal/pass/settings.go`, + `internal/pass/*_test.go` +- Create: `internal/awg/` (moved from `internal/clients/render.go`, `internal/pass/enroll.go`, + `fake_test.go`, `testdata/phone.conf`); Modify: `internal/clients/mirror.go` +- Modify: `deploy/compose.example.yml`, `deploy/compose.e2e.yml` + +- [ ] cherry-pick `bd02bac`, `00113e4`, `f409cac`, `7100137` from `xray-driver`; drop the hunks + for `docs/plans/20261007-xray-driver.md` and for Xray files that do not exist here; reword + the `bd02bac` header without `!` +- [ ] change `Backend.Suffix` to `Suffixes []string` and make `clients.Existing` keep every suffix + of a down backend +- [ ] make `USHER_DNS` required only when `USHER_AWG_SOCKET` is set +- [ ] keep the awg behaviour of `master` unchanged +- [ ] write tests for the seam: two fake backends, a collision, a down backend with two suffixes, + the `USHER_DNS` rule +- [ ] run `go test -race ./...` and the awg e2e (CI) - must pass before Task 2 + +### Task 2: users.json version 2 with Xray credentials + +**Files:** +- Modify: `internal/state/state.go`, `internal/state/state_test.go` +- Modify: `internal/config/config.go`, `internal/config/config_test.go` + +- [ ] start from `64dd6b0` and `69a3ba5`: `Xray map[string]XrayEntry`, `XrayEntry{ID uuid.UUID; + Password string}` with exactly one set; validate names `tag/user`; errors never quote a secret +- [ ] version 2; `Load` accepts 1 and 2; before the first save of a version 1 file write + `users.json.v1` +- [ ] widen the interface name pattern to 32 characters +- [ ] write tests: v1 load and backup, v2 round trip, both credential kinds, invalid entries, no + secret in errors, a 32-character tag +- [ ] run `go test -race ./...` - must pass before Task 3 + +### Task 3: Read the operator's Xray config + +**Files:** +- Modify: `go.mod`, `go.sum` +- Create: `internal/xray/config.go`, `internal/xray/config_test.go` +- Create: `internal/xray/testdata/` (server configs) + +- [ ] require `github.com/xtls/libxray v1.261010.0` and the `xray-core` commit it requires + (`701af60772cd`, release 26.10.10); record the stripped binary size and the cold CI + `go build` time in the progress log +- [ ] read the file with Xray's comment-aware reader, decode inbounds into `infra/conf` types plus + raw maps, normalize the aliases; register the discard log handler +- [ ] keep inbounds whose protocol has an entry (`vless`, `hysteria`, `vmess`, `trojan`); skip + others and untagged ones +- [ ] validate per inbound: single integer port, no unix listen, empty `clients`/`users`, VLESS + `decryption` none, empty server Hysteria auth, masks only salamander without `packetSize` + (plus `quicParams`), no xhttp `downloadSettings`, a usable Reality `serverNames` entry, an + SNI for TLS, the `x-usher` schema (`trust` values, `pin_sha256` only with `pin`, a pin source + for `pin`, the fingerprint allowlist on Reality); each failure names the field +- [ ] write tests for every accepted shape, every failure, every alias and a file with comments +- [ ] run `go test -race ./...` and `docker build .` (CI) - must pass before Task 4 + +### Task 4: Drift guard against the running Xray + +**Files:** +- Create: `internal/xray/drift.go`, `internal/xray/drift_test.go` + +- [ ] compare `Build()` of each managed inbound with one `ListInbounds`: proxy type, proxy settings, + receiver settings, with file-based certificate entries removed on both sides +- [ ] a mismatch or a tag missing at runtime is a per-tag drift reason naming the differing part + only; runtime tags absent from the file are logged each pass +- [ ] write tests with a fake `HandlerServiceClient`: match, port change, `shortIds` change, a + `privateKey` change (reason without the value), protocol change, a `certificateFile` inbound + with changed file contents (no drift), a file tag missing at runtime, an extra runtime tag +- [ ] run `go test -race ./...` - must pass before Task 5 + +### Task 5: Users through HandlerService + +**Files:** +- Create: `internal/xray/protocol.go`, `internal/xray/session.go`, `internal/xray/enroll.go` +- Create: matching `_test.go` files + +- [ ] protocol table: credential kind, account builder and credential comparison for VLESS (`id`, + flow), Hysteria2 (`auth`), VMess (`id`), Trojan (`password`) +- [ ] enroll: create the credential for every referenced user without one; never change an + existing entry +- [ ] session: `Interfaces` reports every managed tag; `Open` runs the drift check; `Apply` lists + all users per inbound, compares credentials, removes foreign and changed emails, adds missing + and changed, logs `users changed`, then returns the tag's drift error if any +- [ ] write tests with a fake service: add, remove, change, foreign email, seeded credential kept, + VMess echo with `SecuritySettings` (no change), removal while the tag is in drift, partial + failure converging, no secret in logs +- [ ] run `go test -race ./...` - must pass before Task 6 + +### Task 6: Stale API socket recovery + +**Files:** +- Create: `internal/xray/socket.go`, `internal/xray/socket_test.go` + +- [ ] before dialing: if the socket path exists and `flock(LOCK_EX|LOCK_NB)` on `.lock` + succeeds, unlink the socket, release the lock and report Xray down for this pass +- [ ] write tests in `t.TempDir()`: a stale socket (listener with `SetUnlinkOnClose(false)`) is + removed; a socket whose lock another descriptor holds is kept; a missing path is left alone +- [ ] run `go test -race ./...` - must pass before Task 7 + +### Task 7: Client mirror + +**Files:** +- Create: `internal/xray/mirror.go`, `internal/xray/mirror_test.go` + +- [ ] transport fields as in Solution Overview, xhttp `extra` packing or verbatim server `extra`, + the VLESS flow rule +- [ ] the TLS trust rule: `ca` default, `pin` from `pin_sha256` or the inline certificate +- [ ] Reality rewrite including the `serverNames` rule and `mldsa65Verify` from `mldsa65Seed` +- [ ] Hysteria2: client auth, salamander password, the udphop mask from `hop_ports` in libxray's + shape +- [ ] `x-usher` overrides +- [ ] write the leak test (Testing Strategy) and tests for each trust case, hop, flow, xhttp + packing, server `extra` and override +- [ ] run `go test -race ./...` - must pass before Task 8 + +### Task 8: Render client files and golden tests + +**Files:** +- Create: `internal/xray/render.go`, `internal/xray/render_test.go` +- Create: `internal/xray/testdata/golden/` + +- [ ] `.json`: complete client config with the mirrored outbound and a local proxy inbound; the + outbound passes `conf.OutboundDetourConfig.Build()` or the inbound fails +- [ ] `.txt`: libxray per outbound; on error no `.txt` and a logged defect; none for a Hysteria2 + pin inbound +- [ ] prepare step writes both files through the seam +- [ ] golden tests for the shapes in Acceptance Evidence item 2 +- [ ] run `go test -race ./...` - must pass before Task 9 + +### Task 9: Wire the backend and the deploy example + +**Files:** +- Modify: `cmd/usher/env.go`, `cmd/usher/main.go`, tests +- Modify: `deploy/compose.example.yml`; Create: `deploy/xray/config.json` + +- [ ] read `USHER_XRAY_SOCKET` and `USHER_XRAY_CONFIG`; build the Xray backend +- [ ] example config: one VLESS+Reality and one Hysteria2 inbound with a `certificateFile` and + `x-usher`, placeholders that Xray refuses to start with; compose mounts the config directory + into both containers (usher read-only), a socket directory usher owns, Xray with + `restart: unless-stopped` and image `26.10.10` with a comment that it follows `go.mod` +- [ ] write env and wiring tests +- [ ] run `go test -race ./...` - must pass before Task 10 + +### Task 10: Xray end-to-end suite + +**Files:** +- Create: `test/e2e/xray/` and `deploy/compose.e2e-xray.yml`, `deploy/e2e/xray/` (from + `c5a9f2c`, `e47b4cc`, `0993e07`, adapted) +- Modify: `.github/workflows/ci.yml` + +- [ ] server config with the four inbounds of Acceptance Evidence item 3, a test CA and leaves + generated by the test, `x-usher` blocks, freedom `finalRules` allowing the target; config + directory mount; Xray with `restart: unless-stopped`; Hysteria2 case first +- [ ] Xray client with `SSL_CERT_FILE` and `useSystemCA` +- [ ] each client connects with the rendered `.json` and with the `.txt` via + `ConvertShareLinksToXrayJson` +- [ ] assert the outcomes of Acceptance Evidence item 3 +- [ ] run the suite in the CI `e2e` job - must pass before Task 11 + +### Task 11: Verify acceptance criteria + +- [ ] verify all requirements from Overview are implemented +- [ ] verify edge cases are handled +- [ ] run full test suite: `go test -race -count=1 ./...` +- [ ] run `scripts/format.sh`, shellcheck and actionlint +- [ ] run e2e tests: `go test -tags e2e -count=1 -v ./test/e2e/...` (CI) +- [ ] verify test coverage meets project standard + +### Task 12: [Final] Update documentation + +- [ ] README: delivery contract (image follows `go.mod`, directory mount, restart policy, socket + directory), `x-usher` block, TLS trust rule, outputs, supported shapes, client compatibility + (Reality in Xray-core clients only, xhttp not in upstream sing-box), the Hysteria2 removal + limit, the hop DNAT prerequisite, config edits seen at the refill tick +- [ ] `docs/architecture/overview.md`: who owns which JSON, the seam, the mirror, drift guard, + state v2, the stale socket rule +- [ ] `AGENTS.md`: layout, the Xray e2e, never-logged fields +- [ ] move this plan to `docs/plans/completed/` + +## Post-Completion + +*Items requiring manual intervention or external systems - no checkboxes, informational only* + +**Manual verification on a public host** +- run the example stack on a test VPS with a public IP and a domain with a public certificate; the + operator opens the test ports in the host firewall (sudo) +- import one VLESS+Reality link, one Hysteria2 link and one Hysteria2 `hop_ports` link into + v2rayNG and open a page through each; the hop link checks how v2rayNG reads a port range in the + authority +- import the ws and Hysteria2 links into Hiddify + +**Release** +- bump the image tag in `deploy/compose.example.yml`; the notes state the new required variables, + `users.json` version 2, the `users.json.v1` backup and the Xray image 26.10.10 From 99a3a137d5760563ba1b749000c7d70eb77b0513 Mon Sep 17 00:00:00 2001 From: EvaTheSalmon <18466855+EvaTheSalmon@users.noreply.github.com> Date: Fri, 9 Oct 2026 16:10:38 +0300 Subject: [PATCH 02/32] feat(usher): read awg socket from environment Co-Authored-By: Claude Sonnet 5.5 --- cmd/usher/env.go | 17 ++++++++++++++ cmd/usher/env_test.go | 47 ++++++++++++++++++++++++-------------- cmd/usher/main.go | 17 +++++++------- cmd/usher/main_test.go | 6 ++--- deploy/compose.e2e.yml | 1 + deploy/compose.example.yml | 1 + internal/pass/settings.go | 1 + 7 files changed, 62 insertions(+), 28 deletions(-) diff --git a/cmd/usher/env.go b/cmd/usher/env.go index a45adf3..af1277d 100644 --- a/cmd/usher/env.go +++ b/cmd/usher/env.go @@ -3,6 +3,7 @@ package main import ( "fmt" "net/netip" + "path/filepath" "strings" "github.com/mrcsin/usher/internal/pass" @@ -20,6 +21,11 @@ func settingsFrom(getenv func(string) string) (pass.Settings, error) { return pass.Settings{}, err } + awgSocket, err := parseSocketPath("USHER_AWG_SOCKET", getenv("USHER_AWG_SOCKET")) + if err != nil { + return pass.Settings{}, err + } + var dns []netip.Addr for _, field := range strings.Split(getenv("USHER_DNS"), ",") { addr, err := parseIPv4("USHER_DNS", strings.TrimSpace(field)) @@ -32,6 +38,7 @@ func settingsFrom(getenv func(string) string) (pass.Settings, error) { return pass.Settings{ Host: host, DNS: dns, + AWGSocket: awgSocket, ConfigPath: configPath, ClientsDir: clientsPath, StatePath: statePath, @@ -48,3 +55,13 @@ func parseIPv4(name, value string) (netip.Addr, error) { } return addr, nil } + +func parseSocketPath(name, value string) (string, error) { + if value == "" { + return "", fmt.Errorf("%s is required", name) + } + if !filepath.IsAbs(value) { + return "", fmt.Errorf("%s: %q is not an absolute path", name, value) + } + return value, nil +} diff --git a/cmd/usher/env_test.go b/cmd/usher/env_test.go index 7e0207a..c4080d7 100644 --- a/cmd/usher/env_test.go +++ b/cmd/usher/env_test.go @@ -18,58 +18,70 @@ func TestSettingsFrom(t *testing.T) { }{ { name: "one DNS address", - env: map[string]string{"USHER_HOST": "203.0.113.10", "USHER_DNS": "1.1.1.1"}, + env: map[string]string{"USHER_HOST": "203.0.113.10", "USHER_AWG_SOCKET": "/run/awg.sock", "USHER_DNS": "1.1.1.1"}, want: pass.Settings{ - Host: netip.MustParseAddr("203.0.113.10"), - DNS: []netip.Addr{netip.MustParseAddr("1.1.1.1")}, + Host: netip.MustParseAddr("203.0.113.10"), + DNS: []netip.Addr{netip.MustParseAddr("1.1.1.1")}, + AWGSocket: "/run/awg.sock", }, }, { name: "two DNS addresses", - env: map[string]string{"USHER_HOST": "203.0.113.10", "USHER_DNS": "1.1.1.1, 1.0.0.1"}, + env: map[string]string{"USHER_HOST": "203.0.113.10", "USHER_AWG_SOCKET": "/run/awg.sock", "USHER_DNS": "1.1.1.1, 1.0.0.1"}, want: pass.Settings{ - Host: netip.MustParseAddr("203.0.113.10"), - DNS: []netip.Addr{netip.MustParseAddr("1.1.1.1"), netip.MustParseAddr("1.0.0.1")}, + Host: netip.MustParseAddr("203.0.113.10"), + DNS: []netip.Addr{netip.MustParseAddr("1.1.1.1"), netip.MustParseAddr("1.0.0.1")}, + AWGSocket: "/run/awg.sock", }, }, { name: "missing host", - env: map[string]string{"USHER_DNS": "1.1.1.1"}, + env: map[string]string{"USHER_AWG_SOCKET": "/run/awg.sock", "USHER_DNS": "1.1.1.1"}, wantErr: "USHER_HOST", }, { name: "host name", - env: map[string]string{"USHER_HOST": "vpn.example.com", "USHER_DNS": "1.1.1.1"}, + env: map[string]string{"USHER_HOST": "vpn.example.com", "USHER_AWG_SOCKET": "/run/awg.sock", "USHER_DNS": "1.1.1.1"}, wantErr: "USHER_HOST", }, { name: "IPv6 host", - env: map[string]string{"USHER_HOST": "2001:db8::1", "USHER_DNS": "1.1.1.1"}, + env: map[string]string{"USHER_HOST": "2001:db8::1", "USHER_AWG_SOCKET": "/run/awg.sock", "USHER_DNS": "1.1.1.1"}, wantErr: "USHER_HOST", }, { name: "host with port", - env: map[string]string{"USHER_HOST": "203.0.113.10:51820", "USHER_DNS": "1.1.1.1"}, + env: map[string]string{"USHER_HOST": "203.0.113.10:51820", "USHER_AWG_SOCKET": "/run/awg.sock", "USHER_DNS": "1.1.1.1"}, wantErr: "USHER_HOST", }, + { + name: "missing awg socket", + env: map[string]string{"USHER_HOST": "203.0.113.10", "USHER_DNS": "1.1.1.1"}, + wantErr: "USHER_AWG_SOCKET", + }, + { + name: "relative awg socket", + env: map[string]string{"USHER_HOST": "203.0.113.10", "USHER_AWG_SOCKET": "awg.sock", "USHER_DNS": "1.1.1.1"}, + wantErr: "USHER_AWG_SOCKET", + }, { name: "empty DNS", - env: map[string]string{"USHER_HOST": "203.0.113.10", "USHER_DNS": ""}, + env: map[string]string{"USHER_HOST": "203.0.113.10", "USHER_AWG_SOCKET": "/run/awg.sock", "USHER_DNS": ""}, wantErr: "USHER_DNS", }, { name: "IPv6 DNS", - env: map[string]string{"USHER_HOST": "203.0.113.10", "USHER_DNS": "2606:4700:4700::1111"}, + env: map[string]string{"USHER_HOST": "203.0.113.10", "USHER_AWG_SOCKET": "/run/awg.sock", "USHER_DNS": "2606:4700:4700::1111"}, wantErr: "USHER_DNS", }, { name: "DNS with port", - env: map[string]string{"USHER_HOST": "203.0.113.10", "USHER_DNS": "1.1.1.1:53"}, + env: map[string]string{"USHER_HOST": "203.0.113.10", "USHER_AWG_SOCKET": "/run/awg.sock", "USHER_DNS": "1.1.1.1:53"}, wantErr: "USHER_DNS", }, { name: "empty DNS element", - env: map[string]string{"USHER_HOST": "203.0.113.10", "USHER_DNS": "1.1.1.1,"}, + env: map[string]string{"USHER_HOST": "203.0.113.10", "USHER_AWG_SOCKET": "/run/awg.sock", "USHER_DNS": "1.1.1.1,"}, wantErr: "USHER_DNS", }, } @@ -85,15 +97,16 @@ func TestSettingsFrom(t *testing.T) { if err != nil { t.Fatalf("unexpected error: %v", err) } - if got.Host != tt.want.Host || !slices.Equal(got.DNS, tt.want.DNS) { - t.Errorf("host/dns = %v %v, want %v %v", got.Host, got.DNS, tt.want.Host, tt.want.DNS) + if got.Host != tt.want.Host || !slices.Equal(got.DNS, tt.want.DNS) || got.AWGSocket != tt.want.AWGSocket { + t.Errorf("host/dns/socket = %v %v %q, want %v %v %q", + got.Host, got.DNS, got.AWGSocket, tt.want.Host, tt.want.DNS, tt.want.AWGSocket) } }) } } func TestSettingsFromFixedPaths(t *testing.T) { - env := map[string]string{"USHER_HOST": "203.0.113.10", "USHER_DNS": "1.1.1.1"} + env := map[string]string{"USHER_HOST": "203.0.113.10", "USHER_AWG_SOCKET": "/run/awg.sock", "USHER_DNS": "1.1.1.1"} got, err := settingsFrom(func(key string) string { return env[key] }) if err != nil { t.Fatal(err) diff --git a/cmd/usher/main.go b/cmd/usher/main.go index 41f5fba..f2e4a1d 100644 --- a/cmd/usher/main.go +++ b/cmd/usher/main.go @@ -19,8 +19,6 @@ import ( "github.com/mrcsin/usher/internal/watch" ) -const socketTarget = "unix:///run/awg-grpc/awg.sock" - // version is set at build time with -X main.version. var version = "dev" @@ -41,7 +39,7 @@ func run(ctx context.Context, args []string, getenv func(string) string, stderr return 1 } - return serve(ctx, settings, dialSocket, stderr) + return serve(ctx, settings, dialSocket(settings.AWGSocket), stderr) } func serve(ctx context.Context, settings pass.Settings, dial pass.Dial, stderr io.Writer) int { @@ -54,10 +52,13 @@ func serve(ctx context.Context, settings pass.Settings, dial pass.Dial, stderr i return 0 } -func dialSocket() (awgv1.ManagementServiceClient, func(), error) { - conn, err := grpc.NewClient(socketTarget, grpc.WithTransportCredentials(insecure.NewCredentials())) - if err != nil { - return nil, nil, fmt.Errorf("opening %s: %w", socketTarget, err) +func dialSocket(path string) pass.Dial { + target := "unix://" + path + return func() (awgv1.ManagementServiceClient, func(), error) { + conn, err := grpc.NewClient(target, grpc.WithTransportCredentials(insecure.NewCredentials())) + if err != nil { + return nil, nil, fmt.Errorf("opening %s: %w", target, err) + } + return awgv1.NewManagementServiceClient(conn), func() { _ = conn.Close() }, nil } - return awgv1.NewManagementServiceClient(conn), func() { _ = conn.Close() }, nil } diff --git a/cmd/usher/main_test.go b/cmd/usher/main_test.go index 25144b3..8b57bb0 100644 --- a/cmd/usher/main_test.go +++ b/cmd/usher/main_test.go @@ -16,7 +16,7 @@ import ( ) func TestRun(t *testing.T) { - validEnv := map[string]string{"USHER_HOST": "203.0.113.10", "USHER_DNS": "1.1.1.1"} + validEnv := map[string]string{"USHER_HOST": "203.0.113.10", "USHER_AWG_SOCKET": "/run/awg.sock", "USHER_DNS": "1.1.1.1"} tests := []struct { name string args []string @@ -27,8 +27,8 @@ func TestRun(t *testing.T) { {name: "no argument", args: nil, env: validEnv, wantCode: 2, wantStderr: "usage"}, {name: "unknown argument", args: []string{"serve"}, env: validEnv, wantCode: 2, wantStderr: "usage"}, {name: "extra argument", args: []string{"run", "x"}, env: validEnv, wantCode: 2, wantStderr: "usage"}, - {name: "missing host", args: []string{"run"}, env: map[string]string{"USHER_DNS": "1.1.1.1"}, wantCode: 1, wantStderr: "USHER_HOST"}, - {name: "bad dns", args: []string{"run"}, env: map[string]string{"USHER_HOST": "203.0.113.10", "USHER_DNS": "dns.example"}, wantCode: 1, wantStderr: "USHER_DNS"}, + {name: "missing host", args: []string{"run"}, env: map[string]string{"USHER_AWG_SOCKET": "/run/awg.sock", "USHER_DNS": "1.1.1.1"}, wantCode: 1, wantStderr: "USHER_HOST"}, + {name: "bad dns", args: []string{"run"}, env: map[string]string{"USHER_HOST": "203.0.113.10", "USHER_AWG_SOCKET": "/run/awg.sock", "USHER_DNS": "dns.example"}, wantCode: 1, wantStderr: "USHER_DNS"}, } for _, tt := range tests { t.Run(tt.name, func(t *testing.T) { diff --git a/deploy/compose.e2e.yml b/deploy/compose.e2e.yml index efaf486..2e405c4 100644 --- a/deploy/compose.e2e.yml +++ b/deploy/compose.e2e.yml @@ -34,6 +34,7 @@ services: - awg-sock:/run/awg-grpc environment: USHER_HOST: 172.30.99.2 + USHER_AWG_SOCKET: /run/awg-grpc/awg.sock USHER_DNS: 1.1.1.1 # Handshake peer of the e2e test; test/e2e writes its config to ../.e2e/client. client: diff --git a/deploy/compose.example.yml b/deploy/compose.example.yml index a10b82f..891102c 100644 --- a/deploy/compose.example.yml +++ b/deploy/compose.example.yml @@ -38,6 +38,7 @@ services: - awg-sock:/run/awg-grpc environment: USHER_HOST: "203.0.113.10" + USHER_AWG_SOCKET: /run/awg-grpc/awg.sock USHER_DNS: "1.1.1.1,1.0.0.1" volumes: awg-sock: diff --git a/internal/pass/settings.go b/internal/pass/settings.go index 00c70b8..5783e7a 100644 --- a/internal/pass/settings.go +++ b/internal/pass/settings.go @@ -7,6 +7,7 @@ import "net/netip" type Settings struct { Host netip.Addr // USHER_HOST DNS []netip.Addr // USHER_DNS + AWGSocket string // USHER_AWG_SOCKET ConfigPath string ClientsDir string StatePath string From 4b016ef8ae226bf68d6ae50af63d4d94fedcc77a Mon Sep 17 00:00:00 2001 From: EvaTheSalmon <18466855+EvaTheSalmon@users.noreply.github.com> Date: Fri, 9 Oct 2026 16:19:58 +0300 Subject: [PATCH 03/32] refactor(pass): add backend seam with awg driver --- cmd/usher/env.go | 28 +- cmd/usher/env_test.go | 16 +- cmd/usher/main.go | 29 +- cmd/usher/main_test.go | 15 +- internal/{pass => awg}/enroll.go | 2 +- internal/{pass => awg}/enroll_test.go | 2 +- internal/{pass => awg}/fake_test.go | 2 +- internal/{clients => awg}/render.go | 3 +- internal/{clients => awg}/render_test.go | 2 +- internal/awg/session.go | 155 +++++++ internal/awg/session_test.go | 283 ++++++++++++ internal/{clients => awg}/testdata/phone.conf | 0 internal/clients/mirror.go | 29 +- internal/clients/mirror_test.go | 39 +- internal/pass/backend.go | 28 ++ internal/pass/pass.go | 233 +++++----- internal/pass/pass_test.go | 415 ++++++++++-------- internal/pass/settings.go | 10 +- 18 files changed, 903 insertions(+), 388 deletions(-) rename internal/{pass => awg}/enroll.go (99%) rename internal/{pass => awg}/enroll_test.go (99%) rename internal/{pass => awg}/fake_test.go (98%) rename internal/{clients => awg}/render.go (93%) rename internal/{clients => awg}/render_test.go (99%) create mode 100644 internal/awg/session.go create mode 100644 internal/awg/session_test.go rename internal/{clients => awg}/testdata/phone.conf (100%) create mode 100644 internal/pass/backend.go diff --git a/cmd/usher/env.go b/cmd/usher/env.go index af1277d..9b973bb 100644 --- a/cmd/usher/env.go +++ b/cmd/usher/env.go @@ -15,33 +15,39 @@ const ( statePath = "/srv/usher/state/users.json" ) -func settingsFrom(getenv func(string) string) (pass.Settings, error) { +// environment holds the server facts from the process environment and the fixed paths. +type environment struct { + pass.Settings + Host netip.Addr // USHER_HOST + DNS []netip.Addr // USHER_DNS + AWGSocket string // USHER_AWG_SOCKET +} + +func environmentFrom(getenv func(string) string) (environment, error) { host, err := parseIPv4("USHER_HOST", getenv("USHER_HOST")) if err != nil { - return pass.Settings{}, err + return environment{}, err } awgSocket, err := parseSocketPath("USHER_AWG_SOCKET", getenv("USHER_AWG_SOCKET")) if err != nil { - return pass.Settings{}, err + return environment{}, err } var dns []netip.Addr for _, field := range strings.Split(getenv("USHER_DNS"), ",") { addr, err := parseIPv4("USHER_DNS", strings.TrimSpace(field)) if err != nil { - return pass.Settings{}, err + return environment{}, err } dns = append(dns, addr) } - return pass.Settings{ - Host: host, - DNS: dns, - AWGSocket: awgSocket, - ConfigPath: configPath, - ClientsDir: clientsPath, - StatePath: statePath, + return environment{ + Settings: pass.Settings{ConfigPath: configPath, ClientsDir: clientsPath, StatePath: statePath}, + Host: host, + DNS: dns, + AWGSocket: awgSocket, }, nil } diff --git a/cmd/usher/env_test.go b/cmd/usher/env_test.go index c4080d7..08a38fa 100644 --- a/cmd/usher/env_test.go +++ b/cmd/usher/env_test.go @@ -5,21 +5,19 @@ import ( "slices" "strings" "testing" - - "github.com/mrcsin/usher/internal/pass" ) -func TestSettingsFrom(t *testing.T) { +func TestEnvironmentFrom(t *testing.T) { tests := []struct { name string env map[string]string - want pass.Settings + want environment wantErr string }{ { name: "one DNS address", env: map[string]string{"USHER_HOST": "203.0.113.10", "USHER_AWG_SOCKET": "/run/awg.sock", "USHER_DNS": "1.1.1.1"}, - want: pass.Settings{ + want: environment{ Host: netip.MustParseAddr("203.0.113.10"), DNS: []netip.Addr{netip.MustParseAddr("1.1.1.1")}, AWGSocket: "/run/awg.sock", @@ -28,7 +26,7 @@ func TestSettingsFrom(t *testing.T) { { name: "two DNS addresses", env: map[string]string{"USHER_HOST": "203.0.113.10", "USHER_AWG_SOCKET": "/run/awg.sock", "USHER_DNS": "1.1.1.1, 1.0.0.1"}, - want: pass.Settings{ + want: environment{ Host: netip.MustParseAddr("203.0.113.10"), DNS: []netip.Addr{netip.MustParseAddr("1.1.1.1"), netip.MustParseAddr("1.0.0.1")}, AWGSocket: "/run/awg.sock", @@ -87,7 +85,7 @@ func TestSettingsFrom(t *testing.T) { } for _, tt := range tests { t.Run(tt.name, func(t *testing.T) { - got, err := settingsFrom(func(key string) string { return tt.env[key] }) + got, err := environmentFrom(func(key string) string { return tt.env[key] }) if tt.wantErr != "" { if err == nil || !strings.Contains(err.Error(), tt.wantErr) { t.Fatalf("err = %v, want it to name %s", err, tt.wantErr) @@ -105,9 +103,9 @@ func TestSettingsFrom(t *testing.T) { } } -func TestSettingsFromFixedPaths(t *testing.T) { +func TestEnvironmentFromFixedPaths(t *testing.T) { env := map[string]string{"USHER_HOST": "203.0.113.10", "USHER_AWG_SOCKET": "/run/awg.sock", "USHER_DNS": "1.1.1.1"} - got, err := settingsFrom(func(key string) string { return env[key] }) + got, err := environmentFrom(func(key string) string { return env[key] }) if err != nil { t.Fatal(err) } diff --git a/cmd/usher/main.go b/cmd/usher/main.go index f2e4a1d..2296521 100644 --- a/cmd/usher/main.go +++ b/cmd/usher/main.go @@ -15,6 +15,7 @@ import ( "google.golang.org/grpc" "google.golang.org/grpc/credentials/insecure" + "github.com/mrcsin/usher/internal/awg" "github.com/mrcsin/usher/internal/pass" "github.com/mrcsin/usher/internal/watch" ) @@ -33,26 +34,42 @@ func run(ctx context.Context, args []string, getenv func(string) string, stderr fmt.Fprintln(stderr, "usage: usher run") return 2 } - settings, err := settingsFrom(getenv) + env, err := environmentFrom(getenv) if err != nil { fmt.Fprintln(stderr, "usher:", err) return 1 } - return serve(ctx, settings, dialSocket(settings.AWGSocket), stderr) + log := slog.New(slog.NewTextHandler(stderr, nil)) + backends := []pass.Backend{awgBackend(env, log)} + return serve(ctx, env.Settings, backends, log) } -func serve(ctx context.Context, settings pass.Settings, dial pass.Dial, stderr io.Writer) int { - log := slog.New(slog.NewTextHandler(stderr, nil)) +func serve(ctx context.Context, settings pass.Settings, backends []pass.Backend, log *slog.Logger) int { log.Info("usher starting", "version", version) - p := pass.New(settings, dial, log) + p := pass.New(settings, backends, log) intervals := watch.Intervals{Poll: 3 * time.Second, Settle: 500 * time.Millisecond, Refill: 30 * time.Second} watch.New(settings.ConfigPath, intervals, p.Run).Run(ctx) log.Info("usher stopped") return 0 } -func dialSocket(path string) pass.Dial { +func awgBackend(env environment, log *slog.Logger) pass.Backend { + dial := dialSocket(env.AWGSocket) + return pass.Backend{ + Name: "awg-grpc", + Suffix: awg.Suffix, + Open: func(ctx context.Context) (pass.Session, error) { + session, err := awg.Open(ctx, dial, env.Host, env.DNS, log) + if err != nil { + return nil, err + } + return session, nil + }, + } +} + +func dialSocket(path string) awg.Dial { target := "unix://" + path return func() (awgv1.ManagementServiceClient, func(), error) { conn, err := grpc.NewClient(target, grpc.WithTransportCredentials(insecure.NewCredentials())) diff --git a/cmd/usher/main_test.go b/cmd/usher/main_test.go index 8b57bb0..0769574 100644 --- a/cmd/usher/main_test.go +++ b/cmd/usher/main_test.go @@ -4,14 +4,12 @@ import ( "bytes" "context" "errors" - "net/netip" + "log/slog" "path/filepath" "strings" "testing" "time" - awgv1 "github.com/mrcsin/awg-grpc/gen/awg/v1" - "github.com/mrcsin/usher/internal/pass" ) @@ -59,20 +57,23 @@ func TestRun(t *testing.T) { func TestServe(t *testing.T) { root := t.TempDir() settings := pass.Settings{ - Host: netip.MustParseAddr("203.0.113.10"), - DNS: []netip.Addr{netip.MustParseAddr("1.1.1.1")}, ConfigPath: filepath.Join(root, "usher.yml"), ClientsDir: filepath.Join(root, "clients"), StatePath: filepath.Join(root, "users.json"), } - dial := func() (awgv1.ManagementServiceClient, func(), error) { return nil, nil, errors.New("no socket") } + backends := []pass.Backend{{ + Name: "fake", + Suffix: ".conf", + Open: func(context.Context) (pass.Session, error) { return nil, errors.New("no socket") }, + }} ctx, cancel := context.WithCancel(context.Background()) defer cancel() time.AfterFunc(200*time.Millisecond, cancel) var stderr bytes.Buffer + log := slog.New(slog.NewTextHandler(&stderr, nil)) done := make(chan int, 1) - go func() { done <- serve(ctx, settings, dial, &stderr) }() + go func() { done <- serve(ctx, settings, backends, log) }() select { case code := <-done: if code != 0 { diff --git a/internal/pass/enroll.go b/internal/awg/enroll.go similarity index 99% rename from internal/pass/enroll.go rename to internal/awg/enroll.go index b2f62e3..03173bc 100644 --- a/internal/pass/enroll.go +++ b/internal/awg/enroll.go @@ -1,4 +1,4 @@ -package pass +package awg import ( "crypto/ecdh" diff --git a/internal/pass/enroll_test.go b/internal/awg/enroll_test.go similarity index 99% rename from internal/pass/enroll_test.go rename to internal/awg/enroll_test.go index 6386016..f292d37 100644 --- a/internal/pass/enroll_test.go +++ b/internal/awg/enroll_test.go @@ -1,4 +1,4 @@ -package pass +package awg import ( "bytes" diff --git a/internal/pass/fake_test.go b/internal/awg/fake_test.go similarity index 98% rename from internal/pass/fake_test.go rename to internal/awg/fake_test.go index df011e4..e53eedc 100644 --- a/internal/pass/fake_test.go +++ b/internal/awg/fake_test.go @@ -1,4 +1,4 @@ -package pass +package awg import ( "context" diff --git a/internal/clients/render.go b/internal/awg/render.go similarity index 93% rename from internal/clients/render.go rename to internal/awg/render.go index 4eb2388..9d202d9 100644 --- a/internal/clients/render.go +++ b/internal/awg/render.go @@ -1,5 +1,4 @@ -// Package clients renders AmneziaWG client configs and mirrors them into the clients directory. -package clients +package awg import ( "encoding/base64" diff --git a/internal/clients/render_test.go b/internal/awg/render_test.go similarity index 99% rename from internal/clients/render_test.go rename to internal/awg/render_test.go index 3469e3c..bfe8907 100644 --- a/internal/clients/render_test.go +++ b/internal/awg/render_test.go @@ -1,4 +1,4 @@ -package clients +package awg import ( "bytes" diff --git a/internal/awg/session.go b/internal/awg/session.go new file mode 100644 index 0000000..8153d24 --- /dev/null +++ b/internal/awg/session.go @@ -0,0 +1,155 @@ +// Package awg is the AmneziaWG backend: it enrolls users, renders their client configs and +// applies their peers through the awg-grpc socket. +package awg + +import ( + "context" + "encoding/base64" + "errors" + "fmt" + "log/slog" + "net/netip" + "slices" + "time" + + awgv1 "github.com/mrcsin/awg-grpc/gen/awg/v1" + + "github.com/mrcsin/usher/internal/clients" + "github.com/mrcsin/usher/internal/state" +) + +// Suffix is the file suffix of an AmneziaWG client config. +const Suffix = ".conf" + +// callTimeout bounds each call to awg-grpc. +const callTimeout = 10 * time.Second + +// Dial opens a connection to awg-grpc. It returns the client and the function that closes the +// connection. +type Dial func() (awgv1.ManagementServiceClient, func(), error) + +// Session is one pass's view of awg-grpc: the interfaces it reported when the session opened. +type Session struct { + client awgv1.ManagementServiceClient + closeConn func() + ifaces map[string]*awgv1.InterfaceStatus + names []string + host netip.Addr + dns []netip.Addr + log *slog.Logger +} + +// Open connects to awg-grpc and lists its interfaces. host and dns go into the client configs. +func Open(ctx context.Context, dial Dial, host netip.Addr, dns []netip.Addr, log *slog.Logger) (*Session, error) { + client, closeConn, err := dial() + if err != nil { + return nil, fmt.Errorf("connecting to awg-grpc: %w", err) + } + callCtx, cancel := context.WithTimeout(ctx, callTimeout) + defer cancel() + status, err := client.GetStatus(callCtx, &awgv1.GetStatusRequest{}) + if err != nil { + closeConn() + return nil, fmt.Errorf("getting status: %w", err) + } + s := &Session{ + client: client, + closeConn: closeConn, + ifaces: make(map[string]*awgv1.InterfaceStatus, len(status.GetInterfaces())), + host: host, + dns: dns, + log: log, + } + for _, iface := range status.GetInterfaces() { + s.ifaces[iface.GetName()] = iface + s.names = append(s.names, iface.GetName()) + } + return s, nil +} + +// Interfaces returns the names of the interfaces awg-grpc reported, in the order it reported them. +func (s *Session) Interfaces() []string { + return s.names +} + +// Prepare enrolls the users on the interface and renders their client configs, keyed by +// clients.Path. It reports whether it changed st. It fails an interface that is not present in +// the kernel. +func (s *Session) Prepare(st *state.State, name string, users []string) (map[string][]byte, bool, error) { + iface := s.ifaces[name] + if !iface.GetPresent() { + return nil, false, errors.New("interface is not present in the kernel") + } + changed, err := enroll(s.log, st, iface, users) + if err != nil { + return nil, false, err + } + files := make(map[string][]byte, len(users)) + for _, user := range users { + content, err := Render(st.AWG[state.EntryName(name, user)], iface, s.host, s.dns) + if err != nil { + return nil, false, fmt.Errorf("user %s: %w", user, err) + } + files[clients.Path(user, name, Suffix)] = content + } + return files, changed, nil +} + +// Apply sends the peers of the users to awg-grpc and logs what changed. +func (s *Session) Apply(ctx context.Context, st *state.State, name string, users []string) error { + peers := make([]*awgv1.Peer, 0, len(users)) + for _, user := range users { + e := st.AWG[state.EntryName(name, user)] + peers = append(peers, &awgv1.Peer{ + PublicKey: e.PublicKey(), + PresharedKey: e.PresharedKey[:], + AllowedIp: e.Route().String(), + }) + } + + callCtx, cancel := context.WithTimeout(ctx, callTimeout) + defer cancel() + resp, err := s.client.ApplyPeers(callCtx, &awgv1.ApplyPeersRequest{ + InterfaceName: name, + Peers: peers, + AllowEmpty: len(peers) == 0, + }) + if err != nil { + return fmt.Errorf("applying peers: %w", err) + } + + names := publicKeyNames(st, name) + added, removed, updated := nameKeys(names, resp.GetAdded()), nameKeys(names, resp.GetRemoved()), nameKeys(names, resp.GetUpdated()) + if len(added)+len(removed)+len(updated) > 0 { + s.log.Info("peers changed", "interface", name, "added", added, "removed", removed, "updated", updated) + } + return nil +} + +// Close closes the connection to awg-grpc. +func (s *Session) Close() { + s.closeConn() +} + +// publicKeyNames maps the public key of every entry of the interface, switched off or not, to +// its user name. +func publicKeyNames(st *state.State, ifaceName string) map[string]string { + names := make(map[string]string) + for user, e := range st.InterfaceEntries(ifaceName) { + names[string(e.PublicKey())] = user + } + return names +} + +func nameKeys(names map[string]string, keys [][]byte) []string { + out := make([]string, 0, len(keys)) + for _, k := range keys { + if user, ok := names[string(k)]; ok { + out = append(out, user) + } else { + out = append(out, base64.StdEncoding.EncodeToString(k)) + } + } + slices.Sort(out) + return out +} diff --git a/internal/awg/session_test.go b/internal/awg/session_test.go new file mode 100644 index 0000000..5e2bd42 --- /dev/null +++ b/internal/awg/session_test.go @@ -0,0 +1,283 @@ +package awg + +import ( + "bytes" + "context" + "encoding/base64" + "errors" + "log/slog" + "maps" + "net/netip" + "slices" + "strings" + "testing" + + awgv1 "github.com/mrcsin/awg-grpc/gen/awg/v1" + + "github.com/mrcsin/usher/internal/state" +) + +type sessionEnv struct { + t *testing.T + client *fakeClient + logs *bytes.Buffer + ifaces []*awgv1.InterfaceStatus + st *state.State +} + +func newSessionEnv(t *testing.T, ifaces ...*awgv1.InterfaceStatus) *sessionEnv { + t.Helper() + e := &sessionEnv{t: t, logs: &bytes.Buffer{}, ifaces: ifaces, st: &state.State{AWG: map[string]state.Entry{}}} + e.client = &fakeClient{status: func() (*awgv1.GetStatusResponse, error) { + return &awgv1.GetStatusResponse{Interfaces: e.ifaces}, nil + }} + return e +} + +func (e *sessionEnv) open() *Session { + e.t.Helper() + dial := func() (awgv1.ManagementServiceClient, func(), error) { return e.client, func() {}, nil } + s, err := Open(context.Background(), dial, netip.MustParseAddr("203.0.113.10"), + []netip.Addr{netip.MustParseAddr("1.1.1.1")}, slog.New(slog.NewTextHandler(e.logs, nil))) + if err != nil { + e.t.Fatal(err) + } + return s +} + +func named(name string, address string, params ...*awgv1.ConfigParam) *awgv1.InterfaceStatus { + return &awgv1.InterfaceStatus{ + Name: name, + Present: true, + PublicKey: bytes.Repeat([]byte{9}, 32), + ListenPort: 51820, + Addresses: []string{address}, + ClientParams: params, + } +} + +func TestOpen(t *testing.T) { + tests := []struct { + name string + dial Dial + wantErr string + }{ + {"dial error", func() (awgv1.ManagementServiceClient, func(), error) { return nil, nil, errors.New("no socket") }, + "connecting to awg-grpc"}, + {"get status error", func() (awgv1.ManagementServiceClient, func(), error) { + c := &fakeClient{status: func() (*awgv1.GetStatusResponse, error) { return nil, errors.New("down") }} + return c, func() {}, nil + }, "getting status"}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + _, err := Open(context.Background(), tt.dial, netip.MustParseAddr("203.0.113.10"), nil, slog.New(slog.DiscardHandler)) + if err == nil || !strings.Contains(err.Error(), tt.wantErr) { + t.Fatalf("err = %v, want it to contain %q", err, tt.wantErr) + } + }) + } + t.Run("lists the interfaces in reported order", func(t *testing.T) { + e := newSessionEnv(t, named("awg1", "10.9.1.1/24"), named("awg0", "10.8.1.1/24")) + if got := e.open().Interfaces(); !slices.Equal(got, []string{"awg1", "awg0"}) { + t.Errorf("Interfaces = %v", got) + } + }) +} + +func TestSessionRequestShape(t *testing.T) { + e := newSessionEnv(t, named("awg0", "10.8.1.1/24"), named("awg1", "10.9.1.1/24")) + s := e.open() + users := map[string][]string{"awg0": {"laptop", "phone"}, "awg1": nil} + + for _, name := range s.Interfaces() { + if _, _, err := s.Prepare(e.st, name, users[name]); err != nil { + t.Fatal(err) + } + if err := s.Apply(context.Background(), e.st, name, users[name]); err != nil { + t.Fatal(err) + } + } + + if len(e.client.applied) != 2 { + t.Fatalf("requests = %d, want 2", len(e.client.applied)) + } + first, second := e.client.applied[0], e.client.applied[1] + if first.GetInterfaceName() != "awg0" || first.GetAllowEmpty() || len(first.GetPeers()) != 2 { + t.Fatalf("awg0 request: allow_empty=%v peers=%d", first.GetAllowEmpty(), len(first.GetPeers())) + } + for i, user := range users["awg0"] { + p := first.GetPeers()[i] + want := e.st.AWG["awg0/"+user] + if !bytes.Equal(p.GetPublicKey(), want.PublicKey()) || len(p.GetPublicKey()) != 32 { + t.Errorf("%s: public key mismatch", user) + } + if !bytes.Equal(p.GetPresharedKey(), want.PresharedKey[:]) { + t.Errorf("%s: preshared key mismatch", user) + } + if p.GetAllowedIp() != want.Route().String() { + t.Errorf("%s: allowed ip = %q", user, p.GetAllowedIp()) + } + } + if second.GetInterfaceName() != "awg1" || !second.GetAllowEmpty() || len(second.GetPeers()) != 0 { + t.Errorf("awg1 request: allow_empty=%v peers=%d", second.GetAllowEmpty(), len(second.GetPeers())) + } +} + +func TestPrepare(t *testing.T) { + t.Run("renders one file per user and reports new entries", func(t *testing.T) { + e := newSessionEnv(t, named("awg0", "10.8.1.1/24")) + s := e.open() + + files, changed, err := s.Prepare(e.st, "awg0", []string{"laptop", "phone"}) + if err != nil || !changed { + t.Fatalf("changed=%v err=%v", changed, err) + } + if got := slices.Sorted(maps.Keys(files)); !slices.Equal(got, []string{"laptop/awg0.conf", "phone/awg0.conf"}) { + t.Errorf("files = %v", got) + } + if !strings.Contains(string(files["phone/awg0.conf"]), "[Peer]") { + t.Errorf("phone config has no peer section") + } + + _, changed, err = s.Prepare(e.st, "awg0", []string{"laptop", "phone"}) + if err != nil || changed { + t.Errorf("second prepare: changed=%v err=%v", changed, err) + } + }) + missing := named("awg0", "10.8.1.1/24") + missing.Present = false + tests := []struct { + name string + iface *awgv1.InterfaceStatus + }{ + {"not present", missing}, + {"no free address", named("awg0", "10.8.1.1/30")}, + {"render error", named("awg0", "10.8.1.1/24", &awgv1.ConfigParam{Key: "S1", Value: "a\nb"})}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + e := newSessionEnv(t, tt.iface) + if _, _, err := e.open().Prepare(e.st, "awg0", []string{"phone", "laptop"}); err == nil { + t.Fatal("want an error") + } + }) + } +} + +func TestApplyError(t *testing.T) { + e := newSessionEnv(t, named("awg0", "10.8.1.1/24")) + e.client.apply = func(*awgv1.ApplyPeersRequest) (*awgv1.ApplyPeersResponse, error) { + return nil, errors.New("rejected") + } + s := e.open() + if _, _, err := s.Prepare(e.st, "awg0", []string{"phone"}); err != nil { + t.Fatal(err) + } + if err := s.Apply(context.Background(), e.st, "awg0", []string{"phone"}); err == nil || !strings.Contains(err.Error(), "rejected") { + t.Fatalf("err = %v", err) + } +} + +func TestApplyLogsResult(t *testing.T) { + unknown := bytes.Repeat([]byte{7}, 32) + tests := []struct { + name string + response *awgv1.ApplyPeersResponse + want []string + silent bool + }{ + {"user names", nil, []string{"added=[phone]"}, false}, + {"key without entry", &awgv1.ApplyPeersResponse{Updated: [][]byte{unknown}}, + []string{base64.StdEncoding.EncodeToString(unknown)}, false}, + {"empty result", &awgv1.ApplyPeersResponse{}, nil, true}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + e := newSessionEnv(t, named("awg0", "10.8.1.1/24")) + e.client.apply = func(r *awgv1.ApplyPeersRequest) (*awgv1.ApplyPeersResponse, error) { + if tt.response != nil { + return tt.response, nil + } + return &awgv1.ApplyPeersResponse{Added: [][]byte{r.GetPeers()[0].GetPublicKey()}}, nil + } + s := e.open() + if _, _, err := s.Prepare(e.st, "awg0", []string{"phone"}); err != nil { + t.Fatal(err) + } + + if err := s.Apply(context.Background(), e.st, "awg0", []string{"phone"}); err != nil { + t.Fatal(err) + } + + for _, w := range tt.want { + if !strings.Contains(e.logs.String(), w) { + t.Errorf("log lacks %q: %s", w, e.logs) + } + } + if tt.silent && strings.Contains(e.logs.String(), "peers changed") { + t.Errorf("empty result was logged: %s", e.logs) + } + }) + } +} + +func TestApplySwitchOff(t *testing.T) { + e := newSessionEnv(t, named("awg0", "10.8.1.1/24")) + s := e.open() + if _, _, err := s.Prepare(e.st, "awg0", []string{"laptop", "phone"}); err != nil { + t.Fatal(err) + } + laptop := e.st.AWG["awg0/laptop"] + e.client.apply = func(*awgv1.ApplyPeersRequest) (*awgv1.ApplyPeersResponse, error) { + return &awgv1.ApplyPeersResponse{Removed: [][]byte{laptop.PublicKey()}}, nil + } + + if err := s.Apply(context.Background(), e.st, "awg0", []string{"phone"}); err != nil { + t.Fatal(err) + } + + if got := e.client.applied[0]; len(got.GetPeers()) != 1 { + t.Fatalf("peers = %d, want 1", len(got.GetPeers())) + } + if !strings.Contains(e.logs.String(), "removed=[laptop]") { + t.Errorf("log does not name the removed user: %s", e.logs) + } +} + +func TestLogHasNoSecrets(t *testing.T) { + const headerKey = "HDRKEYSECRET" + const leak = "LEAKSECRET" + e := newSessionEnv(t, + named("awg0", "10.8.1.1/24", &awgv1.ConfigParam{Key: "HeaderProtectionKey", Value: headerKey}), + named("awg1", "10.9.1.1/24", &awgv1.ConfigParam{Key: "S1", Value: "x\n" + leak})) + e.client.apply = func(r *awgv1.ApplyPeersRequest) (*awgv1.ApplyPeersResponse, error) { + return &awgv1.ApplyPeersResponse{Added: [][]byte{r.GetPeers()[0].GetPublicKey()}}, nil + } + s := e.open() + + for _, name := range s.Interfaces() { + _, _, err := s.Prepare(e.st, name, []string{"phone"}) + if err != nil { + e.logs.WriteString(err.Error()) + continue + } + if err := s.Apply(context.Background(), e.st, name, []string{"phone"}); err != nil { + t.Fatal(err) + } + } + + logs := e.logs.String() + if logs == "" { + t.Fatal("no log output") + } + secrets := []string{headerKey, leak} + for _, entry := range e.st.AWG { + secrets = append(secrets, entry.PrivateKey.String(), entry.PresharedKey.String()) + } + for _, s := range secrets { + if strings.Contains(logs, s) { + t.Errorf("log contains a secret: %q", s) + } + } +} diff --git a/internal/clients/testdata/phone.conf b/internal/awg/testdata/phone.conf similarity index 100% rename from internal/clients/testdata/phone.conf rename to internal/awg/testdata/phone.conf diff --git a/internal/clients/mirror.go b/internal/clients/mirror.go index 8cebee6..c9308a7 100644 --- a/internal/clients/mirror.go +++ b/internal/clients/mirror.go @@ -1,3 +1,4 @@ +// Package clients mirrors client files into the clients directory. package clients import ( @@ -17,19 +18,17 @@ const ( clientDirMode = 0o755 ) -// Path returns the slash-separated path of a user's config on an interface, relative to the -// clients directory. -func Path(user, ifaceName string) string { - return user + "/" + ifaceName + ".conf" +// Path returns the slash-separated path of a user's client file on an interface, relative to the +// clients directory. suffix is the backend's file suffix, such as ".conf". +func Path(user, ifaceName, suffix string) string { + return user + "/" + ifaceName + suffix } -// Existing returns the current content of the configs in dir that belong to the interfaces in -// ifaces, keyed by Path. It reads as much as it can and returns the errors of the rest joined. -func Existing(dir string, ifaces map[string]bool) (map[string][]byte, error) { +// Existing returns the current content of the files in dir that end in suffix and belong to an +// interface for which keep returns true, keyed by Path. It reads as much as it can and returns +// the errors of the rest joined. +func Existing(dir, suffix string, keep func(ifaceName string) bool) (map[string][]byte, error) { files := make(map[string][]byte) - if len(ifaces) == 0 { - return files, nil - } userDirs, err := os.ReadDir(dir) if err != nil { return files, err @@ -39,14 +38,14 @@ func Existing(dir string, ifaces map[string]bool) (map[string][]byte, error) { if !d.IsDir() { continue } - confs, err := os.ReadDir(filepath.Join(dir, d.Name())) + entries, err := os.ReadDir(filepath.Join(dir, d.Name())) if err != nil { errs = append(errs, err) continue } - for _, f := range confs { - ifaceName, ok := strings.CutSuffix(f.Name(), ".conf") - if !ok || f.IsDir() || !ifaces[ifaceName] { + for _, f := range entries { + ifaceName, ok := strings.CutSuffix(f.Name(), suffix) + if !ok || f.IsDir() || !keep(ifaceName) { continue } content, err := os.ReadFile(filepath.Join(dir, d.Name(), f.Name())) @@ -54,7 +53,7 @@ func Existing(dir string, ifaces map[string]bool) (map[string][]byte, error) { errs = append(errs, err) continue } - files[Path(d.Name(), ifaceName)] = content + files[Path(d.Name(), ifaceName, suffix)] = content } } return files, errors.Join(errs...) diff --git a/internal/clients/mirror_test.go b/internal/clients/mirror_test.go index 89cef37..3aeac07 100644 --- a/internal/clients/mirror_test.go +++ b/internal/clients/mirror_test.go @@ -4,6 +4,7 @@ import ( "os" "path/filepath" "reflect" + "slices" "testing" "time" ) @@ -149,23 +150,39 @@ func TestMirrorKeepsMtimeOfUnchangedFile(t *testing.T) { } func TestExisting(t *testing.T) { + only := func(names ...string) func(string) bool { + return func(name string) bool { return slices.Contains(names, name) } + } + all := func(string) bool { return true } + none := func(string) bool { return false } tests := []struct { name string existing map[string]string - ifaces map[string]bool + suffix string + keep func(string) bool want map[string][]byte }{ { - name: "returns only the configs of the named interfaces", + name: "returns only the files of the kept interfaces", existing: map[string]string{ "phone/awg0.conf": "a", "phone/awg1.conf": "b", "laptop/awg1.conf": "c", "phone/notes.txt": "d", }, - ifaces: map[string]bool{"awg1": true}, + suffix: ".conf", + keep: only("awg1"), want: map[string][]byte{"phone/awg1.conf": []byte("b"), "laptop/awg1.conf": []byte("c")}, }, { - name: "no interfaces reads nothing", + name: "keeping all returns every file of the suffix", + existing: map[string]string{"phone/awg0.conf": "a", "phone/vless.txt": "b", "laptop/vless.txt": "c"}, + suffix: ".txt", + keep: all, + want: map[string][]byte{"phone/vless.txt": []byte("b"), "laptop/vless.txt": []byte("c")}, + }, + { + name: "keeping none reads nothing", existing: map[string]string{"phone/awg0.conf": "a"}, + suffix: ".conf", + keep: none, want: map[string][]byte{}, }, } @@ -175,7 +192,7 @@ func TestExisting(t *testing.T) { for rel, content := range tt.existing { writeFile(t, filepath.Join(dir, filepath.FromSlash(rel)), content, 0o600) } - got, err := Existing(dir, tt.ifaces) + got, err := Existing(dir, tt.suffix, tt.keep) if err != nil { t.Fatal(err) } @@ -185,14 +202,20 @@ func TestExisting(t *testing.T) { }) } t.Run("missing directory is an error", func(t *testing.T) { - if _, err := Existing(filepath.Join(t.TempDir(), "gone"), map[string]bool{"awg0": true}); err == nil { + if _, err := Existing(filepath.Join(t.TempDir(), "gone"), ".conf", all); err == nil { t.Fatal("want an error") } }) } func TestPath(t *testing.T) { - if got := Path("phone", "awg0"); got != "phone/awg0.conf" { - t.Errorf("Path = %q, want phone/awg0.conf", got) + tests := []struct{ suffix, want string }{ + {".conf", "phone/awg0.conf"}, + {".txt", "phone/awg0.txt"}, + } + for _, tt := range tests { + if got := Path("phone", "awg0", tt.suffix); got != tt.want { + t.Errorf("Path(%q) = %q, want %q", tt.suffix, got, tt.want) + } } } diff --git a/internal/pass/backend.go b/internal/pass/backend.go new file mode 100644 index 0000000..bf89e54 --- /dev/null +++ b/internal/pass/backend.go @@ -0,0 +1,28 @@ +package pass + +import ( + "context" + + "github.com/mrcsin/usher/internal/state" +) + +// Session is one pass's connection to a backend. Open lists the backend's interfaces; the +// session prepares and applies one interface at a time. +type Session interface { + // Interfaces returns the names of the interfaces the backend reported. + Interfaces() []string + // Prepare enrolls the users on the interface in st and returns their client files, keyed by + // clients.Path, and whether it changed st. The pass saves st before any Apply. + Prepare(st *state.State, name string, users []string) (files map[string][]byte, changed bool, err error) + // Apply makes the interface hold exactly the users. + Apply(ctx context.Context, st *state.State, name string, users []string) error + // Close releases the connection. + Close() +} + +// Backend is a VPN server usher moves users into. +type Backend struct { + Name string // used in logs, such as "awg-grpc" + Suffix string // suffix of the client files the backend renders, such as ".conf" + Open func(ctx context.Context) (Session, error) +} diff --git a/internal/pass/pass.go b/internal/pass/pass.go index 01bebfb..f9d5731 100644 --- a/internal/pass/pass.go +++ b/internal/pass/pass.go @@ -2,43 +2,45 @@ package pass import ( "context" - "encoding/base64" - "errors" "fmt" "log/slog" "maps" "slices" - "time" - - awgv1 "github.com/mrcsin/awg-grpc/gen/awg/v1" + "strings" "github.com/mrcsin/usher/internal/clients" "github.com/mrcsin/usher/internal/config" "github.com/mrcsin/usher/internal/state" ) -// callTimeout bounds each call to awg-grpc. -const callTimeout = 10 * time.Second - -// Dial opens a connection to awg-grpc. It returns the client and the function that closes the -// connection. -type Dial func() (awgv1.ManagementServiceClient, func(), error) - // Pass runs reconcile passes. It keeps the last valid usher.yml it read. type Pass struct { settings Settings - dial Dial + backends []Backend log *slog.Logger last map[string][]string } -// New returns a Pass that works with the paths and server facts in settings. -func New(settings Settings, dial Dial, log *slog.Logger) *Pass { - return &Pass{settings: settings, dial: dial, log: log} +// New returns a Pass that works with the paths in settings and moves users into backends. +func New(settings Settings, backends []Backend, log *slog.Logger) *Pass { + return &Pass{settings: settings, backends: backends, log: log} +} + +// openBackend is a backend with its session for this pass. +type openBackend struct { + Backend + session Session +} + +// interfaceKey identifies an interface of one backend, by the backend's index in the open list. +type interfaceKey struct { + backend int + name string } // Run executes one pass. It logs every failure and never panics on one; a failure of one -// interface leaves the others to apply. +// interface leaves the others to apply, and a backend that is down leaves the other backends to +// apply. func (p *Pass) Run(ctx context.Context) { cfg, ok := p.loadConfig() if !ok { @@ -49,37 +51,36 @@ func (p *Pass) Run(ctx context.Context) { p.log.Error("loading state", "error", err) return } - client, closeConn, err := p.dial() - if err != nil { - p.log.Error("connecting to awg-grpc", "error", err) - return - } - defer closeConn() - - callCtx, cancel := context.WithTimeout(ctx, callTimeout) - status, err := client.GetStatus(callCtx, &awgv1.GetStatusRequest{}) - cancel() - if err != nil { - p.log.Error("getting status", "error", err) - return - } + open, down := p.openBackends(ctx) + defer func() { + for _, b := range open { + b.session.Close() + } + }() byInterface := usersByInterface(cfg) - reported := make(map[string]bool, len(status.GetInterfaces())) - rendered := make(map[string]map[string][]byte) - failed := make(map[string]bool) + owners := interfaceOwners(open) + failed := make(map[interfaceKey]bool) + rendered := make(map[interfaceKey]map[string][]byte) changed := false - for _, iface := range status.GetInterfaces() { - name := iface.GetName() - reported[name] = true - files, entriesChanged, err := p.prepare(st, iface, byInterface[name]) - if err != nil { - p.log.Error("interface failed", "interface", name, "error", err) - failed[name] = true - continue + for i, b := range open { + for _, name := range b.session.Interfaces() { + key := interfaceKey{i, name} + if len(owners[name]) > 1 { + err := fmt.Errorf("reported by %s", strings.Join(owners[name], " and ")) + p.log.Error("interface failed", "backend", b.Name, "interface", name, "error", err) + failed[key] = true + continue + } + files, entriesChanged, err := b.session.Prepare(st, name, byInterface[name]) + if err != nil { + p.log.Error("interface failed", "backend", b.Name, "interface", name, "error", err) + failed[key] = true + continue + } + rendered[key] = files + changed = changed || entriesChanged } - rendered[name] = files - changed = changed || entriesChanged } if changed { @@ -89,35 +90,67 @@ func (p *Pass) Run(ctx context.Context) { } } - for _, iface := range status.GetInterfaces() { - name := iface.GetName() - if failed[name] { - continue - } - if err := p.apply(ctx, client, st, name, byInterface[name]); err != nil { - p.log.Error("interface failed", "interface", name, "error", err) - failed[name] = true + for i, b := range open { + for _, name := range b.session.Interfaces() { + key := interfaceKey{i, name} + if failed[key] { + continue + } + if err := b.session.Apply(ctx, st, name, byInterface[name]); err != nil { + p.log.Error("interface failed", "backend", b.Name, "interface", name, "error", err) + failed[key] = true + } } } - p.logUnknown(cfg, reported) + // A backend that is down hides its interface names, so every unreferenced name could be one. + if len(down) == 0 { + p.logUnknown(cfg, owners) + } desired := make(map[string][]byte) - for name, files := range rendered { - if !failed[name] { + for key, files := range rendered { + if !failed[key] { maps.Copy(desired, files) } } - kept, err := clients.Existing(p.settings.ClientsDir, failed) - if err != nil { - p.log.Error("keeping client configs", "error", err) + for i, b := range open { + p.keepFiles(desired, b.Backend, func(name string) bool { return failed[interfaceKey{i, name}] }) + } + for _, b := range down { + p.keepFiles(desired, b, func(string) bool { return true }) } - maps.Copy(desired, kept) if err := clients.Mirror(p.settings.ClientsDir, desired); err != nil { p.log.Error("mirroring clients directory", "error", err) } } +// openBackends opens a session for every backend and returns the backends that failed to open. +func (p *Pass) openBackends(ctx context.Context) ([]openBackend, []Backend) { + var open []openBackend + var down []Backend + for _, b := range p.backends { + session, err := b.Open(ctx) + if err != nil { + p.log.Error("backend is down", "backend", b.Name, "error", err) + down = append(down, b) + continue + } + open = append(open, openBackend{b, session}) + } + return open, down +} + +// keepFiles copies the current client files of the backend's interfaces that keep selects into +// desired. +func (p *Pass) keepFiles(desired map[string][]byte, b Backend, keep func(ifaceName string) bool) { + kept, err := clients.Existing(p.settings.ClientsDir, b.Suffix, keep) + if err != nil { + p.log.Error("keeping client files", "backend", b.Name, "error", err) + } + maps.Copy(desired, kept) +} + func (p *Pass) loadConfig() (map[string][]string, bool) { cfg, err := config.Load(p.settings.ConfigPath) if err == nil { @@ -132,83 +165,23 @@ func (p *Pass) loadConfig() (map[string][]string, bool) { return p.last, true } -func (p *Pass) prepare(st *state.State, iface *awgv1.InterfaceStatus, users []string) (map[string][]byte, bool, error) { - if !iface.GetPresent() { - return nil, false, errors.New("interface is not present in the kernel") - } - changed, err := enroll(p.log, st, iface, users) - if err != nil { - return nil, false, err - } - files := make(map[string][]byte, len(users)) - for _, user := range users { - content, err := clients.Render(st.AWG[state.EntryName(iface.GetName(), user)], iface, p.settings.Host, p.settings.DNS) - if err != nil { - return nil, false, fmt.Errorf("user %s: %w", user, err) - } - files[clients.Path(user, iface.GetName())] = content - } - return files, changed, nil -} - -func (p *Pass) apply(ctx context.Context, client awgv1.ManagementServiceClient, st *state.State, ifaceName string, users []string) error { - peers := make([]*awgv1.Peer, 0, len(users)) - for _, user := range users { - e := st.AWG[state.EntryName(ifaceName, user)] - peers = append(peers, &awgv1.Peer{ - PublicKey: e.PublicKey(), - PresharedKey: e.PresharedKey[:], - AllowedIp: e.Route().String(), - }) - } - - callCtx, cancel := context.WithTimeout(ctx, callTimeout) - defer cancel() - resp, err := client.ApplyPeers(callCtx, &awgv1.ApplyPeersRequest{ - InterfaceName: ifaceName, - Peers: peers, - AllowEmpty: len(peers) == 0, - }) - if err != nil { - return fmt.Errorf("applying peers: %w", err) - } - - names := publicKeyNames(st, ifaceName) - added, removed, updated := nameKeys(names, resp.GetAdded()), nameKeys(names, resp.GetRemoved()), nameKeys(names, resp.GetUpdated()) - if len(added)+len(removed)+len(updated) > 0 { - p.log.Info("peers changed", "interface", ifaceName, "added", added, "removed", removed, "updated", updated) - } - return nil -} - -// publicKeyNames maps the public key of every entry of the interface, switched off or not, to -// its user name. -func publicKeyNames(st *state.State, ifaceName string) map[string]string { - names := make(map[string]string) - for user, e := range st.InterfaceEntries(ifaceName) { - names[string(e.PublicKey())] = user - } - return names -} - -func nameKeys(names map[string]string, keys [][]byte) []string { - out := make([]string, 0, len(keys)) - for _, k := range keys { - if user, ok := names[string(k)]; ok { - out = append(out, user) - } else { - out = append(out, base64.StdEncoding.EncodeToString(k)) +// interfaceOwners maps every reported interface name to the names of the backends that +// reported it. +func interfaceOwners(open []openBackend) map[string][]string { + owners := make(map[string][]string) + for _, b := range open { + for _, name := range b.session.Interfaces() { + owners[name] = append(owners[name], b.Name) } } - slices.Sort(out) - return out + return owners } -func (p *Pass) logUnknown(cfg map[string][]string, reported map[string]bool) { +func (p *Pass) logUnknown(cfg map[string][]string, owners map[string][]string) { for _, user := range slices.Sorted(maps.Keys(cfg)) { for _, ifaceName := range cfg[user] { - if !reported[ifaceName] { - p.log.Error("interface is not reported by awg-grpc", "interface", ifaceName, "user", user) + if _, ok := owners[ifaceName]; !ok { + p.log.Error("interface is not reported by any backend", "interface", ifaceName, "user", user) } } } diff --git a/internal/pass/pass_test.go b/internal/pass/pass_test.go index 3fc8154..ed4ec5a 100644 --- a/internal/pass/pass_test.go +++ b/internal/pass/pass_test.go @@ -3,8 +3,8 @@ package pass import ( "bytes" "context" - "encoding/base64" "errors" + "fmt" "log/slog" "maps" "net/netip" @@ -15,50 +15,107 @@ import ( "testing" "time" - awgv1 "github.com/mrcsin/awg-grpc/gen/awg/v1" - "github.com/mrcsin/usher/internal/state" ) +// fakeSession is a Session that renders "new " into one file per user and enrolls a +// state entry the first time an interface has users. +type fakeSession struct { + names []string + suffix string + + prepareErr map[string]error + applyErr map[string]error + onApply func(name string) + + applied map[string][]string + closed bool +} + +func (f *fakeSession) Interfaces() []string { return f.names } + +func (f *fakeSession) Prepare(st *state.State, name string, users []string) (map[string][]byte, bool, error) { + if err := f.prepareErr[name]; err != nil { + return nil, false, err + } + files := make(map[string][]byte, len(users)) + for _, user := range users { + files[user+"/"+name+f.suffix] = []byte("new " + name) + } + entryName := state.EntryName("fake", name) + _, enrolled := st.AWG[entryName] + changed := len(users) > 0 && !enrolled + if changed { + st.AWG[entryName] = state.Entry{ + Address: netip.MustParseAddr("10.0.0.2"), + PrivateKey: state.Key(bytes.Repeat([]byte{1}, state.KeyLength)), + PresharedKey: state.Key(bytes.Repeat([]byte{2}, state.KeyLength)), + } + } + return files, changed, nil +} + +func (f *fakeSession) Apply(_ context.Context, _ *state.State, name string, users []string) error { + if f.onApply != nil { + f.onApply(name) + } + if f.applied == nil { + f.applied = map[string][]string{} + } + f.applied[name] = users + return f.applyErr[name] +} + +func (f *fakeSession) Close() { f.closed = true } + +type fakeBackend struct { + session *fakeSession + openErr error + opens int +} + +func (f *fakeBackend) backend(name, suffix string) Backend { + return Backend{Name: name, Suffix: suffix, Open: func(context.Context) (Session, error) { + f.opens++ + if f.openErr != nil { + return nil, f.openErr + } + f.session.suffix = suffix + return f.session, nil + }} +} + type passEnv struct { t *testing.T settings Settings - client *fakeClient logs *bytes.Buffer pass *Pass - dial Dial - ifaces []*awgv1.InterfaceStatus + backends []Backend } -func newPassEnv(t *testing.T, yml string, ifaces ...*awgv1.InterfaceStatus) *passEnv { +func newPassEnv(t *testing.T, yml string, backends ...Backend) *passEnv { t.Helper() root := t.TempDir() e := &passEnv{ t: t, settings: Settings{ - Host: netip.MustParseAddr("203.0.113.10"), - DNS: []netip.Addr{netip.MustParseAddr("1.1.1.1")}, ConfigPath: filepath.Join(root, "usher.yml"), ClientsDir: filepath.Join(root, "clients"), StatePath: filepath.Join(root, "users.json"), }, - logs: &bytes.Buffer{}, - ifaces: ifaces, + logs: &bytes.Buffer{}, + backends: backends, } - e.client = &fakeClient{status: func() (*awgv1.GetStatusResponse, error) { - return &awgv1.GetStatusResponse{Interfaces: e.ifaces}, nil - }} if err := os.MkdirAll(e.settings.ClientsDir, 0o755); err != nil { t.Fatal(err) } e.writeConfig(yml) - e.dial = func() (awgv1.ManagementServiceClient, func(), error) { return e.client, func() {}, nil } e.newPass() return e } func (e *passEnv) newPass() { - e.pass = New(e.settings, e.dial, slog.New(slog.NewTextHandler(e.logs, nil))) + e.pass = New(e.settings, e.backends, slog.New(slog.NewTextHandler(e.logs, nil))) } func (e *passEnv) writeConfig(yml string) { @@ -101,88 +158,53 @@ func (e *passEnv) clientFiles() map[string]string { return files } -func (e *passEnv) loadState() *state.State { - e.t.Helper() - st, err := state.Load(e.settings.StatePath) - if err != nil { - e.t.Fatal(err) - } - return st +func sortedKeys(files map[string]string) []string { + return slices.Sorted(maps.Keys(files)) } -func (e *passEnv) appliedInterfaces() []string { - var names []string - for _, r := range e.client.applied { - names = append(names, r.GetInterfaceName()) - } - return names -} - -func named(name string, address string, params ...*awgv1.ConfigParam) *awgv1.InterfaceStatus { - return &awgv1.InterfaceStatus{ - Name: name, - Present: true, - PublicKey: bytes.Repeat([]byte{9}, 32), - ListenPort: 51820, - Addresses: []string{address}, - ClientParams: params, - } -} - -func TestRunRequestShape(t *testing.T) { - e := newPassEnv(t, "phone: [awg0]\nlaptop: [awg0]\nguest: []\n", - named("awg0", "10.8.1.1/24"), named("awg1", "10.9.1.1/24")) +func TestRunOneBackend(t *testing.T) { + fb := &fakeBackend{session: &fakeSession{names: []string{"awg0", "awg1"}}} + e := newPassEnv(t, "phone: [awg0]\nlaptop: [awg0]\nguest: []\n", fb.backend("awg-grpc", ".conf")) e.run() - if got := e.appliedInterfaces(); !slices.Equal(got, []string{"awg0", "awg1"}) { - t.Fatalf("applied interfaces = %v", got) - } - first, second := e.client.applied[0], e.client.applied[1] - if first.GetAllowEmpty() || len(first.GetPeers()) != 2 { - t.Fatalf("awg0 request: allow_empty=%v peers=%d", first.GetAllowEmpty(), len(first.GetPeers())) + if got := fb.session.applied["awg0"]; !slices.Equal(got, []string{"laptop", "phone"}) { + t.Errorf("awg0 users = %v", got) } - st := e.loadState() - // peers are ordered by user name: laptop, phone - for i, user := range []string{"laptop", "phone"} { - p := first.GetPeers()[i] - want := st.AWG["awg0/"+user] - if !bytes.Equal(p.GetPublicKey(), want.PublicKey()) || len(p.GetPublicKey()) != 32 { - t.Errorf("%s: public key mismatch", user) - } - if !bytes.Equal(p.GetPresharedKey(), want.PresharedKey[:]) { - t.Errorf("%s: preshared key mismatch", user) - } - if p.GetAllowedIp() != want.Route().String() { - t.Errorf("%s: allowed ip = %q", user, p.GetAllowedIp()) - } + if users, ok := fb.session.applied["awg1"]; !ok || len(users) != 0 { + t.Errorf("awg1 applied = %v, %v", users, ok) } - if !second.GetAllowEmpty() || len(second.GetPeers()) != 0 { - t.Errorf("awg1 request: allow_empty=%v peers=%d", second.GetAllowEmpty(), len(second.GetPeers())) + if got := sortedKeys(e.clientFiles()); !slices.Equal(got, []string{"laptop/awg0.conf", "phone/awg0.conf"}) { + t.Errorf("client files = %v", got) } - files := e.clientFiles() - if len(files) != 2 || files["phone/awg0.conf"] == "" || files["laptop/awg0.conf"] == "" { - t.Errorf("clients files = %v", slices.Sorted(maps.Keys(files))) + if !fb.session.closed { + t.Error("session was not closed") } } func TestRunSavesStateBeforeApply(t *testing.T) { - e := newPassEnv(t, "phone: [awg0]\n", named("awg0", "10.8.1.1/24")) + fb := &fakeBackend{session: &fakeSession{names: []string{"awg0"}}} + e := newPassEnv(t, "phone: [awg0]\n", fb.backend("awg-grpc", ".conf")) var entriesAtApply int - e.client.apply = func(*awgv1.ApplyPeersRequest) (*awgv1.ApplyPeersResponse, error) { - entriesAtApply = len(e.loadState().AWG) - return &awgv1.ApplyPeersResponse{}, nil + fb.session.onApply = func(string) { + st, err := state.Load(e.settings.StatePath) + if err != nil { + t.Error(err) + return + } + entriesAtApply = len(st.AWG) } e.run() if entriesAtApply != 1 { - t.Fatalf("state held %d entries at the first ApplyPeers, want 1", entriesAtApply) + t.Fatalf("state held %d entries at the first Apply, want 1", entriesAtApply) } old := time.Date(2020, 1, 1, 0, 0, 0, 0, time.UTC) if err := os.Chtimes(e.settings.StatePath, old, old); err != nil { t.Fatal(err) } + e.writeConfig("phone: []\n") e.run() info, err := os.Stat(e.settings.StatePath) if err != nil { @@ -195,15 +217,16 @@ func TestRunSavesStateBeforeApply(t *testing.T) { func TestRunInvalidConfig(t *testing.T) { t.Run("last valid file is used", func(t *testing.T) { - e := newPassEnv(t, "phone: [awg0]\n", named("awg0", "10.8.1.1/24")) + fb := &fakeBackend{session: &fakeSession{names: []string{"awg0"}}} + e := newPassEnv(t, "phone: [awg0]\n", fb.backend("awg-grpc", ".conf")) e.run() e.writeConfig("phone: [awg0\n") - e.client.applied = nil + fb.session.applied = nil e.run() - if len(e.client.applied) != 1 || len(e.client.applied[0].GetPeers()) != 1 { - t.Fatalf("expected the last valid file to be applied, got %d requests", len(e.client.applied)) + if got := fb.session.applied["awg0"]; !slices.Equal(got, []string{"phone"}) { + t.Fatalf("expected the last valid file to be applied, got %v", got) } if !strings.Contains(e.logs.String(), "usher.yml") { t.Errorf("log does not name the file: %s", e.logs) @@ -213,13 +236,14 @@ func TestRunInvalidConfig(t *testing.T) { } }) t.Run("no earlier valid file", func(t *testing.T) { - e := newPassEnv(t, "phone: [awg0\n", named("awg0", "10.8.1.1/24")) + fb := &fakeBackend{session: &fakeSession{names: []string{"awg0"}}} + e := newPassEnv(t, "phone: [awg0\n", fb.backend("awg-grpc", ".conf")) e.writeClient("keep/awg0.conf", "keep") e.run() - if e.client.statusCalls != 0 || len(e.client.applied) != 0 { - t.Errorf("requests were sent: status=%d apply=%d", e.client.statusCalls, len(e.client.applied)) + if fb.opens != 0 { + t.Errorf("backend was opened") } if _, ok := e.clientFiles()["keep/awg0.conf"]; !ok { t.Errorf("clients directory changed") @@ -234,7 +258,7 @@ func TestRunStopsBeforeRequests(t *testing.T) { tests := []struct { name string setup func(e *passEnv) - wantStatus int + wantOpens int wantLogHas string stateBefore string }{ @@ -243,31 +267,25 @@ func TestRunStopsBeforeRequests(t *testing.T) { t.Fatal(err) } }, 0, "loading state", "{"}, - {"get status error", func(e *passEnv) { - e.client.status = func() (*awgv1.GetStatusResponse, error) { return nil, errors.New("down") } - }, 1, "getting status", ""}, {"state cannot be saved", func(e *passEnv) { e.settings.StatePath = filepath.Join(filepath.Dir(e.settings.StatePath), "missing", "users.json") e.newPass() }, 1, "saving state", ""}, - {"dial error", func(e *passEnv) { - e.dial = func() (awgv1.ManagementServiceClient, func(), error) { return nil, nil, errors.New("no socket") } - e.newPass() - }, 0, "connecting to awg-grpc", ""}, } for _, tt := range tests { t.Run(tt.name, func(t *testing.T) { - e := newPassEnv(t, "phone: [awg0]\n", named("awg0", "10.8.1.1/24")) + fb := &fakeBackend{session: &fakeSession{names: []string{"awg0"}}} + e := newPassEnv(t, "phone: [awg0]\n", fb.backend("awg-grpc", ".conf")) e.writeClient("stale/awg0.conf", "stale") tt.setup(e) e.run() - if len(e.client.applied) != 0 { - t.Errorf("ApplyPeers was called") + if len(fb.session.applied) != 0 { + t.Errorf("Apply was called") } - if e.client.statusCalls != tt.wantStatus { - t.Errorf("GetStatus calls = %d, want %d", e.client.statusCalls, tt.wantStatus) + if fb.opens != tt.wantOpens { + t.Errorf("opens = %d, want %d", fb.opens, tt.wantOpens) } if !strings.Contains(e.logs.String(), tt.wantLogHas) { t.Errorf("log lacks %q: %s", tt.wantLogHas, e.logs) @@ -288,155 +306,174 @@ func TestRunStopsBeforeRequests(t *testing.T) { } func TestRunInterfaceFailure(t *testing.T) { - missing := named("awg0", "10.8.1.1/24") - missing.Present = false tests := []struct { name string - failing *awgv1.InterfaceStatus - applyFails bool + session *fakeSession wantApplied []string }{ - {"not present", missing, false, []string{"awg1"}}, - {"no free address", named("awg0", "10.8.1.1/30"), false, []string{"awg1"}}, - {"render error", named("awg0", "10.8.1.1/24", &awgv1.ConfigParam{Key: "S1", Value: "a\nb"}), false, []string{"awg1"}}, - {"apply error", named("awg0", "10.8.1.1/24"), true, []string{"awg0", "awg1"}}, + {"prepare error", &fakeSession{prepareErr: map[string]error{"awg0": errors.New("not present")}}, []string{"awg1"}}, + {"apply error", &fakeSession{applyErr: map[string]error{"awg0": errors.New("rejected")}}, []string{"awg0", "awg1"}}, } for _, tt := range tests { t.Run(tt.name, func(t *testing.T) { - e := newPassEnv(t, "phone: [awg0, awg1]\nlaptop: [awg0]\n", tt.failing, named("awg1", "10.9.1.1/24")) + tt.session.names = []string{"awg0", "awg1"} + fb := &fakeBackend{session: tt.session} + e := newPassEnv(t, "phone: [awg0, awg1]\nlaptop: [awg0]\n", fb.backend("awg-grpc", ".conf")) e.writeClient("phone/awg0.conf", "old phone") e.writeClient("laptop/awg0.conf", "old laptop") - e.client.apply = func(r *awgv1.ApplyPeersRequest) (*awgv1.ApplyPeersResponse, error) { - if tt.applyFails && r.GetInterfaceName() == "awg0" { - return nil, errors.New("rejected") - } - return &awgv1.ApplyPeersResponse{}, nil - } e.run() - if got := e.appliedInterfaces(); !slices.Equal(got, tt.wantApplied) { + if got := slices.Sorted(maps.Keys(tt.session.applied)); !slices.Equal(got, tt.wantApplied) { t.Errorf("applied = %v, want %v", got, tt.wantApplied) } files := e.clientFiles() if files["phone/awg0.conf"] != "old phone" || files["laptop/awg0.conf"] != "old laptop" { - t.Errorf("failed interface lost its files: %v", slices.Sorted(maps.Keys(files))) + t.Errorf("failed interface lost its files: %v", sortedKeys(files)) } - if !strings.Contains(files["phone/awg1.conf"], "[Peer]") { - t.Errorf("awg1 config missing") + if files["phone/awg1.conf"] != "new awg1" { + t.Errorf("awg1 file missing: %v", sortedKeys(files)) } }) } } func TestRunUnknownInterface(t *testing.T) { - e := newPassEnv(t, "phone: [awg0, awg9]\n", named("awg0", "10.8.1.1/24")) + fb := &fakeBackend{session: &fakeSession{names: []string{"awg0"}}} + e := newPassEnv(t, "phone: [awg0, awg9]\n", fb.backend("awg-grpc", ".conf")) e.writeClient("phone/awg9.conf", "orphan") e.run() - if got := e.appliedInterfaces(); !slices.Equal(got, []string{"awg0"}) { - t.Errorf("applied = %v", got) - } - if !strings.Contains(e.logs.String(), "awg9") { + if !strings.Contains(e.logs.String(), "not reported by any backend") || !strings.Contains(e.logs.String(), "awg9") { t.Errorf("log does not name awg9: %s", e.logs) } files := e.clientFiles() if len(files) != 1 || files["phone/awg0.conf"] == "" { - t.Errorf("clients files = %v", slices.Sorted(maps.Keys(files))) + t.Errorf("clients files = %v", sortedKeys(files)) } } -func TestRunSwitchOff(t *testing.T) { - e := newPassEnv(t, "phone: [awg0]\nlaptop: [awg0]\n", named("awg0", "10.8.1.1/24")) +func TestRunTwoBackends(t *testing.T) { + awg := &fakeBackend{session: &fakeSession{names: []string{"awg0"}}} + xray := &fakeBackend{session: &fakeSession{names: []string{"vless"}}} + e := newPassEnv(t, "phone: [awg0, vless]\n", awg.backend("awg-grpc", ".conf"), xray.backend("xray", ".txt")) + e.run() - laptop := e.loadState().AWG["awg0/laptop"] - laptopPub := laptop.PublicKey() - e.client.applied = nil - e.client.apply = func(*awgv1.ApplyPeersRequest) (*awgv1.ApplyPeersResponse, error) { - return &awgv1.ApplyPeersResponse{Removed: [][]byte{laptopPub}}, nil + + if got := awg.session.applied["awg0"]; !slices.Equal(got, []string{"phone"}) { + t.Errorf("awg applied = %v", awg.session.applied) + } + if got := xray.session.applied["vless"]; !slices.Equal(got, []string{"phone"}) { + t.Errorf("xray applied = %v", xray.session.applied) + } + want := []string{"phone/awg0.conf", "phone/vless.txt"} + if got := sortedKeys(e.clientFiles()); !slices.Equal(got, want) { + t.Errorf("client files = %v, want %v", got, want) + } + if strings.Contains(e.logs.String(), "not reported") { + t.Errorf("unexpected unknown-name log: %s", e.logs) } - e.logs.Reset() +} + +func TestRunNameCollision(t *testing.T) { + awg := &fakeBackend{session: &fakeSession{names: []string{"shared", "awg0"}}} + xray := &fakeBackend{session: &fakeSession{names: []string{"shared"}}} + e := newPassEnv(t, "phone: [shared, awg0]\n", awg.backend("awg-grpc", ".conf"), xray.backend("xray", ".txt")) + e.writeClient("phone/shared.conf", "old conf") + e.writeClient("phone/shared.txt", "old txt") - e.writeConfig("phone: [awg0]\nlaptop: []\n") e.run() - if len(e.client.applied) != 1 || len(e.client.applied[0].GetPeers()) != 1 { - t.Fatalf("expected one request with one peer") + if _, ok := awg.session.applied["shared"]; ok { + t.Error("awg applied the colliding name") } - if _, ok := e.clientFiles()["laptop/awg0.conf"]; ok { - t.Errorf("laptop file stayed") + if _, ok := xray.session.applied["shared"]; ok { + t.Error("xray applied the colliding name") } - if got := e.loadState().AWG["awg0/laptop"]; got != laptop { - t.Errorf("laptop entry changed: %+v", got) + if got := awg.session.applied["awg0"]; !slices.Equal(got, []string{"phone"}) { + t.Errorf("awg0 applied = %v", got) } - if !strings.Contains(e.logs.String(), "removed=[laptop]") { - t.Errorf("log does not name the removed user: %s", e.logs) + if !strings.Contains(e.logs.String(), "awg-grpc and xray") { + t.Errorf("log does not name both backends: %s", e.logs) + } + files := e.clientFiles() + if files["phone/shared.conf"] != "old conf" || files["phone/shared.txt"] != "old txt" { + t.Errorf("colliding interface lost its files: %v", sortedKeys(files)) } } -func TestRunLogsResult(t *testing.T) { - unknown := bytes.Repeat([]byte{7}, 32) - tests := []struct { - name string - response *awgv1.ApplyPeersResponse - want []string - silent bool - }{ - {"user names", nil, []string{"added=[phone]"}, false}, - {"key without entry", &awgv1.ApplyPeersResponse{Updated: [][]byte{unknown}}, - []string{base64.StdEncoding.EncodeToString(unknown)}, false}, - {"empty result", &awgv1.ApplyPeersResponse{}, nil, true}, - } - for _, tt := range tests { - t.Run(tt.name, func(t *testing.T) { - e := newPassEnv(t, "phone: [awg0]\n", named("awg0", "10.8.1.1/24")) - e.client.apply = func(r *awgv1.ApplyPeersRequest) (*awgv1.ApplyPeersResponse, error) { - if tt.response != nil { - return tt.response, nil - } - return &awgv1.ApplyPeersResponse{Added: [][]byte{r.GetPeers()[0].GetPublicKey()}}, nil - } +func TestRunFailedInterfaceKeepsOtherBackend(t *testing.T) { + awg := &fakeBackend{session: &fakeSession{names: []string{"awg0"}, applyErr: map[string]error{"awg0": errors.New("rejected")}}} + xray := &fakeBackend{session: &fakeSession{names: []string{"vless"}}} + e := newPassEnv(t, "phone: [awg0, vless]\n", awg.backend("awg-grpc", ".conf"), xray.backend("xray", ".txt")) + e.writeClient("phone/awg0.conf", "old conf") + e.writeClient("phone/vless.txt", "old txt") - e.run() + e.run() - for _, w := range tt.want { - if !strings.Contains(e.logs.String(), w) { - t.Errorf("log lacks %q: %s", w, e.logs) - } - } - if tt.silent && strings.Contains(e.logs.String(), "peers changed") { - t.Errorf("empty result was logged: %s", e.logs) - } - }) + if got := xray.session.applied["vless"]; !slices.Equal(got, []string{"phone"}) { + t.Errorf("xray applied = %v", xray.session.applied) + } + files := e.clientFiles() + if files["phone/awg0.conf"] != "old conf" { + t.Errorf("failed awg interface lost its file: %v", files) + } + if files["phone/vless.txt"] != "new vless" { + t.Errorf("xray file = %q", files["phone/vless.txt"]) } } -func TestRunLogHasNoSecrets(t *testing.T) { - const headerKey = "HDRKEYSECRET" - const leak = "LEAKSECRET" - e := newPassEnv(t, "phone: [awg0, awg1]\n", - named("awg0", "10.8.1.1/24", &awgv1.ConfigParam{Key: "HeaderProtectionKey", Value: headerKey}), - named("awg1", "10.9.1.1/24", &awgv1.ConfigParam{Key: "S1", Value: "x\n" + leak})) - e.client.apply = func(r *awgv1.ApplyPeersRequest) (*awgv1.ApplyPeersResponse, error) { - return &awgv1.ApplyPeersResponse{Added: [][]byte{r.GetPeers()[0].GetPublicKey()}}, nil - } +func TestRunBackendDown(t *testing.T) { + awg := &fakeBackend{openErr: errors.New("no socket"), session: &fakeSession{}} + xray := &fakeBackend{session: &fakeSession{names: []string{"vless"}}} + e := newPassEnv(t, "phone: [awg0, vless]\n", awg.backend("awg-grpc", ".conf"), xray.backend("xray", ".txt")) + e.writeClient("phone/awg0.conf", "old conf") + e.writeClient("laptop/awg1.conf", "old laptop conf") + e.writeClient("phone/vless.txt", "old txt") - e.run() - e.writeConfig("phone: [awg0\n") e.run() + if got := xray.session.applied["vless"]; !slices.Equal(got, []string{"phone"}) { + t.Errorf("xray applied = %v", xray.session.applied) + } + files := e.clientFiles() + want := map[string]string{ + "phone/awg0.conf": "old conf", + "laptop/awg1.conf": "old laptop conf", + "phone/vless.txt": "new vless", + } + if !maps.Equal(files, want) { + t.Errorf("client files = %v, want %v", files, want) + } logs := e.logs.String() - if logs == "" { - t.Fatal("no log output") + if !strings.Contains(logs, "backend is down") || !strings.Contains(logs, "no socket") { + t.Errorf("log does not report the down backend: %s", logs) } - secrets := []string{headerKey, leak} - for _, entry := range e.loadState().AWG { - secrets = append(secrets, entry.PrivateKey.String(), entry.PresharedKey.String()) + if strings.Contains(logs, "not reported by any backend") { + t.Errorf("unknown-name log while a backend is down: %s", logs) } - for _, s := range secrets { - if strings.Contains(logs, s) { - t.Errorf("log contains a secret: %q", s) - } +} + +func TestRunAllBackendsDownKeepsFiles(t *testing.T) { + fb := &fakeBackend{openErr: errors.New("no socket")} + e := newPassEnv(t, "phone: [awg0]\n", fb.backend("awg-grpc", ".conf")) + e.writeClient("phone/awg0.conf", "old") + + e.run() + + if files := e.clientFiles(); !maps.Equal(files, map[string]string{"phone/awg0.conf": "old"}) { + t.Errorf("client files = %v", files) + } +} + +func TestRunLogsBackendName(t *testing.T) { + fb := &fakeBackend{session: &fakeSession{names: []string{"awg0"}, prepareErr: map[string]error{"awg0": fmt.Errorf("boom")}}} + e := newPassEnv(t, "phone: [awg0]\n", fb.backend("awg-grpc", ".conf")) + + e.run() + + if !strings.Contains(e.logs.String(), "backend=awg-grpc") { + t.Errorf("log lacks the backend name: %s", e.logs) } } diff --git a/internal/pass/settings.go b/internal/pass/settings.go index 5783e7a..e593676 100644 --- a/internal/pass/settings.go +++ b/internal/pass/settings.go @@ -1,13 +1,9 @@ -// Package pass runs one reconcile pass: enrollment, rendering and the requests to awg-grpc. +// Package pass runs one reconcile pass over the backends: enrollment, the state file, the +// requests to each backend and the clients directory. package pass -import "net/netip" - -// Settings holds the server facts and the paths one pass works with. +// Settings holds the paths one pass works with. type Settings struct { - Host netip.Addr // USHER_HOST - DNS []netip.Addr // USHER_DNS - AWGSocket string // USHER_AWG_SOCKET ConfigPath string ClientsDir string StatePath string From af549ec978558a6def0df0ee6372bec6ceebeb3f Mon Sep 17 00:00:00 2001 From: EvaTheSalmon <18466855+EvaTheSalmon@users.noreply.github.com> Date: Fri, 9 Oct 2026 17:55:16 +0300 Subject: [PATCH 04/32] refactor(usher): share Open adapter, name AWG dialer --- cmd/usher/main.go | 19 ++++++++++++------- 1 file changed, 12 insertions(+), 7 deletions(-) diff --git a/cmd/usher/main.go b/cmd/usher/main.go index 2296521..9e1639d 100644 --- a/cmd/usher/main.go +++ b/cmd/usher/main.go @@ -55,21 +55,26 @@ func serve(ctx context.Context, settings pass.Settings, backends []pass.Backend, } func awgBackend(env environment, log *slog.Logger) pass.Backend { - dial := dialSocket(env.AWGSocket) + dial := dialAWGSocket(env.AWGSocket) return pass.Backend{ Name: "awg-grpc", Suffix: awg.Suffix, Open: func(ctx context.Context) (pass.Session, error) { - session, err := awg.Open(ctx, dial, env.Host, env.DNS, log) - if err != nil { - return nil, err - } - return session, nil + return asSession(awg.Open(ctx, dial, env.Host, env.DNS, log)) }, } } -func dialSocket(path string) awg.Dial { +// asSession converts a backend session to pass.Session. On error it returns a nil interface, not +// an interface that holds a nil pointer. +func asSession[S pass.Session](session S, err error) (pass.Session, error) { + if err != nil { + return nil, err + } + return session, nil +} + +func dialAWGSocket(path string) awg.Dial { target := "unix://" + path return func() (awgv1.ManagementServiceClient, func(), error) { conn, err := grpc.NewClient(target, grpc.WithTransportCredentials(insecure.NewCredentials())) From b255b81aa56f6e76f4597ca7191096fde86e2cf8 Mon Sep 17 00:00:00 2001 From: EvaTheSalmon <18466855+EvaTheSalmon@users.noreply.github.com> Date: Fri, 9 Oct 2026 17:55:16 +0300 Subject: [PATCH 05/32] refactor(awg): unexport render --- internal/awg/render.go | 4 ++-- internal/awg/render_test.go | 4 ++-- internal/awg/session.go | 2 +- 3 files changed, 5 insertions(+), 5 deletions(-) diff --git a/internal/awg/render.go b/internal/awg/render.go index 9d202d9..f9b72e6 100644 --- a/internal/awg/render.go +++ b/internal/awg/render.go @@ -12,9 +12,9 @@ import ( "github.com/mrcsin/usher/internal/state" ) -// Render returns the client config of one user on one interface. The error never quotes a +// render returns the client config of one user on one interface. The error never quotes a // client_params value. -func Render(entry state.Entry, iface *awgv1.InterfaceStatus, host netip.Addr, dns []netip.Addr) ([]byte, error) { +func render(entry state.Entry, iface *awgv1.InterfaceStatus, host netip.Addr, dns []netip.Addr) ([]byte, error) { port := iface.GetListenPort() if port > math.MaxUint16 { return nil, fmt.Errorf("listen port %d is not a port number", port) diff --git a/internal/awg/render_test.go b/internal/awg/render_test.go index bfe8907..8e252ca 100644 --- a/internal/awg/render_test.go +++ b/internal/awg/render_test.go @@ -36,7 +36,7 @@ func testInterface() *awgv1.InterfaceStatus { } func TestRenderGolden(t *testing.T) { - got, err := Render(testEntry(), testInterface(), netip.MustParseAddr("203.0.113.10"), + got, err := render(testEntry(), testInterface(), netip.MustParseAddr("203.0.113.10"), []netip.Addr{netip.MustParseAddr("1.1.1.1"), netip.MustParseAddr("1.0.0.1")}) if err != nil { t.Fatal(err) @@ -124,7 +124,7 @@ func TestRender(t *testing.T) { if tt.mutate != nil { tt.mutate(iface) } - got, err := Render(testEntry(), iface, host, tt.dns) + got, err := render(testEntry(), iface, host, tt.dns) if tt.wantErr { if err == nil { t.Fatal("want an error") diff --git a/internal/awg/session.go b/internal/awg/session.go index 8153d24..9a28234 100644 --- a/internal/awg/session.go +++ b/internal/awg/session.go @@ -86,7 +86,7 @@ func (s *Session) Prepare(st *state.State, name string, users []string) (map[str } files := make(map[string][]byte, len(users)) for _, user := range users { - content, err := Render(st.AWG[state.EntryName(name, user)], iface, s.host, s.dns) + content, err := render(st.AWG[state.EntryName(name, user)], iface, s.host, s.dns) if err != nil { return nil, false, fmt.Errorf("user %s: %w", user, err) } From 8acbeab75ccf553f60e2cd4c8fe806674ad6ebb0 Mon Sep 17 00:00:00 2001 From: EvaTheSalmon <18466855+EvaTheSalmon@users.noreply.github.com> Date: Fri, 9 Oct 2026 17:55:16 +0300 Subject: [PATCH 06/32] refactor(pass): keep every suffix of a down backend Backend.Suffix becomes Suffixes, clients.Existing takes a list, and USHER_DNS is parsed only when USHER_AWG_SOCKET is set. Co-Authored-By: Claude Sonnet 5.5 --- cmd/usher/env.go | 32 ++++++++++---- cmd/usher/env_test.go | 5 +++ cmd/usher/main.go | 4 +- cmd/usher/main_test.go | 6 +-- docs/plans/20261014-xray-config-backend.md | 13 +++--- internal/clients/mirror.go | 23 +++++++--- internal/clients/mirror_test.go | 25 +++++++---- internal/pass/backend.go | 6 +-- internal/pass/pass.go | 2 +- internal/pass/pass_test.go | 49 +++++++++++++++++++--- 10 files changed, 122 insertions(+), 43 deletions(-) diff --git a/cmd/usher/env.go b/cmd/usher/env.go index 9b973bb..08b2de8 100644 --- a/cmd/usher/env.go +++ b/cmd/usher/env.go @@ -1,6 +1,7 @@ package main import ( + "errors" "fmt" "net/netip" "path/filepath" @@ -29,18 +30,18 @@ func environmentFrom(getenv func(string) string) (environment, error) { return environment{}, err } - awgSocket, err := parseSocketPath("USHER_AWG_SOCKET", getenv("USHER_AWG_SOCKET")) - if err != nil { - return environment{}, err - } - + var awgSocket string var dns []netip.Addr - for _, field := range strings.Split(getenv("USHER_DNS"), ",") { - addr, err := parseIPv4("USHER_DNS", strings.TrimSpace(field)) - if err != nil { + if getenv("USHER_AWG_SOCKET") != "" { + if awgSocket, err = parseSocketPath("USHER_AWG_SOCKET", getenv("USHER_AWG_SOCKET")); err != nil { + return environment{}, err + } + if dns, err = parseDNS(getenv("USHER_DNS")); err != nil { return environment{}, err } - dns = append(dns, addr) + } + if awgSocket == "" { + return environment{}, errors.New("USHER_AWG_SOCKET is required") } return environment{ @@ -51,6 +52,19 @@ func environmentFrom(getenv func(string) string) (environment, error) { }, nil } +// parseDNS parses the comma-separated USHER_DNS list that the awg backend puts into client configs. +func parseDNS(value string) ([]netip.Addr, error) { + var dns []netip.Addr + for _, field := range strings.Split(value, ",") { + addr, err := parseIPv4("USHER_DNS", strings.TrimSpace(field)) + if err != nil { + return nil, err + } + dns = append(dns, addr) + } + return dns, nil +} + func parseIPv4(name, value string) (netip.Addr, error) { if value == "" { return netip.Addr{}, fmt.Errorf("%s is required", name) diff --git a/cmd/usher/env_test.go b/cmd/usher/env_test.go index 08a38fa..718564c 100644 --- a/cmd/usher/env_test.go +++ b/cmd/usher/env_test.go @@ -62,6 +62,11 @@ func TestEnvironmentFrom(t *testing.T) { env: map[string]string{"USHER_HOST": "203.0.113.10", "USHER_AWG_SOCKET": "awg.sock", "USHER_DNS": "1.1.1.1"}, wantErr: "USHER_AWG_SOCKET", }, + { + name: "missing awg socket does not ask for DNS", + env: map[string]string{"USHER_HOST": "203.0.113.10"}, + wantErr: "USHER_AWG_SOCKET is required", + }, { name: "empty DNS", env: map[string]string{"USHER_HOST": "203.0.113.10", "USHER_AWG_SOCKET": "/run/awg.sock", "USHER_DNS": ""}, diff --git a/cmd/usher/main.go b/cmd/usher/main.go index 9e1639d..793c43a 100644 --- a/cmd/usher/main.go +++ b/cmd/usher/main.go @@ -57,8 +57,8 @@ func serve(ctx context.Context, settings pass.Settings, backends []pass.Backend, func awgBackend(env environment, log *slog.Logger) pass.Backend { dial := dialAWGSocket(env.AWGSocket) return pass.Backend{ - Name: "awg-grpc", - Suffix: awg.Suffix, + Name: "awg-grpc", + Suffixes: []string{awg.Suffix}, Open: func(ctx context.Context) (pass.Session, error) { return asSession(awg.Open(ctx, dial, env.Host, env.DNS, log)) }, diff --git a/cmd/usher/main_test.go b/cmd/usher/main_test.go index 0769574..cb8b69c 100644 --- a/cmd/usher/main_test.go +++ b/cmd/usher/main_test.go @@ -62,9 +62,9 @@ func TestServe(t *testing.T) { StatePath: filepath.Join(root, "users.json"), } backends := []pass.Backend{{ - Name: "fake", - Suffix: ".conf", - Open: func(context.Context) (pass.Session, error) { return nil, errors.New("no socket") }, + Name: "fake", + Suffixes: []string{".conf"}, + Open: func(context.Context) (pass.Session, error) { return nil, errors.New("no socket") }, }} ctx, cancel := context.WithCancel(context.Background()) defer cancel() diff --git a/docs/plans/20261014-xray-config-backend.md b/docs/plans/20261014-xray-config-backend.md index 26ed3d8..e339100 100644 --- a/docs/plans/20261014-xray-config-backend.md +++ b/docs/plans/20261014-xray-config-backend.md @@ -323,16 +323,17 @@ reported by awg-grpc" and nothing reaches Xray. `fake_test.go`, `testdata/phone.conf`); Modify: `internal/clients/mirror.go` - Modify: `deploy/compose.example.yml`, `deploy/compose.e2e.yml` -- [ ] cherry-pick `bd02bac`, `00113e4`, `f409cac`, `7100137` from `xray-driver`; drop the hunks +- [x] cherry-pick `bd02bac`, `00113e4`, `f409cac`, `7100137` from `xray-driver`; drop the hunks for `docs/plans/20261007-xray-driver.md` and for Xray files that do not exist here; reword the `bd02bac` header without `!` -- [ ] change `Backend.Suffix` to `Suffixes []string` and make `clients.Existing` keep every suffix +- [x] change `Backend.Suffix` to `Suffixes []string` and make `clients.Existing` keep every suffix of a down backend -- [ ] make `USHER_DNS` required only when `USHER_AWG_SOCKET` is set -- [ ] keep the awg behaviour of `master` unchanged -- [ ] write tests for the seam: two fake backends, a collision, a down backend with two suffixes, +- [x] make `USHER_DNS` required only when `USHER_AWG_SOCKET` is set +- [x] keep the awg behaviour of `master` unchanged +- [x] write tests for the seam: two fake backends, a collision, a down backend with two suffixes, the `USHER_DNS` rule -- [ ] run `go test -race ./...` and the awg e2e (CI) - must pass before Task 2 +- [x] run `go test -race ./...` and the awg e2e (CI) - must pass before Task 2 (awg e2e deferred + to CI, not run locally) ### Task 2: users.json version 2 with Xray credentials diff --git a/internal/clients/mirror.go b/internal/clients/mirror.go index c9308a7..846ee08 100644 --- a/internal/clients/mirror.go +++ b/internal/clients/mirror.go @@ -24,10 +24,10 @@ func Path(user, ifaceName, suffix string) string { return user + "/" + ifaceName + suffix } -// Existing returns the current content of the files in dir that end in suffix and belong to an -// interface for which keep returns true, keyed by Path. It reads as much as it can and returns +// Existing returns the current content of the files in dir that end in one of suffixes and belong +// to an interface for which keep returns true, keyed by Path. It reads as much as it can and returns // the errors of the rest joined. -func Existing(dir, suffix string, keep func(ifaceName string) bool) (map[string][]byte, error) { +func Existing(dir string, suffixes []string, keep func(ifaceName string) bool) (map[string][]byte, error) { files := make(map[string][]byte) userDirs, err := os.ReadDir(dir) if err != nil { @@ -44,8 +44,11 @@ func Existing(dir, suffix string, keep func(ifaceName string) bool) (map[string] continue } for _, f := range entries { - ifaceName, ok := strings.CutSuffix(f.Name(), suffix) - if !ok || f.IsDir() || !keep(ifaceName) { + if f.IsDir() { + continue + } + ifaceName, suffix, ok := cutSuffixes(f.Name(), suffixes) + if !ok || !keep(ifaceName) { continue } content, err := os.ReadFile(filepath.Join(dir, d.Name(), f.Name())) @@ -59,6 +62,16 @@ func Existing(dir, suffix string, keep func(ifaceName string) bool) (map[string] return files, errors.Join(errs...) } +// cutSuffixes strips the first of suffixes that name ends in and returns it. +func cutSuffixes(name string, suffixes []string) (base, suffix string, ok bool) { + for _, suffix := range suffixes { + if base, ok := strings.CutSuffix(name, suffix); ok { + return base, suffix, true + } + } + return "", "", false +} + // Mirror makes dir hold exactly the desired files. Keys of desired are slash-separated paths // relative to dir, such as "phone/awg0.conf". Files are written atomically with mode 0600 and // only when the content differs. Every other file and every empty directory is diff --git a/internal/clients/mirror_test.go b/internal/clients/mirror_test.go index 3aeac07..3af6346 100644 --- a/internal/clients/mirror_test.go +++ b/internal/clients/mirror_test.go @@ -158,7 +158,7 @@ func TestExisting(t *testing.T) { tests := []struct { name string existing map[string]string - suffix string + suffixes []string keep func(string) bool want map[string][]byte }{ @@ -167,21 +167,30 @@ func TestExisting(t *testing.T) { existing: map[string]string{ "phone/awg0.conf": "a", "phone/awg1.conf": "b", "laptop/awg1.conf": "c", "phone/notes.txt": "d", }, - suffix: ".conf", - keep: only("awg1"), - want: map[string][]byte{"phone/awg1.conf": []byte("b"), "laptop/awg1.conf": []byte("c")}, + suffixes: []string{".conf"}, + keep: only("awg1"), + want: map[string][]byte{"phone/awg1.conf": []byte("b"), "laptop/awg1.conf": []byte("c")}, }, { name: "keeping all returns every file of the suffix", existing: map[string]string{"phone/awg0.conf": "a", "phone/vless.txt": "b", "laptop/vless.txt": "c"}, - suffix: ".txt", + suffixes: []string{".txt"}, keep: all, want: map[string][]byte{"phone/vless.txt": []byte("b"), "laptop/vless.txt": []byte("c")}, }, + { + name: "several suffixes keep the files of each", + existing: map[string]string{ + "phone/hy2.json": "a", "phone/hy2.txt": "b", "phone/hy2.conf": "c", "phone/vless.txt": "d", + }, + suffixes: []string{".json", ".txt"}, + keep: only("hy2"), + want: map[string][]byte{"phone/hy2.json": []byte("a"), "phone/hy2.txt": []byte("b")}, + }, { name: "keeping none reads nothing", existing: map[string]string{"phone/awg0.conf": "a"}, - suffix: ".conf", + suffixes: []string{".conf"}, keep: none, want: map[string][]byte{}, }, @@ -192,7 +201,7 @@ func TestExisting(t *testing.T) { for rel, content := range tt.existing { writeFile(t, filepath.Join(dir, filepath.FromSlash(rel)), content, 0o600) } - got, err := Existing(dir, tt.suffix, tt.keep) + got, err := Existing(dir, tt.suffixes, tt.keep) if err != nil { t.Fatal(err) } @@ -202,7 +211,7 @@ func TestExisting(t *testing.T) { }) } t.Run("missing directory is an error", func(t *testing.T) { - if _, err := Existing(filepath.Join(t.TempDir(), "gone"), ".conf", all); err == nil { + if _, err := Existing(filepath.Join(t.TempDir(), "gone"), []string{".conf"}, all); err == nil { t.Fatal("want an error") } }) diff --git a/internal/pass/backend.go b/internal/pass/backend.go index bf89e54..483e20b 100644 --- a/internal/pass/backend.go +++ b/internal/pass/backend.go @@ -22,7 +22,7 @@ type Session interface { // Backend is a VPN server usher moves users into. type Backend struct { - Name string // used in logs, such as "awg-grpc" - Suffix string // suffix of the client files the backend renders, such as ".conf" - Open func(ctx context.Context) (Session, error) + Name string // used in logs, such as "awg-grpc" + Suffixes []string // suffixes of the client files the backend renders, such as ".conf" + Open func(ctx context.Context) (Session, error) } diff --git a/internal/pass/pass.go b/internal/pass/pass.go index f9d5731..0f33a67 100644 --- a/internal/pass/pass.go +++ b/internal/pass/pass.go @@ -144,7 +144,7 @@ func (p *Pass) openBackends(ctx context.Context) ([]openBackend, []Backend) { // keepFiles copies the current client files of the backend's interfaces that keep selects into // desired. func (p *Pass) keepFiles(desired map[string][]byte, b Backend, keep func(ifaceName string) bool) { - kept, err := clients.Existing(p.settings.ClientsDir, b.Suffix, keep) + kept, err := clients.Existing(p.settings.ClientsDir, b.Suffixes, keep) if err != nil { p.log.Error("keeping client files", "backend", b.Name, "error", err) } diff --git a/internal/pass/pass_test.go b/internal/pass/pass_test.go index ed4ec5a..47e0a09 100644 --- a/internal/pass/pass_test.go +++ b/internal/pass/pass_test.go @@ -21,8 +21,8 @@ import ( // fakeSession is a Session that renders "new " into one file per user and enrolls a // state entry the first time an interface has users. type fakeSession struct { - names []string - suffix string + names []string + suffixes []string prepareErr map[string]error applyErr map[string]error @@ -40,7 +40,9 @@ func (f *fakeSession) Prepare(st *state.State, name string, users []string) (map } files := make(map[string][]byte, len(users)) for _, user := range users { - files[user+"/"+name+f.suffix] = []byte("new " + name) + for _, suffix := range f.suffixes { + files[user+"/"+name+suffix] = []byte("new " + name) + } } entryName := state.EntryName("fake", name) _, enrolled := st.AWG[entryName] @@ -74,13 +76,13 @@ type fakeBackend struct { opens int } -func (f *fakeBackend) backend(name, suffix string) Backend { - return Backend{Name: name, Suffix: suffix, Open: func(context.Context) (Session, error) { +func (f *fakeBackend) backend(name string, suffixes ...string) Backend { + return Backend{Name: name, Suffixes: suffixes, Open: func(context.Context) (Session, error) { f.opens++ if f.openErr != nil { return nil, f.openErr } - f.session.suffix = suffix + f.session.suffixes = suffixes return f.session, nil }} } @@ -477,3 +479,38 @@ func TestRunLogsBackendName(t *testing.T) { t.Errorf("log lacks the backend name: %s", e.logs) } } + +func TestRunTwoSuffixes(t *testing.T) { + fb := &fakeBackend{session: &fakeSession{names: []string{"vless", "hy2"}, prepareErr: map[string]error{"hy2": errors.New("bad")}}} + e := newPassEnv(t, "phone: [vless, hy2]\n", fb.backend("xray", ".json", ".txt")) + e.writeClient("phone/hy2.json", "old json") + e.writeClient("phone/hy2.txt", "old txt") + e.writeClient("phone/notes.md", "foreign") + + e.run() + + want := map[string]string{ + "phone/vless.json": "new vless", + "phone/vless.txt": "new vless", + "phone/hy2.json": "old json", + "phone/hy2.txt": "old txt", + } + if files := e.clientFiles(); !maps.Equal(files, want) { + t.Errorf("client files = %v, want %v", files, want) + } +} + +func TestRunBackendDownKeepsEverySuffix(t *testing.T) { + fb := &fakeBackend{openErr: errors.New("no socket")} + e := newPassEnv(t, "phone: [vless]\n", fb.backend("xray", ".json", ".txt")) + e.writeClient("phone/vless.json", "old json") + e.writeClient("phone/vless.txt", "old txt") + e.writeClient("phone/other.conf", "old conf") + + e.run() + + want := map[string]string{"phone/vless.json": "old json", "phone/vless.txt": "old txt"} + if files := e.clientFiles(); !maps.Equal(files, want) { + t.Errorf("client files = %v, want %v", files, want) + } +} From 16fc05f0c4831ddc0902a3c098c7ad4c1217e2ea Mon Sep 17 00:00:00 2001 From: EvaTheSalmon <18466855+EvaTheSalmon@users.noreply.github.com> Date: Wed, 14 Oct 2026 17:48:24 +0300 Subject: [PATCH 07/32] feat(state): add xray credentials, users.json v2 Co-Authored-By: Claude Sonnet 5.5 --- docs/plans/20261014-xray-config-backend.md | 10 +-- go.mod | 1 + go.sum | 2 + internal/config/config.go | 2 +- internal/config/config_test.go | 6 +- internal/state/state.go | 94 ++++++++++++++++--- internal/state/state_test.go | 100 +++++++++++++++++++-- 7 files changed, 187 insertions(+), 28 deletions(-) diff --git a/docs/plans/20261014-xray-config-backend.md b/docs/plans/20261014-xray-config-backend.md index e339100..4c1ed22 100644 --- a/docs/plans/20261014-xray-config-backend.md +++ b/docs/plans/20261014-xray-config-backend.md @@ -341,14 +341,14 @@ reported by awg-grpc" and nothing reaches Xray. - Modify: `internal/state/state.go`, `internal/state/state_test.go` - Modify: `internal/config/config.go`, `internal/config/config_test.go` -- [ ] start from `64dd6b0` and `69a3ba5`: `Xray map[string]XrayEntry`, `XrayEntry{ID uuid.UUID; +- [x] start from `64dd6b0` and `69a3ba5`: `Xray map[string]XrayEntry`, `XrayEntry{ID uuid.UUID; Password string}` with exactly one set; validate names `tag/user`; errors never quote a secret -- [ ] version 2; `Load` accepts 1 and 2; before the first save of a version 1 file write +- [x] version 2; `Load` accepts 1 and 2; before the first save of a version 1 file write `users.json.v1` -- [ ] widen the interface name pattern to 32 characters -- [ ] write tests: v1 load and backup, v2 round trip, both credential kinds, invalid entries, no +- [x] widen the interface name pattern to 32 characters +- [x] write tests: v1 load and backup, v2 round trip, both credential kinds, invalid entries, no secret in errors, a 32-character tag -- [ ] run `go test -race ./...` - must pass before Task 3 +- [x] run `go test -race ./...` - must pass before Task 3 ### Task 3: Read the operator's Xray config diff --git a/go.mod b/go.mod index 1ba21b3..55b666a 100644 --- a/go.mod +++ b/go.mod @@ -3,6 +3,7 @@ module github.com/mrcsin/usher go 1.27.1 require ( + github.com/google/uuid v1.6.0 github.com/mrcsin/awg-grpc v0.1.0 google.golang.org/grpc v1.84.0 gopkg.in/yaml.v3 v3.0.1 diff --git a/go.sum b/go.sum index c557a28..999caae 100644 --- a/go.sum +++ b/go.sum @@ -2,6 +2,8 @@ github.com/golang/protobuf v1.5.4 h1:i7eJL8qZTpSEXOPTxNKhASYpMn+8e5Q6AdndVa1dWek github.com/golang/protobuf v1.5.4/go.mod h1:lnTiLA8Wa4RWRcIUkrtSVa5nRhsEGBg48fD6rSs7xps= github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8= github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU= +github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0= +github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo= github.com/mrcsin/awg-grpc v0.1.0 h1:hJHs0LijjnuaBuAAYgYO4bC3es9oWeXgIRwIDUwx7dE= github.com/mrcsin/awg-grpc v0.1.0/go.mod h1:QOJcKZz48hMjtVL5xG5AwLb9DHjPQQVvtasFpbpJLOw= golang.org/x/net v0.57.0 h1:K5+3DljvIuDG9/Jv9rvyMywYNFCQ9RSUY6OOTTkT+tE= diff --git a/internal/config/config.go b/internal/config/config.go index 33ce1cc..0de85f8 100644 --- a/internal/config/config.go +++ b/internal/config/config.go @@ -11,7 +11,7 @@ import ( var ( userNamePattern = regexp.MustCompile(`^[a-z0-9][a-z0-9_-]{0,31}$`) - interfaceNamePattern = regexp.MustCompile(`^[A-Za-z0-9_][A-Za-z0-9_=+.-]{0,14}$`) + interfaceNamePattern = regexp.MustCompile(`^[A-Za-z0-9_][A-Za-z0-9_=+.-]{0,31}$`) ) // Load reads path and returns the interface names of every user. A user with an empty list is diff --git a/internal/config/config_test.go b/internal/config/config_test.go index e20ec9c..7b9410c 100644 --- a/internal/config/config_test.go +++ b/internal/config/config_test.go @@ -33,8 +33,8 @@ func TestLoad(t *testing.T) { }, { name: "longest interface name", - content: "alice: [" + strings.Repeat("a", 15) + "]\n", - want: map[string][]string{"alice": {strings.Repeat("a", 15)}}, + content: "alice: [" + strings.Repeat("a", 32) + "]\n", + want: map[string][]string{"alice": {strings.Repeat("a", 32)}}, }, { name: "block list", @@ -73,7 +73,7 @@ func TestLoad(t *testing.T) { }, { name: "interface name too long", - content: "alice: [abcdefghijklmnop]\n", + content: "alice: [" + strings.Repeat("a", 33) + "]\n", errHas: []string{"line 1", "alice"}, }, { diff --git a/internal/state/state.go b/internal/state/state.go index cef19d3..c96e997 100644 --- a/internal/state/state.go +++ b/internal/state/state.go @@ -13,10 +13,17 @@ import ( "os" "strings" + "github.com/google/uuid" "github.com/mrcsin/usher/internal/atomicfile" ) -const version = 1 +// version is the format that Save writes. Load also accepts firstVersion, which has no xray group. +const version = 2 + +const firstVersion = 1 + +// backupSuffix names the copy of a version 1 file that Save makes before it writes version 2. +const backupSuffix = ".v1" const fileMode = 0o600 @@ -68,15 +75,25 @@ func (e Entry) Route() netip.Prefix { return netip.PrefixFrom(e.Address, hostBits) } -// EntryName returns the key of a user's entry on an interface in State.AWG. +// XrayEntry holds the credential of one user on one Xray inbound. Exactly one field is set: ID +// for VLESS and VMess, Password for Trojan and Hysteria2. +type XrayEntry struct { + ID uuid.UUID `json:"id,omitzero"` + Password string `json:"password,omitempty"` +} + +// EntryName returns the key of a user's entry on an interface or inbound, in State.AWG or +// State.Xray. func EntryName(iface, user string) string { return iface + "/" + user } -// State is the content of users.json. AWG maps "interface/user" to its entry. +// State is the content of users.json. AWG and Xray map "interface/user" and "tag/user" to the +// entry of that user. type State struct { - Version int `json:"version"` - AWG map[string]Entry `json:"awg"` + Version int `json:"version"` + AWG map[string]Entry `json:"awg"` + Xray map[string]XrayEntry `json:"xray"` } // InterfaceEntries yields the user name and the entry of every entry on the interface. @@ -95,7 +112,7 @@ func (s *State) InterfaceEntries(iface string) iter.Seq2[string, Entry] { func Load(path string) (*State, error) { data, err := os.ReadFile(path) if errors.Is(err, fs.ErrNotExist) { - return &State{Version: version, AWG: map[string]Entry{}}, nil + return &State{Version: version, AWG: map[string]Entry{}, Xray: map[string]XrayEntry{}}, nil } if err != nil { return nil, fmt.Errorf("reading %s: %w", path, err) @@ -104,22 +121,34 @@ func Load(path string) (*State, error) { if err := json.Unmarshal(data, &s); err != nil { return nil, fmt.Errorf("parsing %s: %w", path, err) } - if s.Version != version { + if s.Version != version && s.Version != firstVersion { return nil, fmt.Errorf("%s: unsupported version %d", path, s.Version) } if s.AWG == nil { s.AWG = map[string]Entry{} } + if s.Xray == nil { + s.Xray = map[string]XrayEntry{} + } for name, e := range s.AWG { if err := validate(name, e); err != nil { return nil, fmt.Errorf("%s: %w", path, err) } } + for name, e := range s.Xray { + if err := validateXray(name, e); err != nil { + return nil, fmt.Errorf("%s: %w", path, err) + } + } return &s, nil } -// Save writes s to path atomically with mode 0600. +// Save writes s to path atomically with mode 0600. When path holds a version 1 file, Save first +// copies it to path+".v1" unless that copy exists, so a rollback is a file rename. func Save(path string, s *State) error { + if err := backupFirstVersion(path); err != nil { + return err + } s.Version = version data, err := json.MarshalIndent(s, "", " ") if err != nil { @@ -130,10 +159,33 @@ func Save(path string, s *State) error { return atomicfile.Write(path, data, fileMode) } +func backupFirstVersion(path string) error { + data, err := os.ReadFile(path) + if errors.Is(err, fs.ErrNotExist) { + return nil + } + if err != nil { + return fmt.Errorf("reading %s: %w", path, err) + } + var head struct { + Version int `json:"version"` + } + if json.Unmarshal(data, &head) != nil || head.Version != firstVersion { + return nil + } + backup := path + backupSuffix + if _, err := os.Stat(backup); err == nil { + return nil + } + if err := atomicfile.Write(backup, data, fileMode); err != nil { + return fmt.Errorf("backing up %s: %w", path, err) + } + return nil +} + func validate(name string, e Entry) error { - iface, user, ok := strings.Cut(name, "/") - if !ok || iface == "" || user == "" || strings.Contains(user, "/") { - return fmt.Errorf("entry %q: name is not interface/user", name) + if err := validateName(name); err != nil { + return err } if !e.Address.Is4() { return fmt.Errorf("entry %q: address is not IPv4", name) @@ -146,3 +198,23 @@ func validate(name string, e Entry) error { } return nil } + +func validateXray(name string, e XrayEntry) error { + if err := validateName(name); err != nil { + return err + } + hasID := e.ID != uuid.Nil + hasPassword := e.Password != "" + if hasID == hasPassword { + return fmt.Errorf("entry %q: exactly one of id and password must be set", name) + } + return nil +} + +func validateName(name string) error { + iface, user, ok := strings.Cut(name, "/") + if !ok || iface == "" || user == "" || strings.Contains(user, "/") { + return fmt.Errorf("entry %q: name is not interface/user", name) + } + return nil +} diff --git a/internal/state/state_test.go b/internal/state/state_test.go index d56e6b0..77ac0b3 100644 --- a/internal/state/state_test.go +++ b/internal/state/state_test.go @@ -11,8 +11,12 @@ import ( "slices" "strings" "testing" + + "github.com/google/uuid" ) +var testUUID = uuid.MustParse("11111111-2222-3333-4444-555555555555") + func keyOf(b byte) Key { return Key(bytes.Repeat([]byte{b}, KeyLength)) } @@ -23,11 +27,15 @@ func testKey(b byte) string { func validState() *State { return &State{ - Version: 1, + Version: 2, AWG: map[string]Entry{ "awg0/phone": {Address: netip.MustParseAddr("10.8.1.2"), PrivateKey: keyOf(1), PresharedKey: keyOf(2)}, "awg1/phone": {Address: netip.MustParseAddr("10.9.1.2"), PrivateKey: keyOf(3), PresharedKey: keyOf(4)}, }, + Xray: map[string]XrayEntry{ + "reality/phone": {ID: testUUID}, + "hy2/phone": {Password: "hunter2hunter2"}, + }, } } @@ -36,8 +44,8 @@ func TestLoadMissingFileIsEmpty(t *testing.T) { if err != nil { t.Fatalf("Load: %v", err) } - if s.Version != 1 || len(s.AWG) != 0 { - t.Fatalf("want empty version 1 state, got %+v", s) + if s.Version != 2 || len(s.AWG) != 0 || len(s.Xray) != 0 { + t.Fatalf("want empty version 2 state, got %+v", s) } } @@ -59,9 +67,15 @@ func TestSaveLoadRoundTrip(t *testing.T) { // TestSaveFormat pins the users.json format: addresses are plain strings and keys are base64. func TestSaveFormat(t *testing.T) { path := filepath.Join(t.TempDir(), "users.json") - s := &State{AWG: map[string]Entry{ - "awg0/phone": {Address: netip.MustParseAddr("10.8.1.2"), PrivateKey: keyOf(1), PresharedKey: keyOf(2)}, - }} + s := &State{ + AWG: map[string]Entry{ + "awg0/phone": {Address: netip.MustParseAddr("10.8.1.2"), PrivateKey: keyOf(1), PresharedKey: keyOf(2)}, + }, + Xray: map[string]XrayEntry{ + "hy2/phone": {Password: "pw"}, + "reality/phone": {ID: testUUID}, + }, + } if err := Save(path, s); err != nil { t.Fatalf("Save: %v", err) } @@ -70,13 +84,21 @@ func TestSaveFormat(t *testing.T) { t.Fatal(err) } want := `{ - "version": 1, + "version": 2, "awg": { "awg0/phone": { "address": "10.8.1.2", "private_key": "` + testKey(1) + `", "preshared_key": "` + testKey(2) + `" } + }, + "xray": { + "hy2/phone": { + "password": "pw" + }, + "reality/phone": { + "id": "11111111-2222-3333-4444-555555555555" + } } } ` @@ -143,7 +165,7 @@ func TestLoadRejects(t *testing.T) { name string json string }{ - {"bad version", `{"version":2,"awg":{}}`}, + {"bad version", `{"version":3,"awg":{}}`}, {"missing version", `{"awg":{}}`}, {"private key not base64", `{"version":1,"awg":{"awg0/a":{"address":"10.8.1.2","private_key":"` + secret + `","preshared_key":"` + testKey(2) + `"}}}`}, {"private key wrong length", `{"version":1,"awg":{"awg0/a":{"address":"10.8.1.2","private_key":"` + encodedSecret + `","preshared_key":"` + testKey(2) + `"}}}`}, @@ -158,6 +180,11 @@ func TestLoadRejects(t *testing.T) { {"private key missing", `{"version":1,"awg":{"awg0/a":{"address":"10.8.1.2","preshared_key":"` + testKey(2) + `"}}}`}, {"preshared key missing", `{"version":1,"awg":{"awg0/a":{"address":"10.8.1.2","private_key":"` + testKey(1) + `"}}}`}, {"address missing", `{"version":1,"awg":{"awg0/a":{"private_key":"` + testKey(1) + `","preshared_key":"` + testKey(2) + `"}}}`}, + {"xray name without slash", `{"version":2,"xray":{"reality":{"id":"` + testUUID.String() + `"}}}`}, + {"xray id and password", `{"version":2,"xray":{"reality/a":{"id":"` + testUUID.String() + `","password":"` + secret + `"}}}`}, + {"xray neither", `{"version":2,"xray":{"reality/a":{}}}`}, + {"xray nil id", `{"version":2,"xray":{"reality/a":{"id":"00000000-0000-0000-0000-000000000000"}}}`}, + {"xray bad id", `{"version":2,"xray":{"reality/a":{"id":"` + secret + `"}}}`}, {"not json", `version: 1`}, } for _, tt := range tests { @@ -182,3 +209,60 @@ func TestLoadRejects(t *testing.T) { }) } } + +func TestLoadVersion1(t *testing.T) { + path := filepath.Join(t.TempDir(), "users.json") + v1 := `{"version":1,"awg":{"awg0/a":{"address":"10.8.1.2","private_key":"` + testKey(1) + + `","preshared_key":"` + testKey(2) + `"}}}` + if err := os.WriteFile(path, []byte(v1), 0o600); err != nil { + t.Fatal(err) + } + s, err := Load(path) + if err != nil { + t.Fatalf("Load: %v", err) + } + if len(s.AWG) != 1 || s.Xray == nil || len(s.Xray) != 0 { + t.Fatalf("v1 state = %+v, want one awg entry and an empty xray map", s) + } + if got, _ := os.ReadFile(path); string(got) != v1 { + t.Fatalf("Load changed the file: %q", got) + } + if _, err := os.Stat(path + ".v1"); err == nil { + t.Fatal("Load wrote a backup") + } + + if err := Save(path, s); err != nil { + t.Fatalf("Save: %v", err) + } + if got, err := os.ReadFile(path + ".v1"); err != nil || string(got) != v1 { + t.Fatalf("backup = %q, %v, want the version 1 file", got, err) + } + info, err := os.Stat(path + ".v1") + if err != nil || info.Mode().Perm() != 0o600 { + t.Fatalf("backup mode = %v, %v, want 0600", info, err) + } + saved, err := Load(path) + if err != nil || saved.Version != 2 { + t.Fatalf("saved = %+v, %v, want version 2", saved, err) + } + + // A second save must not overwrite the backup with version 2 content. + if err := Save(path, saved); err != nil { + t.Fatalf("second Save: %v", err) + } + if got, _ := os.ReadFile(path + ".v1"); string(got) != v1 { + t.Fatalf("backup changed on second save: %q", got) + } +} + +func TestSaveNoBackupForVersion2(t *testing.T) { + path := filepath.Join(t.TempDir(), "users.json") + for range 2 { + if err := Save(path, validState()); err != nil { + t.Fatalf("Save: %v", err) + } + } + if _, err := os.Stat(path + ".v1"); err == nil { + t.Fatal("Save made a backup of a version 2 file") + } +} From b3de8b8d6dc2a18eadf591e8c8020b48fab36665 Mon Sep 17 00:00:00 2001 From: EvaTheSalmon <18466855+EvaTheSalmon@users.noreply.github.com> Date: Wed, 14 Oct 2026 17:55:36 +0300 Subject: [PATCH 08/32] feat(xray): read the operator's Xray server config Co-Authored-By: Claude Sonnet 5.5 --- docs/plans/20261014-xray-config-backend.md | 12 +- go.mod | 50 +- go.sum | 134 ++++- internal/config/config.go | 5 + internal/xray/config.go | 569 +++++++++++++++++++++ internal/xray/config_test.go | 410 +++++++++++++++ internal/xray/testdata/aliases.json | 28 + internal/xray/testdata/server.json | 79 +++ 8 files changed, 1268 insertions(+), 19 deletions(-) create mode 100644 internal/xray/config.go create mode 100644 internal/xray/config_test.go create mode 100644 internal/xray/testdata/aliases.json create mode 100644 internal/xray/testdata/server.json diff --git a/docs/plans/20261014-xray-config-backend.md b/docs/plans/20261014-xray-config-backend.md index 4c1ed22..74a4d0b 100644 --- a/docs/plans/20261014-xray-config-backend.md +++ b/docs/plans/20261014-xray-config-backend.md @@ -357,20 +357,20 @@ reported by awg-grpc" and nothing reaches Xray. - Create: `internal/xray/config.go`, `internal/xray/config_test.go` - Create: `internal/xray/testdata/` (server configs) -- [ ] require `github.com/xtls/libxray v1.261010.0` and the `xray-core` commit it requires +- [x] require `github.com/xtls/libxray v1.261010.0` and the `xray-core` commit it requires (`701af60772cd`, release 26.10.10); record the stripped binary size and the cold CI `go build` time in the progress log -- [ ] read the file with Xray's comment-aware reader, decode inbounds into `infra/conf` types plus +- [x] read the file with Xray's comment-aware reader, decode inbounds into `infra/conf` types plus raw maps, normalize the aliases; register the discard log handler -- [ ] keep inbounds whose protocol has an entry (`vless`, `hysteria`, `vmess`, `trojan`); skip +- [x] keep inbounds whose protocol has an entry (`vless`, `hysteria`, `vmess`, `trojan`); skip others and untagged ones -- [ ] validate per inbound: single integer port, no unix listen, empty `clients`/`users`, VLESS +- [x] validate per inbound: single integer port, no unix listen, empty `clients`/`users`, VLESS `decryption` none, empty server Hysteria auth, masks only salamander without `packetSize` (plus `quicParams`), no xhttp `downloadSettings`, a usable Reality `serverNames` entry, an SNI for TLS, the `x-usher` schema (`trust` values, `pin_sha256` only with `pin`, a pin source for `pin`, the fingerprint allowlist on Reality); each failure names the field -- [ ] write tests for every accepted shape, every failure, every alias and a file with comments -- [ ] run `go test -race ./...` and `docker build .` (CI) - must pass before Task 4 +- [x] write tests for every accepted shape, every failure, every alias and a file with comments +- [x] run `go test -race ./...` and `docker build .` (CI) - must pass before Task 4 (docker build run on `ger`) ### Task 4: Drift guard against the running Xray diff --git a/go.mod b/go.mod index 55b666a..efa0bb6 100644 --- a/go.mod +++ b/go.mod @@ -5,14 +5,56 @@ go 1.27.1 require ( github.com/google/uuid v1.6.0 github.com/mrcsin/awg-grpc v0.1.0 + github.com/xtls/xray-core v1.260327.1-0.20261010092107-701af60772cd google.golang.org/grpc v1.84.0 gopkg.in/yaml.v3 v3.0.1 ) require ( - golang.org/x/net v0.57.0 // indirect - golang.org/x/sys v0.47.0 // indirect - golang.org/x/text v0.40.0 // indirect - google.golang.org/genproto/googleapis/rpc v0.0.0-20260706201446-f0a921348800 // indirect + github.com/andybalholm/brotli v1.2.6 // indirect + github.com/apernet/quic-go v0.61.1-0.20260806010916-184d081eef3e // indirect + github.com/cloudflare/circl v1.6.5 // indirect + github.com/google/btree v1.1.3 // indirect + github.com/gorilla/websocket v1.5.3 // indirect + github.com/huin/goupnp v1.3.0 // indirect + github.com/jackpal/go-nat-pmp v1.1.0 // indirect + github.com/juju/ratelimit v1.0.2 // indirect + github.com/klauspost/compress v1.20.1 // indirect + github.com/klauspost/cpuid/v2 v2.4.0 // indirect + github.com/koron/go-ssdp v0.9.1 // indirect + github.com/kr/text v0.2.0 // indirect + github.com/libp2p/go-nat v1.0.1-0.20250821073202-01afc089f138 // indirect + github.com/libp2p/go-netroute v0.4.0 // indirect + github.com/miekg/dns v1.1.73 // indirect + github.com/pion/dtls/v3 v3.1.10 // indirect + github.com/pion/logging v0.2.4 // indirect + github.com/pion/stun/v3 v3.1.7 // indirect + github.com/pion/transport/v4 v4.1.1 // indirect + github.com/pion/transport/v5 v5.1.1 // indirect + github.com/pires/go-proxyproto v0.15.0 // indirect + github.com/quic-go/qpack v0.6.0 // indirect + github.com/refraction-networking/utls v1.8.3-0.20260301010127-aa6edf4b11af // indirect + github.com/robfig/cron/v3 v3.0.1 // indirect + github.com/vishvananda/netlink v1.3.1 // indirect + github.com/vishvananda/netns v0.0.5 // indirect + github.com/wlynxg/anet v0.0.5 // indirect + github.com/xtls/reality v0.0.0-20260921001439-3c98159dee38 // indirect + github.com/yuin/gopher-lua v1.1.2 // indirect + go4.org/netipx v0.0.0-20260823151212-3075585bcbeb // indirect + golang.org/x/crypto v0.58.0 // indirect + golang.org/x/crypto/x509roots/fallback v0.0.0-20261009200856-99e4382b128e // indirect + golang.org/x/exp v0.0.0-20261009195045-ca0d7ba23607 // indirect + golang.org/x/net v0.61.0 // indirect + golang.org/x/sync v0.24.0 // indirect + golang.org/x/sys v0.49.0 // indirect + golang.org/x/text v0.43.0 // indirect + golang.org/x/time v0.16.0 // indirect + golang.zx2c4.com/wintun v0.0.0-20230126152724-0fa3db229ce2 // indirect + golang.zx2c4.com/wireguard v0.0.0-20261006164505-2631ce99a06f // indirect + golang.zx2c4.com/wireguard/windows v1.1.1 // indirect + google.golang.org/genproto/googleapis/rpc v0.0.0-20261005182115-fad411399dd8 // indirect google.golang.org/protobuf v1.36.12 // indirect + gvisor.dev/gvisor v0.0.0-20261010064756-cd110d9dec04 // indirect + layeh.com/gopher-luar v1.0.11 // indirect + lukechampine.com/blake3 v1.4.1 // indirect ) diff --git a/go.sum b/go.sum index 999caae..2063da2 100644 --- a/go.sum +++ b/go.sum @@ -1,26 +1,142 @@ +github.com/andybalholm/brotli v1.2.6 h1:ftYnfj6usCp+UGV5kSJ3+chpMQgU+gJf/AxsUQ52REI= +github.com/andybalholm/brotli v1.2.6/go.mod h1:rzTDkvFWvIrjDXZHkuS16NPggd91W3kUSvPlQ1pLaKY= +github.com/apernet/quic-go v0.61.1-0.20260806010916-184d081eef3e h1:5mgtR5gwIgBKMiGI1QdXldZZ+SNor06Nbu1wCBulQBg= +github.com/apernet/quic-go v0.61.1-0.20260806010916-184d081eef3e/go.mod h1:x7qxEvX6MCVtDuBKHj3E+88+BtrbEMuAL5qGUKItjW8= +github.com/chzyer/logex v1.1.10/go.mod h1:+Ywpsq7O8HXn0nuIou7OrIPyXbp3wmkHB+jjWRnGsAI= +github.com/chzyer/readline v0.0.0-20180603132655-2972be24d48e/go.mod h1:nSuG5e5PlCu98SY8svDHJxuZscDgtXS6KTTbou5AhLI= +github.com/chzyer/test v0.0.0-20180213035817-a1ea475d72b1/go.mod h1:Q3SI9o4m/ZMnBNeIyt5eFwwo7qiLfzFZmjNmxjkiQlU= +github.com/cloudflare/circl v1.6.5 h1:O64F26HEqNhznd/hrC5KZXVKYuKM2rx4deZDTc4ihQA= +github.com/cloudflare/circl v1.6.5/go.mod h1:h5LNyxAc5nTue9DS5jT+48en2PSDYt3zdGnz5OstK6c= +github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E= +github.com/ghodss/yaml v1.0.1-0.20220118164431-d8423dcdf344 h1:Arcl6UOIS/kgO2nW3A65HN+7CMjSDP/gofXL4CZt1V4= +github.com/ghodss/yaml v1.0.1-0.20220118164431-d8423dcdf344/go.mod h1:GIjDIg/heH5DOkXY3YJ/wNhfHsQHoXGjl8G8amsYQ1I= +github.com/golang/mock v1.7.0-rc.1 h1:YojYx61/OLFsiv6Rw1Z96LpldJIy31o+UHmwAUMJ6/U= +github.com/golang/mock v1.7.0-rc.1/go.mod h1:s42URUywIqd+OcERslBJvOjepvNymP31m3q8d/GkuRs= github.com/golang/protobuf v1.5.4 h1:i7eJL8qZTpSEXOPTxNKhASYpMn+8e5Q6AdndVa1dWek= github.com/golang/protobuf v1.5.4/go.mod h1:lnTiLA8Wa4RWRcIUkrtSVa5nRhsEGBg48fD6rSs7xps= +github.com/google/btree v1.1.3 h1:CVpQJjYgC4VbzxeGVHfvZrv1ctoYCAI8vbl07Fcxlyg= +github.com/google/btree v1.1.3/go.mod h1:qOPhT0dTNdNzV6Z/lhRX0YXUafgPLFUh+gZMl761Gm4= github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8= github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU= github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0= github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo= +github.com/gorilla/websocket v1.5.3 h1:saDtZ6Pbx/0u+bgYQ3q96pZgCzfhKXGPqt7kZ72aNNg= +github.com/gorilla/websocket v1.5.3/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE= +github.com/huin/goupnp v1.3.0 h1:UvLUlWDNpoUdYzb2TCn+MuTWtcjXKSza2n6CBdQ0xXc= +github.com/huin/goupnp v1.3.0/go.mod h1:gnGPsThkYa7bFi/KWmEysQRf48l2dvR5bxr2OFckNX8= +github.com/jackpal/go-nat-pmp v1.1.0 h1:UInMLPV1VQdP860ggNiz0YxGvJH/bWzxL099y+1EdCs= +github.com/jackpal/go-nat-pmp v1.1.0/go.mod h1:m9o4DK1wHA4h2pPpErD5vwzWLf91tJcfNQ3QyUIbh5A= +github.com/juju/ratelimit v1.0.2 h1:sRxmtRiajbvrcLQT7S+JbqU0ntsb9W2yhSdNN8tWfaI= +github.com/juju/ratelimit v1.0.2/go.mod h1:qapgC/Gy+xNh9UxzV13HGGl/6UXNN+ct+vwSgWNm/qk= +github.com/klauspost/compress v1.20.1 h1:T7kKElXUMXrUJ2E9QhQhxFtcK5rPyLdsGZvdbLMPdiQ= +github.com/klauspost/compress v1.20.1/go.mod h1:LUdAzn7YLVvxLpc7y3V1m40wESHTgc1422pwwBSKYuI= +github.com/klauspost/cpuid/v2 v2.4.0 h1:S6Hrbc7+ywsr0r+RLapfGBHfyefhCTwEh3A0tV913Dw= +github.com/klauspost/cpuid/v2 v2.4.0/go.mod h1:19jmZ9mjzoF//ddRSUsv0zfBTJWh3QJh9FNxZTMrGxU= +github.com/koron/go-ssdp v0.9.1 h1:zvxbAAuJftJIZ8Jh8mda+LI7V92hYZf/sKprmOxpxwA= +github.com/koron/go-ssdp v0.9.1/go.mod h1:C43c047jWkDaeg9YuZlSh/QGqOieuWV6dbhWi/jcaLk= +github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE= +github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk= +github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY= +github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE= +github.com/libp2p/go-nat v1.0.1-0.20250821073202-01afc089f138 h1:YohuNPT/1k3VcThCQlBZ43PCPWPfMRS1zcxWBF2SLK8= +github.com/libp2p/go-nat v1.0.1-0.20250821073202-01afc089f138/go.mod h1:TXQg5tfSy+bUjnhT5728j5j/MBj7keIYqqZ1+8k/ui8= +github.com/libp2p/go-netroute v0.4.0 h1:sZZx9hyANYUx9PZyqcgE/E1GUG3iEtTZHUEvdtXT7/Q= +github.com/libp2p/go-netroute v0.4.0/go.mod h1:Nkd5ShYgSMS5MUKy/MU2T57xFoOKvvLR92Lic48LEyA= +github.com/miekg/dns v1.1.73 h1:uhT8nJxmTrPJYClxVxTCX+CVn6qnzSiybRk72Z6DgrE= +github.com/miekg/dns v1.1.73/go.mod h1:RW2Obtfd5NZHvOFe3zYG0W8koWOQtAzyHaLo8vASBuQ= github.com/mrcsin/awg-grpc v0.1.0 h1:hJHs0LijjnuaBuAAYgYO4bC3es9oWeXgIRwIDUwx7dE= github.com/mrcsin/awg-grpc v0.1.0/go.mod h1:QOJcKZz48hMjtVL5xG5AwLb9DHjPQQVvtasFpbpJLOw= -golang.org/x/net v0.57.0 h1:K5+3DljvIuDG9/Jv9rvyMywYNFCQ9RSUY6OOTTkT+tE= -golang.org/x/net v0.57.0/go.mod h1:KpXc8iv+r3XplLAG/f7Jsf9RPszJzdR0f58q9vGOuEU= -golang.org/x/sys v0.47.0 h1:o7XGOvZQCADBQQ4Y7VNq2dRWQR7JmOUW8Kxx4ZsNgWs= -golang.org/x/sys v0.47.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw= -golang.org/x/text v0.40.0 h1:Ub2Z6/xjgF1WrYQz2nuITOEegKFtiIy+rieRJ5lHZKs= -golang.org/x/text v0.40.0/go.mod h1:hpnzDAfGV753zIKo+wk3u1bVKCGPbrnF7+7LBF/UHVY= +github.com/pelletier/go-toml v1.9.5 h1:4yBQzkHv+7BHq2PQUZF3Mx0IYxG7LsP222s7Agd3ve8= +github.com/pelletier/go-toml v1.9.5/go.mod h1:u1nR/EPcESfeI/szUZKdtJ0xRNbUoANCkoOuaOx1Y+c= +github.com/pion/dtls/v3 v3.1.10 h1:HWC+QCZitP/ApADS/6+g7UIw2YmLgoK3CsynnjPJgMo= +github.com/pion/dtls/v3 v3.1.10/go.mod h1:iKFQNYrjsN2TiA2YKKMqB9MOZaFpjFULBI/A4sW0eyc= +github.com/pion/logging v0.2.4 h1:tTew+7cmQ+Mc1pTBLKH2puKsOvhm32dROumOZ655zB8= +github.com/pion/logging v0.2.4/go.mod h1:DffhXTKYdNZU+KtJ5pyQDjvOAh/GsNSyv1lbkFbe3so= +github.com/pion/stun/v3 v3.1.7 h1:uRXMTlGLf89WgItGNyZ6aR5jMTX0NBbybXADpQCzn+E= +github.com/pion/stun/v3 v3.1.7/go.mod h1:Nq77RW4aRrSNrltf2ksUJLjxWeipj4lnlgdsYIxC8g8= +github.com/pion/transport/v4 v4.1.1 h1:kEBtbp1sRad0tnkREDNJ3fmop/CR8p/YxXnpFbn9tqo= +github.com/pion/transport/v4 v4.1.1/go.mod h1:Oxo5oODQvjFF7NkHBBz3f4fZXLB7x03uLVuzfYeA5pE= +github.com/pion/transport/v5 v5.1.1 h1:Sauq8YJqRopgRfIgnpfKpdpKG14g0N4jWlRgpWsEfN8= +github.com/pion/transport/v5 v5.1.1/go.mod h1:Qxw6fCEjFWQkRDZOhS4Vf+neJBcihauvA3uyEa1J1F0= +github.com/pires/go-proxyproto v0.15.0 h1:dTshmNbFm/D+0+sbrxUuddPOZ5Y0B7c5NhtsBkm6LqI= +github.com/pires/go-proxyproto v0.15.0/go.mod h1:OXsCrKwrK2tXS9YrI5tkHx5xaQlO8FH3lFW76orFh24= +github.com/quic-go/go-ossfuzz-seeds v0.1.0 h1:APacT+iIaNF6fd8AGEiN3bT/Jtkd2jz4v4TzM7MFjy0= +github.com/quic-go/go-ossfuzz-seeds v0.1.0/go.mod h1:3IOHRbJIc+L6YKMwfDtJAM9Vj9k0YY4muhuyUYk5tbk= +github.com/quic-go/qpack v0.6.0 h1:g7W+BMYynC1LbYLSqRt8PBg5Tgwxn214ZZR34VIOjz8= +github.com/quic-go/qpack v0.6.0/go.mod h1:lUpLKChi8njB4ty2bFLX2x4gzDqXwUpaO1DP9qMDZII= +github.com/refraction-networking/utls v1.8.3-0.20260301010127-aa6edf4b11af h1:er2acxbi3N1nvEq6HXHUAR1nTWEJmQfqiGR8EVT9rfs= +github.com/refraction-networking/utls v1.8.3-0.20260301010127-aa6edf4b11af/go.mod h1:jkSOEkLqn+S/jtpEHPOsVv/4V4EVnelwbMQl4vCWXAM= +github.com/robfig/cron/v3 v3.0.1 h1:WdRxkvbJztn8LMz/QEvLN5sBU+xKpSqwwUO1Pjr4qDs= +github.com/robfig/cron/v3 v3.0.1/go.mod h1:eQICP3HwyT7UooqI/z+Ov+PtYAWygg1TEWWzGIFLtro= +github.com/rogpeppe/go-internal v1.10.0 h1:TMyTOH3F/DB16zRVcYyreMH6GnZZrwQVAoYjRBZyWFQ= +github.com/rogpeppe/go-internal v1.10.0/go.mod h1:UQnix2H7Ngw/k4C5ijL5+65zddjncjaFoBhdsK/akog= +github.com/stretchr/testify v1.12.1 h1:EuwCh5fleGS7H32xRwO3wRGT7DxrDhLAT6FF8MpWDWE= +github.com/stretchr/testify v1.12.1/go.mod h1:MDEgiDPPsNp5cuIrHPPCyornHKgEVbtFUmoNlxoYthg= +github.com/vishvananda/netlink v1.3.1 h1:3AEMt62VKqz90r0tmNhog0r/PpWKmrEShJU0wJW6bV0= +github.com/vishvananda/netlink v1.3.1/go.mod h1:ARtKouGSTGchR8aMwmkzC0qiNPrrWO5JS/XMVl45+b4= +github.com/vishvananda/netns v0.0.5 h1:DfiHV+j8bA32MFM7bfEunvT8IAqQ/NzSJHtcmW5zdEY= +github.com/vishvananda/netns v0.0.5/go.mod h1:SpkAiCQRtJ6TvvxPnOSyH3BMl6unz3xZlaprSwhNNJM= +github.com/wlynxg/anet v0.0.5 h1:J3VJGi1gvo0JwZ/P1/Yc/8p63SoW98B5dHkYDmpgvvU= +github.com/wlynxg/anet v0.0.5/go.mod h1:eay5PRQr7fIVAMbTbchTnO9gG65Hg/uYGdc7mguHxoA= +github.com/xtls/reality v0.0.0-20260921001439-3c98159dee38 h1:Q5KwiMm+WRyw2kwPW4+fIQxNio1CyrWo/eAEaAHxl/Q= +github.com/xtls/reality v0.0.0-20260921001439-3c98159dee38/go.mod h1:/YQ6FwmAtkDuo8K3ujKQH1Br5eGNFauBipdl1gdiebk= +github.com/xtls/xray-core v1.260327.1-0.20261010092107-701af60772cd h1:Am/9Pzw5PZr7+hMN1Vs5faeSXRoutnrwMAgtUDnh/J8= +github.com/xtls/xray-core v1.260327.1-0.20261010092107-701af60772cd/go.mod h1:YAACF9L/Pu1GznixHOBtrBqVcE95nhFjgU+gIEdWKlo= +github.com/xyproto/randomstring v1.0.5 h1:YtlWPoRdgMu3NZtP45drfy1GKoojuR7hmRcnhZqKjWU= +github.com/xyproto/randomstring v1.0.5/go.mod h1:rgmS5DeNXLivK7YprL0pY+lTuhNQW3iGxZ18UQApw/E= +github.com/yuin/gopher-lua v0.0.0-20190206043414-8bfc7677f583/go.mod h1:gqRgreBUhTSL0GeU64rtZ3Uq3wtjOa/TB2YfrtkCbVQ= +github.com/yuin/gopher-lua v1.1.2 h1:yF/FjE3hD65tBbt0VXLE13HWS9h34fdzJmrWRXwobGA= +github.com/yuin/gopher-lua v1.1.2/go.mod h1:7aRmXIWl37SqRf0koeyylBEzJ+aPt8A+mmkQ4f1ntR8= +go.uber.org/mock v0.5.2 h1:LbtPTcP8A5k9WPXj54PPPbjcI4Y6lhyOZXn+VS7wNko= +go.uber.org/mock v0.5.2/go.mod h1:wLlUxC2vVTPTaE3UD51E0BGOAElKrILxhVSDYQLld5o= +go.yaml.in/yaml/v3 v3.0.5 h1:N6y/pJk8buWs9NY5ERU2HSMfm+IuD/OtfdAnq6kESPw= +go.yaml.in/yaml/v3 v3.0.5/go.mod h1:HVTZu1O7/Vkt2N+BFy8Zza+lnLsABggaTM2ZpNIGuKg= +go4.org/netipx v0.0.0-20260823151212-3075585bcbeb h1:XBM4hvfwGAttkkiTIFfeigdfcL1xIfdKXqFdgiHGtDs= +go4.org/netipx v0.0.0-20260823151212-3075585bcbeb/go.mod h1:PLyyIXexvUFg3Owu6p/WfdlivPbZJsZdgWZlrGope/Y= +golang.org/x/crypto v0.58.0 h1:COkYLr4k7nDI4r6QtMML2AfzeIswWHAJthLS6K/M/54= +golang.org/x/crypto v0.58.0/go.mod h1:Xh+kl5A+M0gMiWDytH2Hhr5Z7WbNCpv/eB/EFgfSk+w= +golang.org/x/crypto/x509roots/fallback v0.0.0-20261009200856-99e4382b128e h1:jsB4A6J5vF93GeMqp72mLAOcFfGXgHtxpgeLk7OJB4M= +golang.org/x/crypto/x509roots/fallback v0.0.0-20261009200856-99e4382b128e/go.mod h1:HPze8vhfG6fO06AM+VSvxRm4E3+5Yk375mgrJ5M2z1E= +golang.org/x/exp v0.0.0-20261009195045-ca0d7ba23607 h1:o259es6R0mYtmCeID0xv+dkp5kQSf6qp8zs0j4MdUaI= +golang.org/x/exp v0.0.0-20261009195045-ca0d7ba23607/go.mod h1:SRjTzy5WcYqRRqquLUvHeeovSwLO0CZ3xcltZNO4Fck= +golang.org/x/net v0.61.0 h1:RR+6j/BTBCrggOLugxa+66R8FvoN6MHlzDlZIbN9YFw= +golang.org/x/net v0.61.0/go.mod h1:WgDCOTH8iwtB66T4/GfI94AHo/hhdb7Hah9NDseQfiI= +golang.org/x/sync v0.0.0-20210220032951-036812b2e83c/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= +golang.org/x/sync v0.24.0 h1:iqiDrGoDgyuvAKrhLQthUEzU/R378N78gXbBjxed7oA= +golang.org/x/sync v0.24.0/go.mod h1:sUUOizhqBxiL6pEWpqNLUiaJn1ShEbZ6BBqskPbjZm0= +golang.org/x/sys v0.0.0-20190204203706-41f3e6584952/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= +golang.org/x/sys v0.2.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.10.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.49.0 h1:XbzkgYJHdqh/8m2Uu0W/dQv8nktxx4BFHp1M0gROTXA= +golang.org/x/sys v0.49.0/go.mod h1:hNLxWAXmnKAxqDtdwIYC4bM9oQPEecfsnNMuSxOs3og= +golang.org/x/text v0.43.0 h1:1QivrlhwAsnMOcqgOdl9my7s9OyZl8O5/UGUTJVXoWc= +golang.org/x/text v0.43.0/go.mod h1:hbSIYA/amXcRXTFSZqh8tWHEoqRvvbKPJ2uz/KU+a+s= +golang.org/x/time v0.16.0 h1:vMb6ptszcQMkcwiRTAuNNU50gom6++Q/6gY2hDM6VDE= +golang.org/x/time v0.16.0/go.mod h1:rVKOqvZeKvrDKTQiAHJ7wmwP0RzleSphoEA9RcdLA0s= +golang.zx2c4.com/wintun v0.0.0-20230126152724-0fa3db229ce2 h1:B82qJJgjvYKsXS9jeunTOisW56dUokqW/FOteYJJ/yg= +golang.zx2c4.com/wintun v0.0.0-20230126152724-0fa3db229ce2/go.mod h1:deeaetjYA+DHMHg+sMSMI58GrEteJUUzzw7en6TJQcI= +golang.zx2c4.com/wireguard v0.0.0-20261006164505-2631ce99a06f h1:gJI6OVshWW0mvGyTVmW8mJoaw2OM69wVCeZkwpX+Jdc= +golang.zx2c4.com/wireguard v0.0.0-20261006164505-2631ce99a06f/go.mod h1:rpwXGsirqLqN2L0JDJQlwOboGHmptD5ZD6T2VmcqhTw= +golang.zx2c4.com/wireguard/windows v1.1.1 h1:8/H97U1v1PNDNcBsMZgU3KFuND9MQdTsU2NOwmCXArE= +golang.zx2c4.com/wireguard/windows v1.1.1/go.mod h1:+fbT3FFdX4zzYDLwJh5+HPEcNN/3HyNdzhNSVsQM+zs= gonum.org/v1/gonum v0.17.0 h1:VbpOemQlsSMrYmn7T2OUvQ4dqxQXU+ouZFQsZOx50z4= gonum.org/v1/gonum v0.17.0/go.mod h1:El3tOrEuMpv2UdMrbNlKEh9vd86bmQ6vqIcDwxEOc1E= -google.golang.org/genproto/googleapis/rpc v0.0.0-20260706201446-f0a921348800 h1:qEHAMpSaUhtD0p3NbEEI83HwNGFxEwaSJ1G9PLnCBZE= -google.golang.org/genproto/googleapis/rpc v0.0.0-20260706201446-f0a921348800/go.mod h1:4Hqkh8ycfw05ld/3BWL7rJOSfebL2Q+DVDeRgYgxUU8= +google.golang.org/genproto/googleapis/rpc v0.0.0-20261005182115-fad411399dd8 h1:0SXBqli5dnuCKMjI/X3FM226zEaEa0GUcSCJtN1vL2o= +google.golang.org/genproto/googleapis/rpc v0.0.0-20261005182115-fad411399dd8/go.mod h1:OaIUM3+LpYcK2GXM4FTmhWoIq371Owdr+Cc7/BsYHHc= google.golang.org/grpc v1.84.0 h1:soMyaPJ8pAak5PIQ0DGBUir0XRo2fRoMqhNWMLlLxO0= google.golang.org/grpc v1.84.0/go.mod h1:ljCht0DrxQrXBDRTZp52Qxh3Ffk8CdYm2sj4O2QN2C0= google.golang.org/protobuf v1.36.12 h1:pJOKDDOyeXErUroCihFAd5LQuwXBSpVnKGrj5o/fwxc= google.golang.org/protobuf v1.36.12/go.mod h1:HTf+CrKn2C3g5S8VImy6tdcUvCska2kB7j23XfzDpco= -gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= +gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk= +gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q= +gopkg.in/yaml.v2 v2.4.0 h1:D8xgwECY7CYvx+Y2n4sBz93Jn9JRvxdiyyo8CTfuKaY= +gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ= gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= +gvisor.dev/gvisor v0.0.0-20261010064756-cd110d9dec04 h1:iVC12KE9T4FE4+ygenM2M0GGtZ5RLIGrXmY9T5UPdZk= +gvisor.dev/gvisor v0.0.0-20261010064756-cd110d9dec04/go.mod h1:LOvux1Z/goEJudXUp4E7WwCxxJeeMf5kVoprLEZvOHo= +layeh.com/gopher-luar v1.0.11 h1:8zJudpKI6HWkoh9eyyNFaTM79PY6CAPcIr6X/KTiliw= +layeh.com/gopher-luar v1.0.11/go.mod h1:TPnIVCZ2RJBndm7ohXyaqfhzjlZ+OA2SZR/YwL8tECk= +lukechampine.com/blake3 v1.4.1 h1:I3Smz7gso8w4/TunLKec6K2fn+kyKtDxr/xcQEN84Wg= +lukechampine.com/blake3 v1.4.1/go.mod h1:QFosUxmjB8mnrWFSNwKmvxHpfY72bmD2tQ0kBMM3kwo= diff --git a/internal/config/config.go b/internal/config/config.go index 0de85f8..a0d9d2d 100644 --- a/internal/config/config.go +++ b/internal/config/config.go @@ -14,6 +14,11 @@ var ( interfaceNamePattern = regexp.MustCompile(`^[A-Za-z0-9_][A-Za-z0-9_=+.-]{0,31}$`) ) +// ValidInterfaceName reports whether name is acceptable as an interface name in usher.yml. +func ValidInterfaceName(name string) bool { + return interfaceNamePattern.MatchString(name) +} + // Load reads path and returns the interface names of every user. A user with an empty list is // valid, but a file without any user/interface pair is not. Every error names the file, and the // line when the parser or a rule violation has one. diff --git a/internal/xray/config.go b/internal/xray/config.go new file mode 100644 index 0000000..494d1e1 --- /dev/null +++ b/internal/xray/config.go @@ -0,0 +1,569 @@ +// Package xray reads the operator's Xray server config and keeps the users of its inbounds in step +// with usher.yml. +package xray + +import ( + "bytes" + "encoding/json" + "errors" + "fmt" + "os" + "regexp" + "slices" + "strconv" + "strings" + "sync" + + "github.com/xtls/xray-core/common/log" + "github.com/xtls/xray-core/infra/conf" + xjson "github.com/xtls/xray-core/infra/conf/json" + + "github.com/mrcsin/usher/internal/config" +) + +// Inbound is one managed inbound of the Xray server config. Raw is the inbound object with the +// config aliases normalized to their canonical names; Detour is the same object decoded into Xray's +// own type. A non-nil Err means the inbound failed validation and must not be used. +type Inbound struct { + Tag string + Protocol string // lower case: vless, hysteria, vmess or trojan + Port int + Network string // raw, ws, grpc, httpupgrade, xhttp or hysteria + Security string // none, tls or reality + // ServerName is the name the client sends: x-usher.sni, else the first usable Reality + // serverNames entry or the tlsSettings serverName. It is empty when security is none. + ServerName string + Hints Hints + Raw map[string]any + Detour conf.InboundDetourConfig + Err error +} + +// Hints is the x-usher block of an inbound with the defaults that do not depend on usher's +// environment applied. An empty Address means USHER_HOST. +type Hints struct { + Address string + Port int + Fingerprint string + Trust string // "ca" or "pin" + PinSHA256 string // lower case hex, empty when the pin comes from the inline certificate + HopPorts string // "from-to", Hysteria2 only +} + +const ( + // TrustCA makes the client verify the server certificate against its CA roots. + TrustCA = "ca" + // TrustPin makes the client also pin the leaf certificate. + TrustPin = "pin" + + defaultFingerprint = "chrome" +) + +var ( + // realityFingerprints are the uTLS fingerprints that carry X25519MLKEM768, which the Reality + // server requires. + realityFingerprints = []string{"chrome", "firefox", "safari"} + + managedProtocols = map[string]bool{"vless": true, "hysteria": true, "vmess": true, "trojan": true} + networkAliases = map[string]string{"tcp": "raw", "splithttp": "xhttp", "websocket": "ws"} + networks = map[string]bool{ + "raw": true, "ws": true, "grpc": true, "httpupgrade": true, "xhttp": true, "hysteria": true, + } + hopPortsPattern = regexp.MustCompile(`^([0-9]{1,5})-([0-9]{1,5})$`) + sha256Pattern = regexp.MustCompile(`^[0-9a-fA-F]{64}$`) +) + +// discardLog drops Xray's global log output. Build prints deprecation warnings, and usher runs no +// Xray instance whose log they belong to. +type discardLog struct{} + +func (discardLog) Handle(log.Message) {} + +var silenceOnce sync.Once + +func silenceXrayLog() { + silenceOnce.Do(func() { log.RegisterHandler(discardLog{}) }) +} + +// ReadConfig reads the Xray server config at path and returns its managed inbounds in file order. +// Inbounds of other protocols and inbounds without a tag are skipped. An inbound that fails +// validation is returned with Err set, so the caller can keep its files. An error means the file +// as a whole is unusable. +func ReadConfig(path string) ([]Inbound, error) { + silenceXrayLog() + f, err := os.Open(path) + if err != nil { + return nil, fmt.Errorf("reading xray config: %w", err) + } + defer f.Close() + + var file struct { + Inbounds []json.RawMessage `json:"inbounds"` + } + if err := json.NewDecoder(&xjson.Reader{Reader: f}).Decode(&file); err != nil { + return nil, fmt.Errorf("parsing xray config %s: %w", path, err) + } + + var inbounds []Inbound + seen := map[string]bool{} + for i, data := range file.Inbounds { + raw, err := decodeObject(data) + if err != nil { + return nil, fmt.Errorf("parsing xray config %s: inbounds[%d]: %w", path, i, err) + } + tag, _ := raw["tag"].(string) + protocol, _ := raw["protocol"].(string) + protocol = strings.ToLower(protocol) + if tag == "" || !managedProtocols[protocol] { + continue + } + in := Inbound{Tag: tag, Protocol: protocol, Raw: raw} + switch { + case seen[tag]: + in.Err = fmt.Errorf("inbound %q: tag: defined twice", tag) + case !config.ValidInterfaceName(tag): + in.Err = fmt.Errorf("inbound %q: tag: not a valid interface name", tag) + default: + in.Err = in.validate() + } + seen[tag] = true + inbounds = append(inbounds, in) + } + return inbounds, nil +} + +func decodeObject(data []byte) (map[string]any, error) { + dec := json.NewDecoder(bytes.NewReader(data)) + dec.UseNumber() + var obj map[string]any + if err := dec.Decode(&obj); err != nil { + return nil, err + } + if obj == nil { + return nil, errors.New("want an object") + } + return obj, nil +} + +// validate normalizes in.Raw, checks it against what the backend supports and fills the derived +// fields. Every error names the field and never quotes a value. +func (in *Inbound) validate() error { + fail := func(field, reason string) error { + return fmt.Errorf("inbound %q: %s: %s", in.Tag, field, reason) + } + + port, err := in.singlePort() + if err != nil { + return fail("port", err.Error()) + } + in.Port = port + if listen, ok := in.Raw["listen"].(string); ok && (strings.HasPrefix(listen, "/") || strings.HasPrefix(listen, "@")) { + return fail("listen", "unix socket is not supported") + } + + settings, err := in.settings() + if err != nil { + return fail("settings", err.Error()) + } + if err := in.checkUsersEmpty(settings); err != nil { + return fail("settings.clients", err.Error()) + } + if in.Protocol == "vless" && lowerString(settings["decryption"]) != "none" { + return fail("settings.decryption", `want "none"`) + } + + stream, err := in.normalizeStream() + if err != nil { + return fail("streamSettings", err.Error()) + } + if field, reason := in.checkTransport(stream); field != "" { + return fail(field, reason) + } + if field, reason := in.checkHysteria(stream); field != "" { + return fail(field, reason) + } + if field, reason := in.checkMasks(stream); field != "" { + return fail(field, reason) + } + if field, reason := in.checkXHTTP(stream); field != "" { + return fail(field, reason) + } + if field, reason := in.readHints(stream); field != "" { + return fail(field, reason) + } + + // The decode catches field type errors that the checks above do not look at. + encoded, err := json.Marshal(in.Raw) + if err != nil { + return fail("inbound", "cannot be encoded") + } + if err := json.Unmarshal(encoded, &in.Detour); err != nil { + return fail("inbound", "does not decode as an Xray inbound: "+err.Error()) + } + return nil +} + +func (in *Inbound) singlePort() (int, error) { + number, ok := in.Raw["port"].(json.Number) + if !ok { + return 0, errors.New("want a single integer") + } + port, err := strconv.Atoi(number.String()) + if err != nil || port < 1 || port > 65535 { + return 0, errors.New("want a single integer from 1 to 65535") + } + return port, nil +} + +func (in *Inbound) settings() (map[string]any, error) { + v, present := in.Raw["settings"] + if !present { + settings := map[string]any{} + in.Raw["settings"] = settings + return settings, nil + } + settings, ok := v.(map[string]any) + if !ok { + return nil, errors.New("want an object") + } + return settings, nil +} + +// checkUsersEmpty folds the Hysteria alias users into clients and requires the list to be empty, +// because usher owns every user of the inbound. +func (in *Inbound) checkUsersEmpty(settings map[string]any) error { + if in.Protocol == "hysteria" { + if users, ok := settings["users"]; ok { + if clients, both := settings["clients"]; both && !isEmptyList(clients) { + return errors.New("users and clients are both set") + } + settings["clients"] = users + delete(settings, "users") + } + } + if clients, ok := settings["clients"]; ok && !isEmptyList(clients) { + return errors.New("must be empty, usher owns the users") + } + return nil +} + +func isEmptyList(v any) bool { + list, ok := v.([]any) + return v == nil || (ok && len(list) == 0) +} + +// normalizeStream rewrites the aliases of streamSettings to their canonical names and fills in +// Network and Security. It returns the streamSettings object, created when the inbound has none. +func (in *Inbound) normalizeStream() (map[string]any, error) { + v, present := in.Raw["streamSettings"] + stream, ok := v.(map[string]any) + if present && !ok { + return nil, errors.New("want an object") + } + if stream == nil { + stream = map[string]any{} + in.Raw["streamSettings"] = stream + } + + // Xray lets method win over network. + if method, ok := stream["method"]; ok { + stream["network"] = method + delete(stream, "method") + } + network := "raw" + if v, ok := stream["network"]; ok { + s, isString := v.(string) + if !isString { + return nil, errors.New("network: want a string") + } + network = strings.ToLower(s) + if canonical, ok := networkAliases[network]; ok { + network = canonical + } + } + stream["network"] = network + in.Network = network + + for alias, canonical := range map[string]string{ + "splithttpSettings": "xhttpSettings", + "tcpSettings": "rawSettings", + } { + v, ok := stream[alias] + if !ok { + continue + } + if _, clash := stream[canonical]; clash { + return nil, fmt.Errorf("%s: conflicts with %s", alias, canonical) + } + stream[canonical] = v + delete(stream, alias) + } + + security := "none" + if v, ok := stream["security"]; ok { + s, isString := v.(string) + if !isString { + return nil, errors.New("security: want a string") + } + if s != "" { + security = strings.ToLower(s) + } + } + stream["security"] = security + in.Security = security + return stream, nil +} + +func (in *Inbound) checkTransport(stream map[string]any) (field, reason string) { + if !networks[in.Network] { + return "streamSettings.network", "unsupported transport" + } + switch in.Security { + case "none", "tls", "reality": + default: + return "streamSettings.security", "want none, tls or reality" + } + if (in.Protocol == "hysteria") != (in.Network == "hysteria") { + return "streamSettings.network", "hysteria transport and hysteria protocol go together" + } + if in.Network == "raw" { + header := object(object(stream["rawSettings"])["header"]) + if t := lowerString(header["type"]); t != "" && t != "none" { + return "streamSettings.rawSettings.header.type", `want "none"` + } + } + return "", "" +} + +func (in *Inbound) checkHysteria(stream map[string]any) (field, reason string) { + if in.Protocol != "hysteria" { + return "", "" + } + if in.Security != "tls" { + return "streamSettings.security", `hysteria needs "tls"` + } + if !isVersion2(object(in.Raw["settings"])["version"]) { + return "settings.version", "want 2" + } + hysteria := object(stream["hysteriaSettings"]) + if !isVersion2(hysteria["version"]) { + return "streamSettings.hysteriaSettings.version", "want 2" + } + if auth, _ := hysteria["auth"].(string); auth != "" { + return "streamSettings.hysteriaSettings.auth", "must be empty, usher gives each user an auth" + } + return "", "" +} + +func isVersion2(v any) bool { + n, ok := v.(json.Number) + return ok && n.String() == "2" +} + +// checkMasks allows only salamander without packetSize in the udp list, and quicParams. +func (in *Inbound) checkMasks(stream map[string]any) (field, reason string) { + mask, present := stream["finalmask"] + if !present { + return "", "" + } + finalmask, ok := mask.(map[string]any) + if !ok { + return "streamSettings.finalmask", "want an object" + } + if tcp, ok := finalmask["tcp"]; ok && !isEmptyList(tcp) { + return "streamSettings.finalmask.tcp", "unsupported mask" + } + udp, ok := finalmask["udp"].([]any) + if !ok && finalmask["udp"] != nil { + return "streamSettings.finalmask.udp", "want a list" + } + for _, item := range udp { + entry := object(item) + if lowerString(entry["type"]) != "salamander" { + return "streamSettings.finalmask.udp", "only salamander is supported" + } + if _, ok := object(entry["settings"])["packetSize"]; ok { + return "streamSettings.finalmask.udp.settings.packetSize", "unsupported with salamander" + } + } + if in.Protocol != "hysteria" && len(udp) > 0 { + return "streamSettings.finalmask.udp", "salamander is supported on hysteria only" + } + return "", "" +} + +func (in *Inbound) checkXHTTP(stream map[string]any) (field, reason string) { + if in.Network != "xhttp" { + return "", "" + } + xhttp := object(stream["xhttpSettings"]) + if _, ok := xhttp["downloadSettings"]; ok { + return "streamSettings.xhttpSettings.downloadSettings", "unsupported" + } + extra := xhttp["extra"] + if s, ok := extra.(string); ok && s != "" { + return "streamSettings.xhttpSettings.extra", "want an object" + } + if _, ok := object(extra)["downloadSettings"]; ok { + return "streamSettings.xhttpSettings.extra.downloadSettings", "unsupported" + } + return "", "" +} + +// xUsher mirrors the x-usher block. Pointers tell an absent field from a zero one. +type xUsher struct { + Address string `json:"address"` + Port *int `json:"port"` + SNI string `json:"sni"` + Fingerprint string `json:"fingerprint"` + Trust string `json:"trust"` + PinSHA256 string `json:"pin_sha256"` + HopPorts string `json:"hop_ports"` +} + +// readHints decodes x-usher, checks it against the inbound and fills Hints and ServerName. +func (in *Inbound) readHints(stream map[string]any) (field, reason string) { + var hint xUsher + if block, ok := in.Raw["x-usher"]; ok { + encoded, err := json.Marshal(block) + if err != nil { + return "x-usher", "cannot be encoded" + } + dec := json.NewDecoder(bytes.NewReader(encoded)) + dec.DisallowUnknownFields() + if err := dec.Decode(&hint); err != nil { + return "x-usher", err.Error() + } + } + + port := in.Port + if hint.Port != nil { + if *hint.Port < 1 || *hint.Port > 65535 { + return "x-usher.port", "want 1 to 65535" + } + port = *hint.Port + } + + trust := hint.Trust + switch trust { + case "": + trust = TrustCA + case TrustCA, TrustPin: + default: + return "x-usher.trust", `want "ca" or "pin"` + } + if trust == TrustPin && in.Security != "tls" { + return "x-usher.trust", `"pin" needs security "tls"` + } + if hint.PinSHA256 != "" { + if trust != TrustPin { + return "x-usher.pin_sha256", `valid only with trust "pin"` + } + if !sha256Pattern.MatchString(hint.PinSHA256) { + return "x-usher.pin_sha256", "want 64 hex characters" + } + } + if trust == TrustPin && hint.PinSHA256 == "" && !hasInlineCertificate(stream) { + return "x-usher.pin_sha256", `trust "pin" needs it or an inline certificate` + } + + fingerprint := hint.Fingerprint + if fingerprint == "" { + fingerprint = defaultFingerprint + } + if in.Security == "reality" && !slices.Contains(realityFingerprints, fingerprint) { + return "x-usher.fingerprint", "reality supports chrome, firefox and safari only" + } + + if hint.HopPorts != "" { + if in.Protocol != "hysteria" { + return "x-usher.hop_ports", "valid on hysteria only" + } + if !validHopPorts(hint.HopPorts) { + return "x-usher.hop_ports", `want "from-to" with 1 <= from <= to <= 65535` + } + } + + serverName, field, reason := in.serverName(stream, hint.SNI) + if field != "" { + return field, reason + } + in.ServerName = serverName + in.Hints = Hints{ + Address: hint.Address, + Port: port, + Fingerprint: fingerprint, + Trust: trust, + PinSHA256: strings.ToLower(hint.PinSHA256), + HopPorts: hint.HopPorts, + } + return "", "" +} + +// serverName resolves the name the client sends. An explicit x-usher.sni wins over the server +// config, which is how an operator with only wildcard Reality names gets a usable link. +func (in *Inbound) serverName(stream map[string]any, override string) (name, field, reason string) { + switch in.Security { + case "tls": + name = override + if name == "" { + name, _ = object(stream["tlsSettings"])["serverName"].(string) + } + if name == "" { + return "", "x-usher.sni", "tls needs it or tlsSettings.serverName" + } + case "reality": + name = override + if name == "" { + names, _ := object(stream["realitySettings"])["serverNames"].([]any) + for _, v := range names { + if s, _ := v.(string); s != "" && !strings.Contains(s, "*") { + name = s + break + } + } + } + if name == "" { + return "", "streamSettings.realitySettings.serverNames", "no entry without a wildcard, set x-usher.sni" + } + } + return name, "", "" +} + +func hasInlineCertificate(stream map[string]any) bool { + certs, _ := object(stream["tlsSettings"])["certificates"].([]any) + for _, c := range certs { + switch certificate := object(c)["certificate"].(type) { + case string: + if certificate != "" { + return true + } + case []any: + if len(certificate) > 0 { + return true + } + } + } + return false +} + +func validHopPorts(s string) bool { + m := hopPortsPattern.FindStringSubmatch(s) + if m == nil { + return false + } + from, _ := strconv.Atoi(m[1]) + to, _ := strconv.Atoi(m[2]) + return from >= 1 && from <= to && to <= 65535 +} + +// object returns v as a JSON object, or nil when it is not one. A read from nil is safe. +func object(v any) map[string]any { + m, _ := v.(map[string]any) + return m +} + +func lowerString(v any) string { + s, _ := v.(string) + return strings.ToLower(s) +} diff --git a/internal/xray/config_test.go b/internal/xray/config_test.go new file mode 100644 index 0000000..1666971 --- /dev/null +++ b/internal/xray/config_test.go @@ -0,0 +1,410 @@ +package xray + +import ( + "encoding/json" + "fmt" + "os" + "path/filepath" + "strings" + "testing" +) + +func TestReadConfigShapes(t *testing.T) { + inbounds, err := ReadConfig("testdata/server.json") + if err != nil { + t.Fatal(err) + } + want := []Inbound{ + {Tag: "vless-reality", Protocol: "vless", Port: 443, Network: "raw", Security: "reality", + ServerName: "www.example.com", + Hints: Hints{Address: "203.0.113.10", Port: 443, Fingerprint: "chrome", Trust: "ca"}}, + {Tag: "xh", Protocol: "vless", Port: 2053, Network: "xhttp", Security: "tls", + ServerName: "xh.example.com", Hints: Hints{Port: 2053, Fingerprint: "chrome", Trust: "ca"}}, + {Tag: "xh-extra", Protocol: "vless", Port: 2054, Network: "xhttp", Security: "tls", + ServerName: "xh.example.com", Hints: Hints{Port: 2054, Fingerprint: "chrome", Trust: "ca"}}, + {Tag: "ws", Protocol: "vless", Port: 2083, Network: "ws", Security: "tls", + ServerName: "ws.example.com", Hints: Hints{Port: 2083, Fingerprint: "chrome", Trust: "ca"}}, + {Tag: "grpc", Protocol: "vless", Port: 2087, Network: "grpc", Security: "tls", + ServerName: "g.example.com", Hints: Hints{Port: 2087, Fingerprint: "chrome", Trust: "ca"}}, + {Tag: "hu", Protocol: "vless", Port: 2088, Network: "httpupgrade", Security: "tls", + ServerName: "hu.example.com", Hints: Hints{Port: 2088, Fingerprint: "chrome", Trust: "ca"}}, + {Tag: "hy2", Protocol: "hysteria", Port: 8443, Network: "hysteria", Security: "tls", + ServerName: "hy.example.com", + Hints: Hints{Port: 8443, Fingerprint: "chrome", Trust: "pin", HopPorts: "20000-30000", + PinSHA256: "aabbccddeeff00112233445566778899aabbccddeeff00112233445566778899"}}, + {Tag: "vmess-ws", Protocol: "vmess", Port: 2096, Network: "ws", Security: "none", + Hints: Hints{Port: 2096, Fingerprint: "chrome", Trust: "ca"}}, + {Tag: "trojan", Protocol: "trojan", Port: 8444, Network: "raw", Security: "tls", + ServerName: "t.example.com", Hints: Hints{Port: 8444, Fingerprint: "chrome", Trust: "ca"}}, + } + if len(inbounds) != len(want) { + t.Fatalf("got %d inbounds, want %d", len(inbounds), len(want)) + } + for i, got := range inbounds { + w := want[i] + t.Run(w.Tag, func(t *testing.T) { + if got.Err != nil { + t.Fatalf("unexpected error: %v", got.Err) + } + if got.Tag != w.Tag || got.Protocol != w.Protocol || got.Port != w.Port || + got.Network != w.Network || got.Security != w.Security || + got.ServerName != w.ServerName || got.Hints != w.Hints { + t.Errorf("got %+v, want %+v", summary(got), summary(w)) + } + if got.Detour.Tag != w.Tag || got.Detour.Protocol != w.Protocol { + t.Errorf("detour = %s/%s, want %s/%s", got.Detour.Tag, got.Detour.Protocol, w.Tag, w.Protocol) + } + if w.Tag == "xh" { + return // Build reads the certificate files, which do not exist here + } + if _, err := got.Detour.Build(); err != nil { + t.Errorf("Detour.Build: %v", err) + } + }) + } +} + +func summary(in Inbound) string { + return fmt.Sprintf("%s %s %d %s %s %q %+v", in.Tag, in.Protocol, in.Port, in.Network, in.Security, + in.ServerName, in.Hints) +} + +func TestReadConfigAliases(t *testing.T) { + inbounds, err := ReadConfig("testdata/aliases.json") + if err != nil { + t.Fatal(err) + } + byTag := map[string]Inbound{} + for _, in := range inbounds { + byTag[in.Tag] = in + } + tests := []struct { + tag string + network string + check func(t *testing.T, in Inbound) + }{ + {"a1", "xhttp", func(t *testing.T, in Inbound) { // method wins over network + stream := object(in.Raw["streamSettings"]) + if _, ok := stream["method"]; ok { + t.Error("method kept") + } + if object(stream["xhttpSettings"])["path"] != "/x" { + t.Errorf("xhttpSettings = %v", stream["xhttpSettings"]) + } + if in.Protocol != "vless" || in.Security != "tls" { + t.Errorf("protocol/security = %s/%s", in.Protocol, in.Security) + } + }}, + {"a2", "ws", nil}, + {"a3", "raw", func(t *testing.T, in Inbound) { + if _, ok := object(in.Raw["streamSettings"])["rawSettings"]; !ok { + t.Error("tcpSettings not moved to rawSettings") + } + }}, + {"a4", "hysteria", func(t *testing.T, in Inbound) { + settings := object(in.Raw["settings"]) + if _, ok := settings["users"]; ok { + t.Error("users kept") + } + if _, ok := settings["clients"]; !ok { + t.Error("users not moved to clients") + } + }}, + } + for _, tt := range tests { + t.Run(tt.tag, func(t *testing.T) { + in, ok := byTag[tt.tag] + if !ok { + t.Fatal("inbound missing") + } + if in.Err != nil { + t.Fatalf("unexpected error: %v", in.Err) + } + if in.Network != tt.network { + t.Errorf("network = %q, want %q", in.Network, tt.network) + } + if tt.check != nil { + tt.check(t, in) + } + }) + } +} + +func baseInbound(t *testing.T, kind string) map[string]any { + t.Helper() + docs := map[string]string{ + "vless-tls": `{"tag":"t","protocol":"vless","port":443,"settings":{"clients":[],"decryption":"none"}, + "streamSettings":{"network":"ws","security":"tls","tlsSettings":{"serverName":"a.example.com"}}}`, + "reality": `{"tag":"t","protocol":"vless","port":443,"settings":{"clients":[],"decryption":"none"}, + "streamSettings":{"network":"raw","security":"reality","realitySettings":{"serverNames":["a.example.com"]}}}`, + "xhttp": `{"tag":"t","protocol":"vless","port":443,"settings":{"clients":[],"decryption":"none"}, + "streamSettings":{"network":"xhttp","security":"tls","tlsSettings":{"serverName":"a.example.com"}, + "xhttpSettings":{"path":"/x"}}}`, + "hysteria": `{"tag":"t","protocol":"hysteria","port":443,"settings":{"version":2,"clients":[]}, + "streamSettings":{"network":"hysteria","security":"tls","tlsSettings":{"serverName":"a.example.com"}, + "hysteriaSettings":{"version":2}, + "finalmask":{"udp":[{"type":"salamander","settings":{"password":"p"}}]}}}`, + } + var m map[string]any + if err := json.Unmarshal([]byte(docs[kind]), &m); err != nil { + t.Fatal(err) + } + return m +} + +// setPath sets the dotted path in m; a nil value deletes the key. +func setPath(m map[string]any, path string, value any) { + parts := strings.Split(path, ".") + for _, p := range parts[:len(parts)-1] { + next, ok := m[p].(map[string]any) + if !ok { + next = map[string]any{} + m[p] = next + } + m = next + } + last := parts[len(parts)-1] + if value == nil { + delete(m, last) + return + } + m[last] = value +} + +func readOne(t *testing.T, inbound map[string]any) Inbound { + t.Helper() + data, err := json.Marshal(map[string]any{"inbounds": []any{inbound}}) + if err != nil { + t.Fatal(err) + } + path := filepath.Join(t.TempDir(), "config.json") + if err := os.WriteFile(path, data, 0o600); err != nil { + t.Fatal(err) + } + inbounds, err := ReadConfig(path) + if err != nil { + t.Fatal(err) + } + if len(inbounds) != 1 { + t.Fatalf("got %d inbounds, want 1", len(inbounds)) + } + return inbounds[0] +} + +func TestReadConfigRejects(t *testing.T) { + tests := []struct { + name string + kind string + set map[string]any + want string // error substring, a field name + }{ + {"port range", "vless-tls", map[string]any{"port": "443-445"}, "port: want a single integer"}, + {"port list", "vless-tls", map[string]any{"port": "443,444"}, "port:"}, + {"port zero", "vless-tls", map[string]any{"port": 0}, "port:"}, + {"port missing", "vless-tls", map[string]any{"port": nil}, "port:"}, + {"unix listen", "vless-tls", map[string]any{"listen": "/run/x.sock"}, "listen:"}, + {"abstract listen", "vless-tls", map[string]any{"listen": "@x"}, "listen:"}, + {"clients not empty", "vless-tls", map[string]any{"settings.clients": []any{map[string]any{"id": "x"}}}, "settings.clients"}, + {"decryption", "vless-tls", map[string]any{"settings.decryption": "mlkem"}, "settings.decryption"}, + {"decryption missing", "vless-tls", map[string]any{"settings.decryption": nil}, "settings.decryption"}, + {"kcp", "vless-tls", map[string]any{"streamSettings.network": "kcp"}, "streamSettings.network"}, + {"network not string", "vless-tls", map[string]any{"streamSettings.network": 3}, "streamSettings"}, + {"security", "vless-tls", map[string]any{"streamSettings.security": "xtls"}, "streamSettings.security"}, + {"raw http header", "vless-tls", map[string]any{ + "streamSettings.network": "raw", "streamSettings.rawSettings.header.type": "http"}, + "streamSettings.rawSettings.header.type"}, + {"alias clash", "vless-tls", map[string]any{ + "streamSettings.tcpSettings": map[string]any{}, "streamSettings.rawSettings": map[string]any{}}, + "tcpSettings"}, + {"tls without sni", "vless-tls", map[string]any{"streamSettings.tlsSettings.serverName": nil}, "x-usher.sni"}, + {"reality wildcard only", "reality", map[string]any{ + "streamSettings.realitySettings.serverNames": []any{"*.example.com", ""}}, + "realitySettings.serverNames"}, + {"reality fingerprint", "reality", map[string]any{"x-usher.fingerprint": "ios"}, "x-usher.fingerprint"}, + {"unknown x-usher key", "vless-tls", map[string]any{"x-usher.adress": "x"}, "x-usher"}, + {"x-usher port", "vless-tls", map[string]any{"x-usher.port": 70000}, "x-usher.port"}, + {"x-usher port type", "vless-tls", map[string]any{"x-usher.port": "443"}, "x-usher"}, + {"trust value", "vless-tls", map[string]any{"x-usher.trust": "none"}, "x-usher.trust"}, + {"pin without tls", "reality", map[string]any{"x-usher.trust": "pin"}, "x-usher.trust"}, + {"pin_sha256 without pin", "vless-tls", map[string]any{"x-usher.pin_sha256": strings.Repeat("a", 64)}, "x-usher.pin_sha256"}, + {"pin_sha256 format", "vless-tls", map[string]any{"x-usher.trust": "pin", "x-usher.pin_sha256": "abc"}, "x-usher.pin_sha256"}, + {"pin without source", "vless-tls", map[string]any{"x-usher.trust": "pin"}, "x-usher.pin_sha256"}, + {"pin with certificateFile only", "vless-tls", map[string]any{ + "x-usher.trust": "pin", + "streamSettings.tlsSettings.certificates": []any{map[string]any{"certificateFile": "/c.pem"}}}, + "x-usher.pin_sha256"}, + {"hop on vless", "vless-tls", map[string]any{"x-usher.hop_ports": "20000-30000"}, "x-usher.hop_ports"}, + {"hop format", "hysteria", map[string]any{"x-usher.hop_ports": "30000-20000"}, "x-usher.hop_ports"}, + {"hop single port", "hysteria", map[string]any{"x-usher.hop_ports": "20000"}, "x-usher.hop_ports"}, + {"xhttp downloadSettings", "xhttp", map[string]any{ + "streamSettings.xhttpSettings.downloadSettings": map[string]any{}}, + "xhttpSettings.downloadSettings"}, + {"xhttp extra downloadSettings", "xhttp", map[string]any{ + "streamSettings.xhttpSettings.extra": map[string]any{"downloadSettings": map[string]any{}}}, + "xhttpSettings.extra.downloadSettings"}, + {"hysteria users and clients", "hysteria", map[string]any{ + "settings.users": []any{map[string]any{"auth": "x"}}, "settings.clients": []any{map[string]any{"auth": "y"}}}, + "settings.clients"}, + {"hysteria users", "hysteria", map[string]any{"settings.users": []any{map[string]any{"auth": "x"}}}, "settings.clients"}, + {"hysteria auth", "hysteria", map[string]any{"streamSettings.hysteriaSettings.auth": "secret"}, "hysteriaSettings.auth"}, + {"hysteria version", "hysteria", map[string]any{"streamSettings.hysteriaSettings.version": 1}, "hysteriaSettings.version"}, + {"hysteria settings version", "hysteria", map[string]any{"settings.version": nil}, "settings.version"}, + {"hysteria without tls", "hysteria", map[string]any{"streamSettings.security": "none"}, "streamSettings.security"}, + {"hysteria protocol on ws", "hysteria", map[string]any{"streamSettings.network": "ws"}, "streamSettings.network"}, + {"hysteria transport on vless", "vless-tls", map[string]any{"streamSettings.network": "hysteria"}, "streamSettings.network"}, + {"mask type", "hysteria", map[string]any{ + "streamSettings.finalmask.udp": []any{map[string]any{"type": "noise"}}}, + "finalmask.udp"}, + {"salamander packetSize", "hysteria", map[string]any{ + "streamSettings.finalmask.udp": []any{map[string]any{ + "type": "salamander", "settings": map[string]any{"password": "p", "packetSize": "100-200"}}}}, + "packetSize"}, + {"tcp mask", "hysteria", map[string]any{ + "streamSettings.finalmask.tcp": []any{map[string]any{"type": "fragment"}}}, + "finalmask.tcp"}, + {"salamander on vless", "vless-tls", map[string]any{ + "streamSettings.finalmask.udp": []any{map[string]any{"type": "salamander"}}}, + "finalmask.udp"}, + {"tag pattern", "vless-tls", map[string]any{"tag": "a/b"}, "tag:"}, + {"tag too long", "vless-tls", map[string]any{"tag": strings.Repeat("a", 33)}, "tag:"}, + {"sockopt type", "vless-tls", map[string]any{"streamSettings.sockopt": "x"}, "inbound:"}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + m := baseInbound(t, tt.kind) + for path, v := range tt.set { + setPath(m, path, v) + } + in := readOne(t, m) + if in.Err == nil { + t.Fatal("no error") + } + if !strings.Contains(in.Err.Error(), tt.want) { + t.Errorf("error %q does not contain %q", in.Err, tt.want) + } + }) + } +} + +func TestReadConfigAcceptsEdgeShapes(t *testing.T) { + tests := []struct { + name string + kind string + set map[string]any + check func(t *testing.T, in Inbound) + }{ + {"no streamSettings", "vless-tls", map[string]any{"streamSettings": nil}, func(t *testing.T, in Inbound) { + if in.Network != "raw" || in.Security != "none" || in.ServerName != "" { + t.Errorf("got %s/%s/%q", in.Network, in.Security, in.ServerName) + } + }}, + {"sni override beats wildcard names", "reality", map[string]any{ + "streamSettings.realitySettings.serverNames": []any{"*.example.com"}, + "x-usher.sni": "front.example.com"}, func(t *testing.T, in Inbound) { + if in.ServerName != "front.example.com" { + t.Errorf("ServerName = %q", in.ServerName) + } + }}, + {"reality safari", "reality", map[string]any{"x-usher.fingerprint": "safari"}, func(t *testing.T, in Inbound) { + if in.Hints.Fingerprint != "safari" { + t.Errorf("fingerprint = %q", in.Hints.Fingerprint) + } + }}, + {"pin from inline certificate", "vless-tls", map[string]any{ + "x-usher.trust": "pin", + "streamSettings.tlsSettings.certificates": []any{map[string]any{"certificate": []any{"-----BEGIN CERTIFICATE-----"}}}}, + func(t *testing.T, in Inbound) { + if in.Hints.Trust != TrustPin || in.Hints.PinSHA256 != "" { + t.Errorf("hints = %+v", in.Hints) + } + }}, + {"x-usher port and sni", "vless-tls", map[string]any{"x-usher.port": 8443, "x-usher.sni": "o.example.com"}, + func(t *testing.T, in Inbound) { + if in.Hints.Port != 8443 || in.ServerName != "o.example.com" || in.Port != 443 { + t.Errorf("port %d hints %+v sni %q", in.Port, in.Hints, in.ServerName) + } + }}, + {"hysteria quicParams", "hysteria", map[string]any{ + "streamSettings.finalmask.quicParams": map[string]any{"congestion": "bbr"}}, nil}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + m := baseInbound(t, tt.kind) + for path, v := range tt.set { + setPath(m, path, v) + } + in := readOne(t, m) + if in.Err != nil { + t.Fatalf("unexpected error: %v", in.Err) + } + if tt.check != nil { + tt.check(t, in) + } + }) + } +} + +func TestReadConfigErrorsDoNotQuoteValues(t *testing.T) { + const secret = "s3cr3t-value-0123" + for _, path := range []string{ + "streamSettings.hysteriaSettings.auth", + "streamSettings.finalmask.udp", + } { + m := baseInbound(t, "hysteria") + value := any(secret) + if path == "streamSettings.finalmask.udp" { + value = []any{map[string]any{"type": "salamander", "settings": map[string]any{"password": secret, "packetSize": secret}}} + } + setPath(m, path, value) + in := readOne(t, m) + if in.Err == nil { + t.Fatalf("%s: no error", path) + } + if strings.Contains(in.Err.Error(), secret) { + t.Errorf("%s: error quotes the value: %v", path, in.Err) + } + } +} + +func TestReadConfigSkipsAndDuplicates(t *testing.T) { + path := filepath.Join(t.TempDir(), "config.json") + doc := `{"inbounds":[ + {"tag":"x","protocol":"vless","port":1,"settings":{"clients":[],"decryption":"none"}}, + {"tag":"x","protocol":"trojan","port":2,"settings":{"clients":[]}}, + {"tag":"s","protocol":"shadowsocks","port":3,"settings":{}}, + {"protocol":"trojan","port":4,"settings":{"clients":[]}} + ]}` + if err := os.WriteFile(path, []byte(doc), 0o600); err != nil { + t.Fatal(err) + } + inbounds, err := ReadConfig(path) + if err != nil { + t.Fatal(err) + } + if len(inbounds) != 2 { + t.Fatalf("got %d inbounds, want 2", len(inbounds)) + } + if inbounds[0].Err != nil { + t.Errorf("first: %v", inbounds[0].Err) + } + if inbounds[1].Err == nil || !strings.Contains(inbounds[1].Err.Error(), "tag: defined twice") { + t.Errorf("second: %v", inbounds[1].Err) + } +} + +func TestReadConfigFileErrors(t *testing.T) { + dir := t.TempDir() + bad := filepath.Join(dir, "bad.json") + if err := os.WriteFile(bad, []byte(`{"inbounds": [`), 0o600); err != nil { + t.Fatal(err) + } + notObject := filepath.Join(dir, "scalar.json") + if err := os.WriteFile(notObject, []byte(`{"inbounds": [3]}`), 0o600); err != nil { + t.Fatal(err) + } + for _, path := range []string{filepath.Join(dir, "missing.json"), bad, notObject} { + if _, err := ReadConfig(path); err == nil { + t.Errorf("%s: no error", filepath.Base(path)) + } + } +} diff --git a/internal/xray/testdata/aliases.json b/internal/xray/testdata/aliases.json new file mode 100644 index 0000000..70da81a --- /dev/null +++ b/internal/xray/testdata/aliases.json @@ -0,0 +1,28 @@ +{ + "inbounds": [ + { + "tag": "a1", "protocol": "VLESS", "port": 2053, + "settings": {"clients": null, "decryption": "none"}, + "streamSettings": { + "method": "splithttp", "network": "ws", "security": "TLS", + "tlsSettings": {"serverName": "a.example.com"}, + "splithttpSettings": {"path": "/x"} + } + }, + { + "tag": "a2", "protocol": "vless", "port": 2083, + "settings": {"clients": [], "decryption": "none"}, + "streamSettings": {"network": "websocket", "security": "tls", "tlsSettings": {"serverName": "a.example.com"}} + }, + { + "tag": "a3", "protocol": "vless", "port": 2084, + "settings": {"clients": [], "decryption": "none"}, + "streamSettings": {"network": "tcp", "tcpSettings": {"header": {"type": "none"}}} + }, + { + "tag": "a4", "protocol": "hysteria", "port": 8443, + "settings": {"version": 2, "users": []}, + "streamSettings": {"network": "hysteria", "security": "tls", "tlsSettings": {"serverName": "h.example.com"}, "hysteriaSettings": {"version": 2}} + } + ] +} diff --git a/internal/xray/testdata/server.json b/internal/xray/testdata/server.json new file mode 100644 index 0000000..96d95e6 --- /dev/null +++ b/internal/xray/testdata/server.json @@ -0,0 +1,79 @@ +// Operator server config: every shape the backend accepts. +{ + "api": {"listen": "/run/xray/api.sock,0660", "services": ["HandlerService"], "tag": "api"}, + /* inbounds the backend skips */ + "inbounds": [ + {"tag": "api-in", "protocol": "dokodemo-door", "port": 10085, "settings": {"address": "127.0.0.1"}}, + {"protocol": "vless", "port": 9999, "settings": {"clients": [], "decryption": "none"}}, + { + "tag": "vless-reality", "protocol": "vless", "port": 443, + "settings": {"clients": [], "decryption": "none"}, + "streamSettings": { + "network": "raw", "security": "reality", + "realitySettings": { + "privateKey": "cEUEZw0y-gOyW9gPA93c1bt6TrCFqbsXu4o9dTC0Zt0", + "serverNames": ["*.example.com", "", "www.example.com"], + "shortIds": ["ab12"], "target": "www.example.com:443" + } + }, + "x-usher": {"address": "203.0.113.10"} # trailing comment + }, + { + "tag": "xh", "protocol": "vless", "port": 2053, + "settings": {"clients": [], "decryption": "none"}, + "streamSettings": { + "network": "xhttp", "security": "tls", + "tlsSettings": {"serverName": "xh.example.com", "certificates": [{"certificateFile": "/c/xh.pem", "keyFile": "/c/xh.key"}]}, + "xhttpSettings": {"path": "/xh", "mode": "auto", "xPaddingBytes": "2000-3000"} + } + }, + { + "tag": "xh-extra", "protocol": "vless", "port": 2054, + "settings": {"clients": [], "decryption": "none"}, + "streamSettings": { + "network": "xhttp", "security": "tls", + "tlsSettings": {"serverName": "xh.example.com"}, + "xhttpSettings": {"path": "/xh", "extra": {"xPaddingBytes": "100-200"}} + } + }, + { + "tag": "ws", "protocol": "vless", "port": 2083, + "settings": {"clients": [], "decryption": "none"}, + "streamSettings": {"network": "ws", "security": "tls", "tlsSettings": {"serverName": "ws.example.com"}, "wsSettings": {"path": "/ws"}} + }, + { + "tag": "grpc", "protocol": "vless", "port": 2087, + "settings": {"clients": [], "decryption": "none"}, + "streamSettings": {"network": "grpc", "security": "tls", "tlsSettings": {"serverName": "g.example.com"}, "grpcSettings": {"serviceName": "svc"}} + }, + { + "tag": "hu", "protocol": "vless", "port": 2088, + "settings": {"clients": [], "decryption": "none"}, + "streamSettings": {"network": "httpupgrade", "security": "tls", "tlsSettings": {"serverName": "hu.example.com"}, "httpupgradeSettings": {"path": "/hu"}} + }, + { + "tag": "hy2", "protocol": "hysteria", "port": 8443, + "settings": {"version": 2, "clients": []}, + "streamSettings": { + "network": "hysteria", "security": "tls", + "tlsSettings": {"serverName": "hy.example.com", "alpn": ["h3"]}, + "hysteriaSettings": {"version": 2, "udpIdleTimeout": 60}, + "finalmask": { + "udp": [{"type": "salamander", "settings": {"password": "obfs"}}], + "quicParams": {"congestion": "bbr"} + } + }, + "x-usher": {"hop_ports": "20000-30000", "trust": "pin", "pin_sha256": "AABBCCDDEEFF00112233445566778899AABBCCDDEEFF00112233445566778899"} + }, + { + "tag": "vmess-ws", "protocol": "vmess", "port": 2096, + "settings": {"clients": []}, + "streamSettings": {"network": "ws", "security": "none", "wsSettings": {"path": "/vm"}} + }, + { + "tag": "trojan", "protocol": "trojan", "port": 8444, + "settings": {"clients": [], "fallbacks": [{"dest": 80}]}, + "streamSettings": {"network": "tcp", "security": "tls", "tlsSettings": {"serverName": "t.example.com"}} + } + ] +} From cb50c9985d74d51922fb6cb0a6d68974664d7a2f Mon Sep 17 00:00:00 2001 From: EvaTheSalmon <18466855+EvaTheSalmon@users.noreply.github.com> Date: Wed, 14 Oct 2026 17:58:41 +0300 Subject: [PATCH 09/32] feat(xray): detect drift against the running Xray Co-Authored-By: Claude Sonnet 5.5 --- docs/plans/20261014-xray-config-backend.md | 8 +- internal/xray/drift.go | 207 ++++++++++++++++ internal/xray/drift_test.go | 263 +++++++++++++++++++++ 3 files changed, 474 insertions(+), 4 deletions(-) create mode 100644 internal/xray/drift.go create mode 100644 internal/xray/drift_test.go diff --git a/docs/plans/20261014-xray-config-backend.md b/docs/plans/20261014-xray-config-backend.md index 74a4d0b..bc480fc 100644 --- a/docs/plans/20261014-xray-config-backend.md +++ b/docs/plans/20261014-xray-config-backend.md @@ -377,14 +377,14 @@ reported by awg-grpc" and nothing reaches Xray. **Files:** - Create: `internal/xray/drift.go`, `internal/xray/drift_test.go` -- [ ] compare `Build()` of each managed inbound with one `ListInbounds`: proxy type, proxy settings, +- [x] compare `Build()` of each managed inbound with one `ListInbounds`: proxy type, proxy settings, receiver settings, with file-based certificate entries removed on both sides -- [ ] a mismatch or a tag missing at runtime is a per-tag drift reason naming the differing part +- [x] a mismatch or a tag missing at runtime is a per-tag drift reason naming the differing part only; runtime tags absent from the file are logged each pass -- [ ] write tests with a fake `HandlerServiceClient`: match, port change, `shortIds` change, a +- [x] write tests with a fake `HandlerServiceClient`: match, port change, `shortIds` change, a `privateKey` change (reason without the value), protocol change, a `certificateFile` inbound with changed file contents (no drift), a file tag missing at runtime, an extra runtime tag -- [ ] run `go test -race ./...` - must pass before Task 5 +- [x] run `go test -race ./...` - must pass before Task 5 ### Task 5: Users through HandlerService diff --git a/internal/xray/drift.go b/internal/xray/drift.go new file mode 100644 index 0000000..8637694 --- /dev/null +++ b/internal/xray/drift.go @@ -0,0 +1,207 @@ +package xray + +import ( + "context" + "encoding/json" + "errors" + "fmt" + "log/slog" + "slices" + "strings" + + "github.com/xtls/xray-core/app/proxyman" + "github.com/xtls/xray-core/app/proxyman/command" + "github.com/xtls/xray-core/common/serial" + "github.com/xtls/xray-core/core" + "github.com/xtls/xray-core/infra/conf" + "github.com/xtls/xray-core/transport/internet" + "github.com/xtls/xray-core/transport/internet/tls" + "google.golang.org/protobuf/proto" +) + +// managedProxyTypes are the proxy settings types of the inbounds the backend manages. +var managedProxyTypes = []string{ + "xray.proxy.vless.inbound.Config", + "xray.proxy.hysteria.ServerConfig", + "xray.proxy.vmess.inbound.Config", + "xray.proxy.trojan.ServerConfig", +} + +// checkDrift compares every valid inbound of the file with the inbound of the same tag in the +// running Xray. The result maps a tag to the reason its running inbound differs from the file or +// is missing; a tag without an entry matches. A mismatch means Xray runs an older file, so the +// client files for the tag are not trustworthy until Xray restarts. +// +// Certificate entries that name a file are left out on both sides: usher cannot read the key +// files, and Xray reloads those files itself. A reason names the differing parts and never a +// value. Running inbounds of a managed protocol that the file does not define are logged. +func checkDrift(ctx context.Context, client command.HandlerServiceClient, log *slog.Logger, + inbounds []Inbound) (map[string]error, error) { + resp, err := client.ListInbounds(ctx, &command.ListInboundsRequest{}) + if err != nil { + return nil, fmt.Errorf("listing xray inbounds: %w", err) + } + running := make(map[string]*core.InboundHandlerConfig, len(resp.GetInbounds())) + for _, handler := range resp.GetInbounds() { + running[handler.GetTag()] = handler + } + + drift := map[string]error{} + inFile := make(map[string]bool, len(inbounds)) + for _, in := range inbounds { + inFile[in.Tag] = true + if in.Err != nil { + continue + } + live, ok := running[in.Tag] + if !ok { + drift[in.Tag] = fmt.Errorf("inbound %q: not running in Xray, restart Xray to load the file", in.Tag) + continue + } + parts, err := differingParts(in, live) + switch { + case err != nil: + drift[in.Tag] = fmt.Errorf("inbound %q: comparing with the running Xray: %w", in.Tag, err) + case len(parts) > 0: + drift[in.Tag] = fmt.Errorf("inbound %q: differs from the running Xray in %s, restart Xray to load the file", + in.Tag, strings.Join(parts, ", ")) + } + } + + for tag, handler := range running { + if !inFile[tag] && slices.Contains(managedProxyTypes, handler.GetProxySettings().GetType()) { + log.Warn("xray runs an inbound that the config file does not define", "inbound", tag) + } + } + return drift, nil +} + +// differingParts returns the names of the parts of the file inbound that differ from the running +// one: "proxy type", "proxy settings", "port", "stream settings" and "receiver settings". +func differingParts(in Inbound, live *core.InboundHandlerConfig) ([]string, error) { + raw, err := withoutFileCertificates(in.Raw) + if err != nil { + return nil, err + } + encoded, err := json.Marshal(raw) + if err != nil { + return nil, fmt.Errorf("encoding the file inbound: %w", err) + } + var detour conf.InboundDetourConfig + if err := json.Unmarshal(encoded, &detour); err != nil { + return nil, fmt.Errorf("decoding the file inbound: %w", err) + } + built, err := detour.Build() + if err != nil { + return nil, fmt.Errorf("building the file inbound: %w", err) + } + + var parts []string + if built.GetProxySettings().GetType() != live.GetProxySettings().GetType() { + parts = append(parts, "proxy type") + } else if equal, err := equalMessages(built.GetProxySettings(), live.GetProxySettings()); err != nil { + return nil, err + } else if !equal { + parts = append(parts, "proxy settings") + } + + fileReceiver, err := receiver(built) + if err != nil { + return nil, err + } + liveReceiver, err := receiver(live) + if err != nil { + return nil, err + } + for _, r := range []*proxyman.ReceiverConfig{fileReceiver, liveReceiver} { + if err := dropPathCertificates(r.GetStreamSettings()); err != nil { + return nil, err + } + } + if !proto.Equal(fileReceiver.GetPortList(), liveReceiver.GetPortList()) { + parts = append(parts, "port") + } + if !proto.Equal(fileReceiver.GetStreamSettings(), liveReceiver.GetStreamSettings()) { + parts = append(parts, "stream settings") + } + for _, r := range []*proxyman.ReceiverConfig{fileReceiver, liveReceiver} { + r.PortList = nil + r.StreamSettings = nil + } + if !proto.Equal(fileReceiver, liveReceiver) { + parts = append(parts, "receiver settings") + } + return parts, nil +} + +func equalMessages(a, b *serial.TypedMessage) (bool, error) { + left, err := a.GetInstance() + if err != nil { + return false, fmt.Errorf("decoding proxy settings: %w", err) + } + right, err := b.GetInstance() + if err != nil { + return false, fmt.Errorf("decoding proxy settings: %w", err) + } + return proto.Equal(left, right), nil +} + +// receiver unpacks the receiver settings into a fresh message the caller may modify. +func receiver(handler *core.InboundHandlerConfig) (*proxyman.ReceiverConfig, error) { + instance, err := handler.GetReceiverSettings().GetInstance() + if err != nil { + return nil, fmt.Errorf("decoding receiver settings: %w", err) + } + r, ok := instance.(*proxyman.ReceiverConfig) + if !ok { + return nil, errors.New("receiver settings are not a receiver config") + } + return r, nil +} + +var tlsConfigType = string((&tls.Config{}).ProtoReflect().Descriptor().FullName()) + +// dropPathCertificates removes the certificates that Xray loads from a file from the TLS settings +// of stream. +func dropPathCertificates(stream *internet.StreamConfig) error { + for i, settings := range stream.GetSecuritySettings() { + if settings.GetType() != tlsConfigType { + continue + } + instance, err := settings.GetInstance() + if err != nil { + return fmt.Errorf("decoding tls settings: %w", err) + } + config := instance.(*tls.Config) + config.Certificate = slices.DeleteFunc(config.Certificate, func(c *tls.Certificate) bool { + return c.GetCertificatePath() != "" || c.GetKeyPath() != "" + }) + stream.SecuritySettings[i] = serial.ToTypedMessage(config) + } + return nil +} + +// withoutFileCertificates returns a copy of an inbound object without the tlsSettings certificate +// entries that name a file. +func withoutFileCertificates(raw map[string]any) (map[string]any, error) { + encoded, err := json.Marshal(raw) + if err != nil { + return nil, fmt.Errorf("encoding the file inbound: %w", err) + } + clone, err := decodeObject(encoded) + if err != nil { + return nil, err + } + tlsSettings := object(object(clone["streamSettings"])["tlsSettings"]) + certs, ok := tlsSettings["certificates"].([]any) + if !ok { + return clone, nil + } + tlsSettings["certificates"] = slices.DeleteFunc(certs, func(c any) bool { + entry := object(c) + file, _ := entry["certificateFile"].(string) + key, _ := entry["keyFile"].(string) + return file != "" || key != "" + }) + return clone, nil +} diff --git a/internal/xray/drift_test.go b/internal/xray/drift_test.go new file mode 100644 index 0000000..48e1ca4 --- /dev/null +++ b/internal/xray/drift_test.go @@ -0,0 +1,263 @@ +package xray + +import ( + "bytes" + "context" + "encoding/base64" + "errors" + "fmt" + "log/slog" + "os" + "path/filepath" + "strings" + "testing" + + "github.com/xtls/xray-core/app/proxyman/command" + "github.com/xtls/xray-core/core" + "google.golang.org/grpc" +) + +// fakeHandlerService answers ListInbounds only; any other call panics on the nil embedded client. +type fakeHandlerService struct { + command.HandlerServiceClient + inbounds []*core.InboundHandlerConfig + err error +} + +func (f *fakeHandlerService) ListInbounds(_ context.Context, _ *command.ListInboundsRequest, + _ ...grpc.CallOption) (*command.ListInboundsResponse, error) { + if f.err != nil { + return nil, f.err + } + return &command.ListInboundsResponse{Inbounds: f.inbounds}, nil +} + +// driftConfig holds the parts of the test server config that the cases change. +type driftConfig struct { + dir string + realityPort int + realityKey string + shortID string + realityProto string + certContent string + hysteriaCerts string + extra string // extra inbound objects, each followed by a comma +} + +func newDriftConfig(dir string) driftConfig { + return driftConfig{ + dir: dir, + realityPort: 443, + realityKey: testKey(1), + shortID: "ab12", + realityProto: "vless", + certContent: "first-certificate", + hysteriaCerts: `[{"certificate": ["inline-cert"], "key": ["inline-key"]}]`, + } +} + +// testKey returns a valid Reality private key: 32 bytes of fill in base64url. +func testKey(fill byte) string { + return base64.RawURLEncoding.EncodeToString(bytes.Repeat([]byte{fill}, 32)) +} + +func (c driftConfig) write(t *testing.T) string { + t.Helper() + for name, content := range map[string]string{"ws.pem": c.certContent, "ws.key": "ws-key"} { + if err := os.WriteFile(filepath.Join(c.dir, name), []byte(content), 0o600); err != nil { + t.Fatal(err) + } + } + doc := fmt.Sprintf(`{ + "inbounds": [ + %[8]s + {"tag": "api-in", "protocol": "dokodemo-door", "port": 10085, "settings": {"address": "127.0.0.1"}}, + {"tag": "reality", "protocol": %[1]q, "port": %[2]d, + "settings": {"clients": [], "decryption": "none"}, + "streamSettings": {"network": "raw", "security": "reality", + "realitySettings": {"privateKey": %[3]q, "serverNames": ["www.example.com"], + "shortIds": [%[4]q], "target": "www.example.com:443"}}}, + {"tag": "ws", "protocol": "vless", "port": 2083, + "settings": {"clients": [], "decryption": "none"}, + "streamSettings": {"network": "ws", "security": "tls", + "tlsSettings": {"serverName": "ws.example.com", "certificates": [ + {"certificateFile": %[5]q, "keyFile": %[6]q}]}, + "wsSettings": {"path": "/ws"}}}, + {"tag": "hy2", "protocol": "hysteria", "port": 8443, + "settings": {"version": 2, "clients": []}, + "streamSettings": {"network": "hysteria", "security": "tls", + "tlsSettings": {"serverName": "hy.example.com", "alpn": ["h3"], "certificates": %[7]s}, + "hysteriaSettings": {"version": 2}}} + ] +}`, c.realityProto, c.realityPort, c.realityKey, c.shortID, + filepath.Join(c.dir, "ws.pem"), filepath.Join(c.dir, "ws.key"), c.hysteriaCerts, c.extra) + path := filepath.Join(c.dir, "config.json") + if err := os.WriteFile(path, []byte(doc), 0o600); err != nil { + t.Fatal(err) + } + return path +} + +// runningFrom builds every inbound of the file at path the way Xray does at start. +func runningFrom(t *testing.T, path string) []*core.InboundHandlerConfig { + t.Helper() + inbounds, err := ReadConfig(path) + if err != nil { + t.Fatal(err) + } + var running []*core.InboundHandlerConfig + for _, in := range inbounds { + if in.Err != nil { + t.Fatalf("%s: %v", in.Tag, in.Err) + } + built, err := in.Detour.Build() + if err != nil { + t.Fatalf("%s: %v", in.Tag, err) + } + running = append(running, built) + } + return running +} + +func TestCheckDrift(t *testing.T) { + tests := []struct { + name string + change func(c *driftConfig) + running func(running []*core.InboundHandlerConfig) []*core.InboundHandlerConfig + want map[string][]string // tag to the substrings of its reason + }{ + {name: "match"}, + {name: "port change", change: func(c *driftConfig) { c.realityPort = 8444 }, + want: map[string][]string{"reality": {"port"}}}, + {name: "shortIds change", change: func(c *driftConfig) { c.shortID = "cd34" }, + want: map[string][]string{"reality": {"stream settings"}}}, + {name: "privateKey change", change: func(c *driftConfig) { c.realityKey = testKey(2) }, + want: map[string][]string{"reality": {"stream settings"}}}, + {name: "protocol change", change: func(c *driftConfig) { c.realityProto = "vmess" }, + want: map[string][]string{"reality": {"proxy type"}}}, + {name: "inline certificate change", + change: func(c *driftConfig) { + c.hysteriaCerts = `[{"certificate": ["other-cert"], "key": ["inline-key"]}]` + }, + want: map[string][]string{"hy2": {"stream settings"}}}, + {name: "two changed parts", change: func(c *driftConfig) { + c.realityPort = 8444 + c.shortID = "cd34" + }, want: map[string][]string{"reality": {"port", "stream settings"}}}, + {name: "certificate file contents change", change: func(c *driftConfig) { c.certContent = "second-certificate" }}, + {name: "tag missing at runtime", + running: func(running []*core.InboundHandlerConfig) []*core.InboundHandlerConfig { + var kept []*core.InboundHandlerConfig + for _, r := range running { + if r.Tag != "ws" { + kept = append(kept, r) + } + } + return kept + }, + want: map[string][]string{"ws": {"not running"}}}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + base := newDriftConfig(t.TempDir()) + running := runningFrom(t, base.write(t)) + if tt.running != nil { + running = tt.running(running) + } + changed := base + if tt.change != nil { + tt.change(&changed) + } + inbounds, err := ReadConfig(changed.write(t)) + if err != nil { + t.Fatal(err) + } + + var logs bytes.Buffer + drift, err := checkDrift(context.Background(), &fakeHandlerService{inbounds: running}, + slog.New(slog.NewTextHandler(&logs, nil)), inbounds) + if err != nil { + t.Fatal(err) + } + if len(drift) != len(tt.want) { + t.Errorf("drift = %v, want tags %v", drift, tt.want) + } + for tag, substrings := range tt.want { + reason := drift[tag] + if reason == nil { + t.Errorf("%s: no drift reported", tag) + continue + } + for _, s := range substrings { + if !strings.Contains(reason.Error(), s) { + t.Errorf("%s: reason %q lacks %q", tag, reason, s) + } + } + for _, secret := range []string{base.realityKey, changed.realityKey, changed.shortID, + "inline-key", "other-cert", "ws-key"} { + if strings.Contains(reason.Error(), secret) { + t.Errorf("%s: reason quotes a value: %q", tag, reason) + } + } + } + if logs.Len() != 0 { + t.Errorf("unexpected log: %s", logs.String()) + } + }) + } +} + +func TestCheckDriftExtraRuntimeTag(t *testing.T) { + base := newDriftConfig(t.TempDir()) + stray := base + stray.extra = `{"tag": "stray", "protocol": "vless", "port": 2999, + "settings": {"clients": [], "decryption": "none"}},` + running := runningFrom(t, stray.write(t)) + inbounds, err := ReadConfig(base.write(t)) + if err != nil { + t.Fatal(err) + } + + var logs bytes.Buffer + drift, err := checkDrift(context.Background(), &fakeHandlerService{inbounds: running}, + slog.New(slog.NewTextHandler(&logs, nil)), inbounds) + if err != nil { + t.Fatal(err) + } + if len(drift) != 0 { + t.Errorf("drift = %v, want none", drift) + } + if !strings.Contains(logs.String(), "stray") { + t.Errorf("log does not name the stray inbound: %q", logs.String()) + } + if strings.Contains(logs.String(), "api-in") { + t.Errorf("log names an inbound of an unmanaged protocol: %q", logs.String()) + } +} + +func TestCheckDriftSkipsInvalidInbound(t *testing.T) { + base := newDriftConfig(t.TempDir()) + running := runningFrom(t, base.write(t)) + inbounds, err := ReadConfig(base.write(t)) + if err != nil { + t.Fatal(err) + } + inbounds[0].Err = errors.New("invalid") + + drift, err := checkDrift(context.Background(), &fakeHandlerService{inbounds: running}, + slog.New(slog.NewTextHandler(&bytes.Buffer{}, nil)), inbounds) + if err != nil { + t.Fatal(err) + } + if len(drift) != 0 { + t.Errorf("drift = %v, want none for an inbound that failed validation", drift) + } +} + +func TestCheckDriftListError(t *testing.T) { + _, err := checkDrift(context.Background(), &fakeHandlerService{err: errors.New("unavailable")}, + slog.New(slog.NewTextHandler(&bytes.Buffer{}, nil)), nil) + if err == nil || !strings.Contains(err.Error(), "unavailable") { + t.Errorf("err = %v, want the list error wrapped", err) + } +} From ae429c4055c2d3c96f71f3a1191c25192a6d8e90 Mon Sep 17 00:00:00 2001 From: EvaTheSalmon <18466855+EvaTheSalmon@users.noreply.github.com> Date: Wed, 14 Oct 2026 18:02:20 +0300 Subject: [PATCH 10/32] feat(xray): manage users through HandlerService Co-Authored-By: Claude Sonnet 5.5 --- docs/plans/20261014-xray-config-backend.md | 10 +- internal/xray/enroll.go | 54 +++ internal/xray/enroll_test.go | 80 ++++ internal/xray/protocol.go | 92 +++++ internal/xray/session.go | 205 +++++++++++ internal/xray/session_test.go | 404 +++++++++++++++++++++ 6 files changed, 840 insertions(+), 5 deletions(-) create mode 100644 internal/xray/enroll.go create mode 100644 internal/xray/enroll_test.go create mode 100644 internal/xray/protocol.go create mode 100644 internal/xray/session.go create mode 100644 internal/xray/session_test.go diff --git a/docs/plans/20261014-xray-config-backend.md b/docs/plans/20261014-xray-config-backend.md index bc480fc..511420a 100644 --- a/docs/plans/20261014-xray-config-backend.md +++ b/docs/plans/20261014-xray-config-backend.md @@ -392,17 +392,17 @@ reported by awg-grpc" and nothing reaches Xray. - Create: `internal/xray/protocol.go`, `internal/xray/session.go`, `internal/xray/enroll.go` - Create: matching `_test.go` files -- [ ] protocol table: credential kind, account builder and credential comparison for VLESS (`id`, +- [x] protocol table: credential kind, account builder and credential comparison for VLESS (`id`, flow), Hysteria2 (`auth`), VMess (`id`), Trojan (`password`) -- [ ] enroll: create the credential for every referenced user without one; never change an +- [x] enroll: create the credential for every referenced user without one; never change an existing entry -- [ ] session: `Interfaces` reports every managed tag; `Open` runs the drift check; `Apply` lists +- [x] session: `Interfaces` reports every managed tag; `Open` runs the drift check; `Apply` lists all users per inbound, compares credentials, removes foreign and changed emails, adds missing and changed, logs `users changed`, then returns the tag's drift error if any -- [ ] write tests with a fake service: add, remove, change, foreign email, seeded credential kept, +- [x] write tests with a fake service: add, remove, change, foreign email, seeded credential kept, VMess echo with `SecuritySettings` (no change), removal while the tag is in drift, partial failure converging, no secret in logs -- [ ] run `go test -race ./...` - must pass before Task 6 +- [x] run `go test -race ./...` - must pass before Task 6 ### Task 6: Stale API socket recovery diff --git a/internal/xray/enroll.go b/internal/xray/enroll.go new file mode 100644 index 0000000..fe90448 --- /dev/null +++ b/internal/xray/enroll.go @@ -0,0 +1,54 @@ +package xray + +import ( + "crypto/rand" + "encoding/base64" + "fmt" + "maps" + + "github.com/google/uuid" + + "github.com/mrcsin/usher/internal/state" +) + +// passwordBytes is the entropy of a generated Trojan or Hysteria2 password. +const passwordBytes = 32 + +// enroll makes st hold a credential of the kind the inbound's protocol uses for every user in +// users. It creates the missing ones and never changes an existing entry; an existing entry of the +// wrong kind is an error. It reports whether it created any. On error st is left as it was. +func enroll(st *state.State, in Inbound, users []string) (bool, error) { + spec := protocols[in.Protocol] + created := make(map[string]state.XrayEntry) + for _, user := range users { + name := state.EntryName(in.Tag, user) + if e, ok := st.Xray[name]; ok { + if hasID := e.ID != uuid.Nil; hasID != spec.usesID { + return false, fmt.Errorf("user %s: stored credential does not fit protocol %s", user, in.Protocol) + } + continue + } + e, err := newXrayEntry(spec.usesID) + if err != nil { + return false, fmt.Errorf("user %s: %w", user, err) + } + created[name] = e + } + maps.Copy(st.Xray, created) + return len(created) > 0, nil +} + +func newXrayEntry(usesID bool) (state.XrayEntry, error) { + if usesID { + id, err := uuid.NewRandom() + if err != nil { + return state.XrayEntry{}, fmt.Errorf("generating id: %w", err) + } + return state.XrayEntry{ID: id}, nil + } + buf := make([]byte, passwordBytes) + if _, err := rand.Read(buf); err != nil { + return state.XrayEntry{}, fmt.Errorf("generating password: %w", err) + } + return state.XrayEntry{Password: base64.RawURLEncoding.EncodeToString(buf)}, nil +} diff --git a/internal/xray/enroll_test.go b/internal/xray/enroll_test.go new file mode 100644 index 0000000..7478aff --- /dev/null +++ b/internal/xray/enroll_test.go @@ -0,0 +1,80 @@ +package xray + +import ( + "encoding/base64" + "maps" + "strings" + "testing" + + "github.com/google/uuid" + + "github.com/mrcsin/usher/internal/state" +) + +func TestEnroll(t *testing.T) { + seededID := uuid.New() + tests := []struct { + name string + in Inbound + seed map[string]state.XrayEntry + users []string + wantChanged bool + wantErr string + }{ + {name: "vless gets ids", in: Inbound{Tag: "t", Protocol: "vless"}, users: []string{"alice", "bob"}, wantChanged: true}, + {name: "vmess gets ids", in: Inbound{Tag: "t", Protocol: "vmess"}, users: []string{"alice"}, wantChanged: true}, + {name: "trojan gets passwords", in: Inbound{Tag: "t", Protocol: "trojan"}, users: []string{"alice"}, wantChanged: true}, + {name: "hysteria gets passwords", in: Inbound{Tag: "t", Protocol: "hysteria"}, users: []string{"alice"}, wantChanged: true}, + {name: "seeded entry kept", in: Inbound{Tag: "t", Protocol: "vless"}, + seed: map[string]state.XrayEntry{"t/alice": {ID: seededID}}, users: []string{"alice"}}, + {name: "no users", in: Inbound{Tag: "t", Protocol: "vless"}}, + {name: "entry of the wrong kind", in: Inbound{Tag: "t", Protocol: "hysteria"}, + seed: map[string]state.XrayEntry{"t/alice": {ID: seededID}}, + users: []string{"bob", "alice"}, wantErr: "does not fit protocol hysteria"}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + st := newState() + maps.Copy(st.Xray, tt.seed) + before := maps.Clone(st.Xray) + + changed, err := enroll(st, tt.in, tt.users) + if tt.wantErr != "" { + if err == nil || !strings.Contains(err.Error(), tt.wantErr) { + t.Fatalf("enroll error = %v, want %q", err, tt.wantErr) + } + if strings.Contains(err.Error(), seededID.String()) { + t.Errorf("error quotes the credential: %v", err) + } + if !maps.Equal(st.Xray, before) { + t.Error("state changed on error") + } + return + } + if err != nil { + t.Fatal(err) + } + if changed != tt.wantChanged { + t.Errorf("changed = %v, want %v", changed, tt.wantChanged) + } + for name, e := range tt.seed { + if st.Xray[name] != e { + t.Errorf("seeded entry %s changed", name) + } + } + for _, user := range tt.users { + e := st.Xray["t/"+user] + if protocols[tt.in.Protocol].usesID { + if e.ID == uuid.Nil || e.Password != "" { + t.Errorf("%s: entry is not an id", user) + } + continue + } + raw, err := base64.RawURLEncoding.DecodeString(e.Password) + if err != nil || len(raw) != passwordBytes || e.ID != uuid.Nil { + t.Errorf("%s: entry is not a %d-byte base64url password", user, passwordBytes) + } + } + }) + } +} diff --git a/internal/xray/protocol.go b/internal/xray/protocol.go new file mode 100644 index 0000000..4f56266 --- /dev/null +++ b/internal/xray/protocol.go @@ -0,0 +1,92 @@ +package xray + +import ( + "github.com/xtls/xray-core/common/serial" + "github.com/xtls/xray-core/proxy/hysteria/account" + "github.com/xtls/xray-core/proxy/trojan" + "github.com/xtls/xray-core/proxy/vless" + "github.com/xtls/xray-core/proxy/vmess" + "google.golang.org/protobuf/proto" + + "github.com/mrcsin/usher/internal/state" +) + +// vlessVisionFlow is the VLESS flow of a raw inbound behind TLS or Reality. +const vlessVisionFlow = "xtls-rprx-vision" + +// protocolSpec describes how one inbound protocol holds its users. +type protocolSpec struct { + // usesID is true when the credential is a UUID in XrayEntry.ID, false when it is a password in + // XrayEntry.Password. + usesID bool + // account builds the account Xray stores for the entry on the inbound. + account func(in Inbound, e state.XrayEntry) proto.Message + // credential returns the part of an account that identifies what the client must present, or + // "" for an account of another type. Two accounts with the same credential are the same user. + credential func(account proto.Message) string +} + +// protocols maps the lower case protocol of an inbound to its spec. +var protocols = map[string]protocolSpec{ + "vless": { + usesID: true, + account: func(in Inbound, e state.XrayEntry) proto.Message { + return &vless.Account{Id: e.ID.String(), Flow: vlessFlow(in), Encryption: "none"} + }, + credential: func(a proto.Message) string { + if v, ok := a.(*vless.Account); ok { + return v.GetId() + " " + v.GetFlow() + } + return "" + }, + }, + "vmess": { + usesID: true, + account: func(_ Inbound, e state.XrayEntry) proto.Message { + return &vmess.Account{Id: e.ID.String()} + }, + // Xray adds SecuritySettings to a stored VMess account, so only the id is compared. + credential: func(a proto.Message) string { + if v, ok := a.(*vmess.Account); ok { + return v.GetId() + } + return "" + }, + }, + "trojan": { + account: func(_ Inbound, e state.XrayEntry) proto.Message { + return &trojan.Account{Password: e.Password} + }, + credential: func(a proto.Message) string { + if v, ok := a.(*trojan.Account); ok { + return v.GetPassword() + } + return "" + }, + }, + "hysteria": { + account: func(_ Inbound, e state.XrayEntry) proto.Message { + return &account.Account{Auth: e.Password} + }, + credential: func(a proto.Message) string { + if v, ok := a.(*account.Account); ok { + return v.GetAuth() + } + return "" + }, + }, +} + +// vlessFlow returns the flow of the VLESS users of the inbound: vision for raw behind TLS or +// Reality, none otherwise. +func vlessFlow(in Inbound) string { + if in.Network == "raw" && (in.Security == "tls" || in.Security == "reality") { + return vlessVisionFlow + } + return "" +} + +// typedAccount wraps the account of the entry for an AddUserOperation. +func typedAccount(in Inbound, e state.XrayEntry) *serial.TypedMessage { + return serial.ToTypedMessage(protocols[in.Protocol].account(in, e)) +} diff --git a/internal/xray/session.go b/internal/xray/session.go new file mode 100644 index 0000000..72082af --- /dev/null +++ b/internal/xray/session.go @@ -0,0 +1,205 @@ +package xray + +import ( + "context" + "errors" + "fmt" + "log/slog" + "maps" + "slices" + "time" + + "github.com/xtls/xray-core/app/proxyman/command" + "github.com/xtls/xray-core/common/protocol" + "github.com/xtls/xray-core/common/serial" + "google.golang.org/protobuf/proto" + + "github.com/mrcsin/usher/internal/state" +) + +// callTimeout bounds each call to Xray. +const callTimeout = 10 * time.Second + +// Dial opens a connection to the Xray API. It returns the client and the function that closes the +// connection. +type Dial func() (command.HandlerServiceClient, func(), error) + +// Session is one pass's view of Xray: the managed inbounds of the config file and the drift of +// each against the running Xray. +type Session struct { + client command.HandlerServiceClient + closeConn func() + inbounds []Inbound + byTag map[string]Inbound + drift map[string]error + log *slog.Logger +} + +// Open reads the Xray config at configPath, connects to the API and checks every inbound of the +// file against the running Xray. +func Open(ctx context.Context, dial Dial, configPath string, log *slog.Logger) (*Session, error) { + inbounds, err := ReadConfig(configPath) + if err != nil { + return nil, err + } + client, closeConn, err := dial() + if err != nil { + return nil, fmt.Errorf("connecting to xray: %w", err) + } + callCtx, cancel := context.WithTimeout(ctx, callTimeout) + defer cancel() + drift, err := checkDrift(callCtx, client, log, inbounds) + if err != nil { + closeConn() + return nil, err + } + byTag := make(map[string]Inbound, len(inbounds)) + for _, in := range inbounds { + byTag[in.Tag] = in + } + return &Session{ + client: client, + closeConn: closeConn, + inbounds: inbounds, + byTag: byTag, + drift: drift, + log: log, + }, nil +} + +// Interfaces returns the tag of every managed inbound of the file, valid or not, so a tag that +// fails keeps its client files. +func (s *Session) Interfaces() []string { + tags := make([]string, 0, len(s.inbounds)) + for _, in := range s.inbounds { + tags = append(tags, in.Tag) + } + return tags +} + +// Prepare enrolls the users on the inbound in st. It fails an inbound that did not validate. +func (s *Session) Prepare(st *state.State, name string, users []string) (map[string][]byte, bool, error) { + in := s.byTag[name] + if in.Err != nil { + return nil, false, in.Err + } + changed, err := enroll(st, in, users) + if err != nil { + return nil, false, err + } + return map[string][]byte{}, changed, nil +} + +// Apply makes the inbound hold exactly the users, then returns the drift error of the tag, if +// any. Users are reconciled even under drift, so a revoked user loses access. +func (s *Session) Apply(ctx context.Context, st *state.State, name string, users []string) error { + in := s.byTag[name] + if err := s.reconcile(ctx, st, in, users); err != nil { + return err + } + return s.drift[name] +} + +// Close closes the connection to Xray. +func (s *Session) Close() { + s.closeConn() +} + +// userChanges is what reconcile did to an inbound, by email. +type userChanges struct { + added, removed, updated []string +} + +// reconcile lists the users of the inbound, removes foreign and changed ones, adds missing and +// changed ones, and logs the difference. It tries every operation and joins the errors, so one +// failure does not hold back a removal; the next pass converges from the listed state. +func (s *Session) reconcile(ctx context.Context, st *state.State, in Inbound, users []string) error { + desired := make(map[string]*serial.TypedMessage, len(users)) + credentials := make(map[string]string, len(users)) + spec := protocols[in.Protocol] + for _, user := range users { + e := st.Xray[state.EntryName(in.Tag, user)] + desired[state.EntryName(in.Tag, user)] = typedAccount(in, e) + credentials[state.EntryName(in.Tag, user)] = spec.credential(spec.account(in, e)) + } + + live, err := s.liveCredentials(ctx, in) + if err != nil { + return err + } + + var changes userChanges + var errs []error + present := make(map[string]bool, len(live)) + for _, email := range slices.Sorted(maps.Keys(live)) { + want, wanted := credentials[email] + if wanted && live[email] == want { + present[email] = true + continue + } + if err := s.alter(ctx, in.Tag, &command.RemoveUserOperation{Email: email}); err != nil { + errs = append(errs, fmt.Errorf("removing %s: %w", email, err)) + present[email] = true + continue + } + if wanted { + changes.updated = append(changes.updated, email) + } else { + changes.removed = append(changes.removed, email) + } + } + for _, email := range slices.Sorted(maps.Keys(desired)) { + if present[email] { + continue + } + op := &command.AddUserOperation{User: &protocol.User{Email: email, Account: desired[email]}} + if err := s.alter(ctx, in.Tag, op); err != nil { + errs = append(errs, fmt.Errorf("adding %s: %w", email, err)) + continue + } + if _, listed := live[email]; !listed { + changes.added = append(changes.added, email) + } + } + if len(changes.added)+len(changes.removed)+len(changes.updated) > 0 { + s.log.Info("users changed", "inbound", in.Tag, "added", changes.added, + "removed", changes.removed, "updated", changes.updated) + } + if err := errors.Join(errs...); err != nil { + return fmt.Errorf("applying users: %w", err) + } + return nil +} + +// liveCredentials maps the email of every user of the inbound to the credential of its account. +func (s *Session) liveCredentials(ctx context.Context, in Inbound) (map[string]string, error) { + callCtx, cancel := context.WithTimeout(ctx, callTimeout) + defer cancel() + resp, err := s.client.GetInboundUsers(callCtx, &command.GetInboundUserRequest{Tag: in.Tag}) + if err != nil { + return nil, fmt.Errorf("listing users: %w", err) + } + spec := protocols[in.Protocol] + live := make(map[string]string, len(resp.GetUsers())) + for _, user := range resp.GetUsers() { + if user == nil { + continue + } + var credential string + if instance, err := user.GetAccount().GetInstance(); err == nil { + credential = spec.credential(instance) + } + live[user.GetEmail()] = credential + } + return live, nil +} + +func (s *Session) alter(ctx context.Context, tag string, op proto.Message) error { + callCtx, cancel := context.WithTimeout(ctx, callTimeout) + defer cancel() + _, err := s.client.AlterInbound(callCtx, &command.AlterInboundRequest{ + Tag: tag, + Operation: serial.ToTypedMessage(op), + }) + return err +} diff --git a/internal/xray/session_test.go b/internal/xray/session_test.go new file mode 100644 index 0000000..7220ede --- /dev/null +++ b/internal/xray/session_test.go @@ -0,0 +1,404 @@ +package xray + +import ( + "bytes" + "context" + "errors" + "log/slog" + "maps" + "slices" + "strings" + "testing" + + "github.com/google/uuid" + "github.com/xtls/xray-core/app/proxyman/command" + "github.com/xtls/xray-core/common/protocol" + "github.com/xtls/xray-core/common/serial" + "github.com/xtls/xray-core/core" + "github.com/xtls/xray-core/proxy/vmess" + "google.golang.org/grpc" + + "github.com/mrcsin/usher/internal/state" +) + +// userService is a fake Xray HandlerService that keeps the users of each inbound in memory. +type userService struct { + command.HandlerServiceClient + running []*core.InboundHandlerConfig + users map[string][]*protocol.User // tag to users + failAdd map[string]bool // emails whose AddUser fails + alters int +} + +func newUserService(running []*core.InboundHandlerConfig) *userService { + return &userService{running: running, users: map[string][]*protocol.User{}, failAdd: map[string]bool{}} +} + +func (f *userService) ListInbounds(_ context.Context, _ *command.ListInboundsRequest, + _ ...grpc.CallOption) (*command.ListInboundsResponse, error) { + return &command.ListInboundsResponse{Inbounds: f.running}, nil +} + +func (f *userService) GetInboundUsers(_ context.Context, req *command.GetInboundUserRequest, + _ ...grpc.CallOption) (*command.GetInboundUserResponse, error) { + return &command.GetInboundUserResponse{Users: f.users[req.GetTag()]}, nil +} + +func (f *userService) AlterInbound(_ context.Context, req *command.AlterInboundRequest, + _ ...grpc.CallOption) (*command.AlterInboundResponse, error) { + f.alters++ + op, err := req.GetOperation().GetInstance() + if err != nil { + return nil, err + } + tag := req.GetTag() + switch op := op.(type) { + case *command.AddUserOperation: + email := op.GetUser().GetEmail() + if f.failAdd[email] { + return nil, errors.New("add refused") + } + for _, u := range f.users[tag] { + if u.GetEmail() == email { + return nil, errors.New("user already exists") + } + } + f.users[tag] = append(f.users[tag], op.GetUser()) + case *command.RemoveUserOperation: + before := len(f.users[tag]) + f.users[tag] = slices.DeleteFunc(f.users[tag], func(u *protocol.User) bool { + return u.GetEmail() == op.GetEmail() + }) + if len(f.users[tag]) == before { + return nil, errors.New("user not found") + } + default: + return nil, errors.New("unknown operation") + } + return &command.AlterInboundResponse{}, nil +} + +// credentials maps the emails of the tag to their credentials. +func (f *userService) credentials(t *testing.T, in Inbound) map[string]string { + t.Helper() + out := map[string]string{} + for _, u := range f.users[in.Tag] { + instance, err := u.GetAccount().GetInstance() + if err != nil { + t.Fatal(err) + } + out[u.GetEmail()] = protocols[in.Protocol].credential(instance) + } + return out +} + +func (f *userService) seed(in Inbound, email string, e state.XrayEntry) { + f.users[in.Tag] = append(f.users[in.Tag], &protocol.User{Email: email, Account: typedAccount(in, e)}) +} + +type sessionFixture struct { + session *Session + service *userService + logs *bytes.Buffer + cfg driftConfig +} + +func newSessionFixture(t *testing.T, change func(c *driftConfig)) sessionFixture { + t.Helper() + cfg := newDriftConfig(t.TempDir()) + if change != nil { + change(&cfg) + } + path := cfg.write(t) + service := newUserService(runningValid(t, path)) + logs := &bytes.Buffer{} + log := slog.New(slog.NewTextHandler(logs, nil)) + dial := func() (command.HandlerServiceClient, func(), error) { return service, func() {}, nil } + session, err := Open(t.Context(), dial, path, log) + if err != nil { + t.Fatal(err) + } + return sessionFixture{session: session, service: service, logs: logs, cfg: cfg} +} + +func (f sessionFixture) inbound(tag string) Inbound { + return f.session.byTag[tag] +} + +func newState() *state.State { + return &state.State{Version: 2, AWG: map[string]state.Entry{}, Xray: map[string]state.XrayEntry{}} +} + +func idEntry() state.XrayEntry { return state.XrayEntry{ID: uuid.New()} } + +func TestSessionInterfaces(t *testing.T) { + f := newSessionFixture(t, func(c *driftConfig) { + c.extra = `{"tag": "bad", "protocol": "vless", "port": 9000, "settings": {"clients": [{"id": "x"}]}},` + }) + if got, want := f.session.Interfaces(), []string{"bad", "reality", "ws", "hy2"}; !slices.Equal(got, want) { + t.Errorf("Interfaces() = %v, want %v", got, want) + } + if _, _, err := f.session.Prepare(newState(), "bad", []string{"alice"}); err == nil { + t.Error("Prepare on an invalid inbound succeeded") + } +} + +func TestSessionPrepare(t *testing.T) { + f := newSessionFixture(t, nil) + st := newState() + seeded := idEntry() + st.Xray["reality/bob"] = seeded + + files, changed, err := f.session.Prepare(st, "reality", []string{"alice", "bob"}) + if err != nil { + t.Fatal(err) + } + if !changed || files == nil { + t.Errorf("changed = %v, files = %v, want changed and a non-nil map", changed, files) + } + if st.Xray["reality/alice"].ID == uuid.Nil { + t.Error("alice has no id") + } + if st.Xray["reality/bob"] != seeded { + t.Error("the seeded entry of bob changed") + } + + _, changed, err = f.session.Prepare(st, "reality", []string{"alice", "bob"}) + if err != nil || changed { + t.Errorf("second Prepare = changed %v, err %v, want unchanged", changed, err) + } +} + +func TestSessionApply(t *testing.T) { + tests := []struct { + name string + tag string + setup func(f sessionFixture, st *state.State) + users []string + want func(st *state.State) map[string]string // email to credential + alter int // AlterInbound calls expected + logs []string + }{ + { + name: "adds missing users", + tag: "reality", + users: []string{"alice", "bob"}, + setup: func(_ sessionFixture, st *state.State) { + st.Xray["reality/alice"], st.Xray["reality/bob"] = idEntry(), idEntry() + }, + want: func(st *state.State) map[string]string { + return map[string]string{ + "reality/alice": st.Xray["reality/alice"].ID.String() + " xtls-rprx-vision", + "reality/bob": st.Xray["reality/bob"].ID.String() + " xtls-rprx-vision", + } + }, + alter: 2, + logs: []string{"users changed", "added", "reality/alice"}, + }, + { + name: "no change makes no call", + tag: "ws", + users: []string{"alice"}, + setup: func(f sessionFixture, st *state.State) { + st.Xray["ws/alice"] = idEntry() + f.service.seed(f.inbound("ws"), "ws/alice", st.Xray["ws/alice"]) + }, + want: func(st *state.State) map[string]string { + return map[string]string{"ws/alice": st.Xray["ws/alice"].ID.String() + " "} + }, + }, + { + name: "removes a user who left and a foreign email", + tag: "hy2", + users: []string{"alice"}, + setup: func(f sessionFixture, st *state.State) { + st.Xray["hy2/alice"] = state.XrayEntry{Password: "alice-pass"} + in := f.inbound("hy2") + f.service.seed(in, "hy2/alice", st.Xray["hy2/alice"]) + f.service.seed(in, "hy2/carol", state.XrayEntry{Password: "carol-pass"}) + f.service.seed(in, "stranger", state.XrayEntry{Password: "other-pass"}) + }, + want: func(*state.State) map[string]string { return map[string]string{"hy2/alice": "alice-pass"} }, + alter: 2, + logs: []string{"removed", "hy2/carol", "stranger"}, + }, + { + name: "replaces a changed credential", + tag: "hy2", + users: []string{"alice"}, + setup: func(f sessionFixture, st *state.State) { + st.Xray["hy2/alice"] = state.XrayEntry{Password: "new-pass"} + f.service.seed(f.inbound("hy2"), "hy2/alice", state.XrayEntry{Password: "old-pass"}) + }, + want: func(*state.State) map[string]string { return map[string]string{"hy2/alice": "new-pass"} }, + alter: 2, + logs: []string{"updated", "hy2/alice"}, + }, + { + name: "no users empties the inbound", + tag: "hy2", + users: nil, + setup: func(f sessionFixture, _ *state.State) { + f.service.seed(f.inbound("hy2"), "hy2/alice", state.XrayEntry{Password: "alice-pass"}) + }, + want: func(*state.State) map[string]string { return map[string]string{} }, + alter: 1, + }, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + f := newSessionFixture(t, nil) + st := newState() + tt.setup(f, st) + f.service.alters = 0 + if err := f.session.Apply(t.Context(), st, tt.tag, tt.users); err != nil { + t.Fatal(err) + } + got := f.service.credentials(t, f.inbound(tt.tag)) + if want := tt.want(st); !maps.Equal(got, want) { + t.Errorf("users = %v, want %v", got, want) + } + if f.service.alters != tt.alter { + t.Errorf("AlterInbound calls = %d, want %d", f.service.alters, tt.alter) + } + for _, s := range tt.logs { + if !strings.Contains(f.logs.String(), s) { + t.Errorf("log lacks %q:\n%s", s, f.logs) + } + } + if strings.Contains(f.logs.String(), "-pass") { + t.Errorf("log holds a password:\n%s", f.logs) + } + for _, e := range st.Xray { + if e.ID != uuid.Nil && strings.Contains(f.logs.String(), e.ID.String()) { + t.Errorf("log holds an id:\n%s", f.logs) + } + } + }) + } +} + +func TestSessionApplyUnderDrift(t *testing.T) { + f := newSessionFixture(t, nil) + // The file now says a different port than the running Xray. + f.cfg.realityPort = 8444 + path := f.cfg.write(t) + dial := func() (command.HandlerServiceClient, func(), error) { return f.service, func() {}, nil } + session, err := Open(t.Context(), dial, path, slog.New(slog.DiscardHandler)) + if err != nil { + t.Fatal(err) + } + + in := session.byTag["reality"] + f.service.seed(in, "reality/gone", idEntry()) + f.service.seed(in, "reality/alice", idEntry()) + st := newState() + st.Xray["reality/alice"] = idEntry() + + err = session.Apply(t.Context(), st, "reality", []string{"alice"}) + if err == nil || !strings.Contains(err.Error(), "port") { + t.Fatalf("Apply error = %v, want a drift error naming the port", err) + } + want := map[string]string{"reality/alice": st.Xray["reality/alice"].ID.String() + " xtls-rprx-vision"} + if got := f.service.credentials(t, in); !maps.Equal(got, want) { + t.Errorf("users = %v, want %v", got, want) + } +} + +func TestSessionApplyPartialFailureConverges(t *testing.T) { + f := newSessionFixture(t, nil) + st := newState() + st.Xray["ws/alice"], st.Xray["ws/bob"] = idEntry(), idEntry() + f.service.failAdd["ws/alice"] = true + + err := f.session.Apply(t.Context(), st, "ws", []string{"alice", "bob"}) + if err == nil || !strings.Contains(err.Error(), "ws/alice") { + t.Fatalf("Apply error = %v, want one naming ws/alice", err) + } + if got := f.service.credentials(t, f.inbound("ws")); len(got) != 1 || got["ws/bob"] == "" { + t.Fatalf("after the failure users = %v, want only ws/bob", got) + } + + delete(f.service.failAdd, "ws/alice") + if err := f.session.Apply(t.Context(), st, "ws", []string{"alice", "bob"}); err != nil { + t.Fatal(err) + } + if got := f.service.credentials(t, f.inbound("ws")); len(got) != 2 { + t.Errorf("after the retry users = %v, want both", got) + } +} + +func TestVMessEchoIsNoChange(t *testing.T) { + in := Inbound{Tag: "vm", Protocol: "vmess"} + e := idEntry() + service := newUserService(nil) + // Xray stores a VMess account with SecuritySettings added. + service.users["vm"] = []*protocol.User{{Email: "vm/alice", Account: serial.ToTypedMessage(&vmess.Account{ + Id: e.ID.String(), + SecuritySettings: &protocol.SecurityConfig{Type: protocol.SecurityType_AES128_GCM}, + })}} + session := &Session{client: service, byTag: map[string]Inbound{"vm": in}, log: slog.New(slog.DiscardHandler)} + st := newState() + st.Xray["vm/alice"] = e + + if err := session.Apply(t.Context(), st, "vm", []string{"alice"}); err != nil { + t.Fatal(err) + } + if service.alters != 0 { + t.Errorf("AlterInbound calls = %d, want 0", service.alters) + } +} + +func TestProtocolCredential(t *testing.T) { + id := uuid.New() + tests := []struct { + name string + in Inbound + entry state.XrayEntry + want string + wrongFor string // protocol whose credential function must return "" for this account + }{ + {"vless vision", Inbound{Protocol: "vless", Network: "raw", Security: "reality"}, + state.XrayEntry{ID: id}, id.String() + " xtls-rprx-vision", "trojan"}, + {"vless ws", Inbound{Protocol: "vless", Network: "ws", Security: "tls"}, + state.XrayEntry{ID: id}, id.String() + " ", "vmess"}, + {"vless raw without security", Inbound{Protocol: "vless", Network: "raw", Security: "none"}, + state.XrayEntry{ID: id}, id.String() + " ", "hysteria"}, + {"vmess", Inbound{Protocol: "vmess"}, state.XrayEntry{ID: id}, id.String(), "vless"}, + {"trojan", Inbound{Protocol: "trojan"}, state.XrayEntry{Password: "pw"}, "pw", "hysteria"}, + {"hysteria", Inbound{Protocol: "hysteria"}, state.XrayEntry{Password: "pw"}, "pw", "trojan"}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + spec := protocols[tt.in.Protocol] + account := spec.account(tt.in, tt.entry) + if got := spec.credential(account); got != tt.want { + t.Errorf("credential = %q, want %q", got, tt.want) + } + if got := protocols[tt.wrongFor].credential(account); got != "" { + t.Errorf("%s credential of a %s account = %q, want empty", tt.wrongFor, tt.in.Protocol, got) + } + }) + } +} + +// runningValid builds the valid inbounds of the file the way Xray does at start. +func runningValid(t *testing.T, path string) []*core.InboundHandlerConfig { + t.Helper() + inbounds, err := ReadConfig(path) + if err != nil { + t.Fatal(err) + } + var running []*core.InboundHandlerConfig + for _, in := range inbounds { + if in.Err != nil { + continue + } + built, err := in.Detour.Build() + if err != nil { + t.Fatal(err) + } + running = append(running, built) + } + return running +} From 060a4448d02a620f88dea9526c6043b1e8fc2a82 Mon Sep 17 00:00:00 2001 From: EvaTheSalmon <18466855+EvaTheSalmon@users.noreply.github.com> Date: Wed, 14 Oct 2026 18:04:17 +0300 Subject: [PATCH 11/32] feat(xray): recover a stale API socket Co-Authored-By: Claude Sonnet 5.5 --- docs/plans/20261014-xray-config-backend.md | 6 +- internal/xray/session.go | 13 +++- internal/xray/session_test.go | 26 +++++++- internal/xray/socket.go | 36 +++++++++++ internal/xray/socket_test.go | 70 ++++++++++++++++++++++ 5 files changed, 144 insertions(+), 7 deletions(-) create mode 100644 internal/xray/socket.go create mode 100644 internal/xray/socket_test.go diff --git a/docs/plans/20261014-xray-config-backend.md b/docs/plans/20261014-xray-config-backend.md index 511420a..7df2b52 100644 --- a/docs/plans/20261014-xray-config-backend.md +++ b/docs/plans/20261014-xray-config-backend.md @@ -409,11 +409,11 @@ reported by awg-grpc" and nothing reaches Xray. **Files:** - Create: `internal/xray/socket.go`, `internal/xray/socket_test.go` -- [ ] before dialing: if the socket path exists and `flock(LOCK_EX|LOCK_NB)` on `.lock` +- [x] before dialing: if the socket path exists and `flock(LOCK_EX|LOCK_NB)` on `.lock` succeeds, unlink the socket, release the lock and report Xray down for this pass -- [ ] write tests in `t.TempDir()`: a stale socket (listener with `SetUnlinkOnClose(false)`) is +- [x] write tests in `t.TempDir()`: a stale socket (listener with `SetUnlinkOnClose(false)`) is removed; a socket whose lock another descriptor holds is kept; a missing path is left alone -- [ ] run `go test -race ./...` - must pass before Task 7 +- [x] run `go test -race ./...` - must pass before Task 7 ### Task 7: Client mirror diff --git a/internal/xray/session.go b/internal/xray/session.go index 72082af..39f9291 100644 --- a/internal/xray/session.go +++ b/internal/xray/session.go @@ -36,12 +36,21 @@ type Session struct { } // Open reads the Xray config at configPath, connects to the API and checks every inbound of the -// file against the running Xray. -func Open(ctx context.Context, dial Dial, configPath string, log *slog.Logger) (*Session, error) { +// file against the running Xray. When socketPath is a stale socket of a killed Xray, Open removes +// it and fails, so the pass reports Xray down and the restart policy can start Xray again. +func Open(ctx context.Context, dial Dial, socketPath, configPath string, log *slog.Logger) (*Session, error) { inbounds, err := ReadConfig(configPath) if err != nil { return nil, err } + removed, err := removeStaleSocket(socketPath) + if err != nil { + return nil, err + } + if removed { + log.Warn("removed stale xray socket", "path", socketPath) + return nil, errors.New("xray is down: removed its stale api socket") + } client, closeConn, err := dial() if err != nil { return nil, fmt.Errorf("connecting to xray: %w", err) diff --git a/internal/xray/session_test.go b/internal/xray/session_test.go index 7220ede..a2af8d7 100644 --- a/internal/xray/session_test.go +++ b/internal/xray/session_test.go @@ -6,6 +6,8 @@ import ( "errors" "log/slog" "maps" + "os" + "path/filepath" "slices" "strings" "testing" @@ -114,7 +116,7 @@ func newSessionFixture(t *testing.T, change func(c *driftConfig)) sessionFixture logs := &bytes.Buffer{} log := slog.New(slog.NewTextHandler(logs, nil)) dial := func() (command.HandlerServiceClient, func(), error) { return service, func() {}, nil } - session, err := Open(t.Context(), dial, path, log) + session, err := Open(t.Context(), dial, filepath.Join(t.TempDir(), "xray.sock"), path, log) if err != nil { t.Fatal(err) } @@ -284,7 +286,7 @@ func TestSessionApplyUnderDrift(t *testing.T) { f.cfg.realityPort = 8444 path := f.cfg.write(t) dial := func() (command.HandlerServiceClient, func(), error) { return f.service, func() {}, nil } - session, err := Open(t.Context(), dial, path, slog.New(slog.DiscardHandler)) + session, err := Open(t.Context(), dial, filepath.Join(t.TempDir(), "xray.sock"), path, slog.New(slog.DiscardHandler)) if err != nil { t.Fatal(err) } @@ -402,3 +404,23 @@ func runningValid(t *testing.T, path string) []*core.InboundHandlerConfig { } return running } + +func TestOpenStaleSocketReportsDown(t *testing.T) { + cfg := newDriftConfig(t.TempDir()) + path := cfg.write(t) + socket := staleSocket(t) + dialed := false + dial := func() (command.HandlerServiceClient, func(), error) { + dialed = true + return nil, func() {}, nil + } + if _, err := Open(t.Context(), dial, socket, path, slog.New(slog.DiscardHandler)); err == nil { + t.Fatal("want an error for a stale socket") + } + if dialed { + t.Fatal("dialed a stale socket") + } + if _, err := os.Lstat(socket); !os.IsNotExist(err) { + t.Fatalf("socket still there: %v", err) + } +} diff --git a/internal/xray/socket.go b/internal/xray/socket.go new file mode 100644 index 0000000..5eaa4a7 --- /dev/null +++ b/internal/xray/socket.go @@ -0,0 +1,36 @@ +package xray + +import ( + "errors" + "fmt" + "io/fs" + "os" + "syscall" +) + +// removeStaleSocket unlinks the API socket when no Xray holds its lock. Xray holds an exclusive +// flock on .lock while it listens, so a lock that is free means a killed Xray left the +// socket behind and the next start would fail on it. It reports whether it removed the socket. +func removeStaleSocket(socketPath string) (bool, error) { + if _, err := os.Lstat(socketPath); errors.Is(err, fs.ErrNotExist) { + return false, nil + } else if err != nil { + return false, fmt.Errorf("checking xray socket: %w", err) + } + lock, err := os.OpenFile(socketPath+".lock", os.O_RDWR|os.O_CREATE, 0o600) + if err != nil { + return false, fmt.Errorf("opening xray socket lock: %w", err) + } + defer lock.Close() + if err := syscall.Flock(int(lock.Fd()), syscall.LOCK_EX|syscall.LOCK_NB); err != nil { + if errors.Is(err, syscall.EWOULDBLOCK) { + return false, nil + } + return false, fmt.Errorf("locking xray socket: %w", err) + } + defer syscall.Flock(int(lock.Fd()), syscall.LOCK_UN) //nolint:errcheck // closing the file releases it + if err := os.Remove(socketPath); err != nil && !errors.Is(err, fs.ErrNotExist) { + return false, fmt.Errorf("removing stale xray socket: %w", err) + } + return true, nil +} diff --git a/internal/xray/socket_test.go b/internal/xray/socket_test.go new file mode 100644 index 0000000..434c798 --- /dev/null +++ b/internal/xray/socket_test.go @@ -0,0 +1,70 @@ +package xray + +import ( + "net" + "os" + "path/filepath" + "syscall" + "testing" +) + +func staleSocket(t *testing.T) string { + t.Helper() + // t.TempDir paths exceed the unix socket path limit on macOS. + dir, err := os.MkdirTemp("", "usher") + if err != nil { + t.Fatal(err) + } + t.Cleanup(func() { os.RemoveAll(dir) }) + path := filepath.Join(dir, "x.sock") + l, err := net.Listen("unix", path) + if err != nil { + t.Fatal(err) + } + l.(*net.UnixListener).SetUnlinkOnClose(false) + if err := l.Close(); err != nil { + t.Fatal(err) + } + return path +} + +func TestRemoveStaleSocket(t *testing.T) { + t.Run("free lock removes the socket", func(t *testing.T) { + path := staleSocket(t) + removed, err := removeStaleSocket(path) + if err != nil || !removed { + t.Fatalf("removed=%v err=%v", removed, err) + } + if _, err := os.Lstat(path); !os.IsNotExist(err) { + t.Fatalf("socket still there: %v", err) + } + }) + t.Run("held lock keeps the socket", func(t *testing.T) { + path := staleSocket(t) + lock, err := os.OpenFile(path+".lock", os.O_RDWR|os.O_CREATE, 0o600) + if err != nil { + t.Fatal(err) + } + defer lock.Close() + if err := syscall.Flock(int(lock.Fd()), syscall.LOCK_EX); err != nil { + t.Fatal(err) + } + removed, err := removeStaleSocket(path) + if err != nil || removed { + t.Fatalf("removed=%v err=%v", removed, err) + } + if _, err := os.Lstat(path); err != nil { + t.Fatalf("socket gone: %v", err) + } + }) + t.Run("missing path is left alone", func(t *testing.T) { + path := filepath.Join(t.TempDir(), "xray.sock") + removed, err := removeStaleSocket(path) + if err != nil || removed { + t.Fatalf("removed=%v err=%v", removed, err) + } + if _, err := os.Lstat(path + ".lock"); !os.IsNotExist(err) { + t.Fatalf("lock file created: %v", err) + } + }) +} From f02b182ba91e042cf44a2fc368552c86f3f7a34a Mon Sep 17 00:00:00 2001 From: EvaTheSalmon <18466855+EvaTheSalmon@users.noreply.github.com> Date: Wed, 14 Oct 2026 18:07:53 +0300 Subject: [PATCH 12/32] feat(xray): mirror inbounds into client outbounds Co-Authored-By: Claude Sonnet 5.5 --- docs/plans/20261014-xray-config-backend.md | 14 +- go.mod | 4 +- internal/xray/mirror.go | 262 ++++++++++ internal/xray/mirror_test.go | 568 +++++++++++++++++++++ 4 files changed, 839 insertions(+), 9 deletions(-) create mode 100644 internal/xray/mirror.go create mode 100644 internal/xray/mirror_test.go diff --git a/docs/plans/20261014-xray-config-backend.md b/docs/plans/20261014-xray-config-backend.md index 7df2b52..907140f 100644 --- a/docs/plans/20261014-xray-config-backend.md +++ b/docs/plans/20261014-xray-config-backend.md @@ -420,16 +420,16 @@ reported by awg-grpc" and nothing reaches Xray. **Files:** - Create: `internal/xray/mirror.go`, `internal/xray/mirror_test.go` -- [ ] transport fields as in Solution Overview, xhttp `extra` packing or verbatim server `extra`, +- [x] transport fields as in Solution Overview, xhttp `extra` packing or verbatim server `extra`, the VLESS flow rule -- [ ] the TLS trust rule: `ca` default, `pin` from `pin_sha256` or the inline certificate -- [ ] Reality rewrite including the `serverNames` rule and `mldsa65Verify` from `mldsa65Seed` -- [ ] Hysteria2: client auth, salamander password, the udphop mask from `hop_ports` in libxray's +- [x] the TLS trust rule: `ca` default, `pin` from `pin_sha256` or the inline certificate +- [x] Reality rewrite including the `serverNames` rule and `mldsa65Verify` from `mldsa65Seed` +- [x] Hysteria2: client auth, salamander password, the udphop mask from `hop_ports` in libxray's shape -- [ ] `x-usher` overrides -- [ ] write the leak test (Testing Strategy) and tests for each trust case, hop, flow, xhttp +- [x] `x-usher` overrides +- [x] write the leak test (Testing Strategy) and tests for each trust case, hop, flow, xhttp packing, server `extra` and override -- [ ] run `go test -race ./...` - must pass before Task 8 +- [x] run `go test -race ./...` - must pass before Task 8 ### Task 8: Render client files and golden tests diff --git a/go.mod b/go.mod index efa0bb6..34764da 100644 --- a/go.mod +++ b/go.mod @@ -3,17 +3,18 @@ module github.com/mrcsin/usher go 1.27.1 require ( + github.com/cloudflare/circl v1.6.5 github.com/google/uuid v1.6.0 github.com/mrcsin/awg-grpc v0.1.0 github.com/xtls/xray-core v1.260327.1-0.20261010092107-701af60772cd google.golang.org/grpc v1.84.0 + google.golang.org/protobuf v1.36.12 gopkg.in/yaml.v3 v3.0.1 ) require ( github.com/andybalholm/brotli v1.2.6 // indirect github.com/apernet/quic-go v0.61.1-0.20260806010916-184d081eef3e // indirect - github.com/cloudflare/circl v1.6.5 // indirect github.com/google/btree v1.1.3 // indirect github.com/gorilla/websocket v1.5.3 // indirect github.com/huin/goupnp v1.3.0 // indirect @@ -53,7 +54,6 @@ require ( golang.zx2c4.com/wireguard v0.0.0-20261006164505-2631ce99a06f // indirect golang.zx2c4.com/wireguard/windows v1.1.1 // indirect google.golang.org/genproto/googleapis/rpc v0.0.0-20261005182115-fad411399dd8 // indirect - google.golang.org/protobuf v1.36.12 // indirect gvisor.dev/gvisor v0.0.0-20261010064756-cd110d9dec04 // indirect layeh.com/gopher-luar v1.0.11 // indirect lukechampine.com/blake3 v1.4.1 // indirect diff --git a/internal/xray/mirror.go b/internal/xray/mirror.go new file mode 100644 index 0000000..85b93d7 --- /dev/null +++ b/internal/xray/mirror.go @@ -0,0 +1,262 @@ +package xray + +import ( + "crypto/ecdh" + "crypto/sha256" + "encoding/base64" + "encoding/hex" + "encoding/pem" + "errors" + "fmt" + "strings" + + "github.com/cloudflare/circl/sign/mldsa/mldsa65" + + "github.com/mrcsin/usher/internal/state" +) + +const ( + // hopInterval is the udphop interval in seconds; libxray omits it from a link at this value. + hopInterval = 30 + hopMode = "intervalLocal,intervalRemote" +) + +// clientOutbound returns the Xray client outbound for one user on the inbound. It copies only what +// the client needs from the server config and rebuilds the TLS and Reality blocks, so nothing +// secret or server-only reaches the result. host is the address clients dial unless x-usher sets +// one. The xhttp tuning is packed into extra, the one form both the .json and the share link carry. +func clientOutbound(in Inbound, host string, entry state.XrayEntry) (map[string]any, error) { + stream, err := clientStream(in, entry) + if err != nil { + return nil, err + } + if in.Hints.Address != "" { + host = in.Hints.Address + } + endpoint := map[string]any{"address": host, "port": in.Hints.Port} + + var settings map[string]any + switch in.Protocol { + case "vless": + settings = map[string]any{"id": entry.ID.String(), "encryption": "none", "flow": vlessFlow(in)} + case "vmess": + settings = map[string]any{"id": entry.ID.String(), "security": "auto"} + case "trojan": + settings = map[string]any{"password": entry.Password} + case "hysteria": + settings = map[string]any{"version": 2} + default: + return nil, fmt.Errorf("inbound %q: protocol: unsupported", in.Tag) + } + for k, v := range endpoint { + settings[k] = v + } + return map[string]any{ + "tag": in.Tag, + "protocol": in.Protocol, + "settings": settings, + "streamSettings": stream, + }, nil +} + +func clientStream(in Inbound, entry state.XrayEntry) (map[string]any, error) { + server := object(in.Raw["streamSettings"]) + stream := map[string]any{"network": in.Network, "security": in.Security} + + switch in.Network { + case "ws": + stream["wsSettings"] = pick(object(server["wsSettings"]), "path", "host") + case "grpc": + stream["grpcSettings"] = pick(object(server["grpcSettings"]), "serviceName", "authority") + case "httpupgrade": + stream["httpupgradeSettings"] = pick(object(server["httpupgradeSettings"]), "path", "host") + case "xhttp": + stream["xhttpSettings"] = xhttpSettings(object(server["xhttpSettings"])) + case "hysteria": + stream["hysteriaSettings"] = map[string]any{"version": 2, "auth": entry.Password} + if mask := hysteriaMask(in, server); mask != nil { + stream["finalmask"] = mask + } + } + + switch in.Security { + case "tls": + tls, err := clientTLS(in, object(server["tlsSettings"])) + if err != nil { + return nil, err + } + stream["tlsSettings"] = tls + case "reality": + reality, err := clientReality(in, object(server["realitySettings"])) + if err != nil { + return nil, err + } + stream["realitySettings"] = reality + } + return stream, nil +} + +// pick returns a copy of the listed keys of m that are present. +func pick(m map[string]any, keys ...string) map[string]any { + out := map[string]any{} + for _, k := range keys { + if v, ok := m[k]; ok { + out[k] = deepCopy(v) + } + } + return out +} + +// xhttpSettings keeps path, host and mode at the top. A server extra replaces every other field on +// the server, so it is mirrored verbatim; without one the other fields move into extra. +func xhttpSettings(server map[string]any) map[string]any { + out := pick(server, "path", "host", "mode") + if extra := object(server["extra"]); len(extra) > 0 { + out["extra"] = deepCopy(extra) + return out + } + extra := map[string]any{} + for k, v := range server { + if _, top := out[k]; !top && k != "extra" { + extra[k] = deepCopy(v) + } + } + if len(extra) > 0 { + out["extra"] = extra + } + return out +} + +// hysteriaMask returns the client finalmask: the salamander password of the server, then a udphop +// entry when x-usher.hop_ports is set. libxray reads a link only in that order. +func hysteriaMask(in Inbound, server map[string]any) map[string]any { + var udp []any + serverUDP, _ := object(server["finalmask"])["udp"].([]any) + for _, item := range serverUDP { + entry := object(item) + if lowerString(entry["type"]) == "salamander" { + udp = append(udp, map[string]any{ + "type": "salamander", + "settings": pick(object(entry["settings"]), "password"), + }) + } + } + if in.Hints.HopPorts != "" { + udp = append(udp, map[string]any{ + "type": "udphop", + "settings": map[string]any{ + "mode": hopMode, + "interval": hopInterval, + "remotePorts": in.Hints.HopPorts, + }, + }) + } + if len(udp) == 0 { + return nil + } + return map[string]any{"udp": udp} +} + +// clientTLS rebuilds the tlsSettings block: the name, the ALPN list, the fingerprint and, for +// trust "pin", the pinned leaf hash. +func clientTLS(in Inbound, server map[string]any) (map[string]any, error) { + tls := map[string]any{"serverName": in.ServerName, "fingerprint": in.Hints.Fingerprint} + if alpn, ok := server["alpn"]; ok { + tls["alpn"] = deepCopy(alpn) + } + if in.Hints.Trust != TrustPin { + return tls, nil + } + pin := in.Hints.PinSHA256 + if pin == "" { + var err error + if pin, err = inlineCertificatePin(server); err != nil { + return nil, fmt.Errorf("inbound %q: x-usher.pin_sha256: %w", in.Tag, err) + } + } + tls["pinnedPeerCertSha256"] = pin + return tls, nil +} + +// inlineCertificatePin returns the lower case hex SHA-256 of the first certificate that an inline +// certificates entry carries. +func inlineCertificatePin(tls map[string]any) (string, error) { + certs, _ := tls["certificates"].([]any) + for _, c := range certs { + pemLines, _ := object(c)["certificate"].([]any) + if len(pemLines) == 0 { + continue + } + lines := make([]string, len(pemLines)) + for i, line := range pemLines { + lines[i], _ = line.(string) + } + block, _ := pem.Decode([]byte(strings.Join(lines, "\n"))) + if block == nil || block.Type != "CERTIFICATE" { + return "", errors.New("the inline certificate is not a PEM certificate") + } + sum := sha256.Sum256(block.Bytes) + return hex.EncodeToString(sum[:]), nil + } + return "", errors.New("no inline certificate") +} + +// clientReality rebuilds the realitySettings block from the server's: the public key of its +// privateKey, its first short id and, with an mldsa65Seed, the matching verify key. +func clientReality(in Inbound, server map[string]any) (map[string]any, error) { + fail := func(field, reason string) error { + return fmt.Errorf("inbound %q: streamSettings.realitySettings.%s: %s", in.Tag, field, reason) + } + + private, _ := server["privateKey"].(string) + raw, err := base64.RawURLEncoding.DecodeString(private) + if err != nil { + return nil, fail("privateKey", "want base64url") + } + key, err := ecdh.X25519().NewPrivateKey(raw) + if err != nil { + return nil, fail("privateKey", "want a 32 byte X25519 key") + } + + shortID := "" + if ids, _ := server["shortIds"].([]any); len(ids) > 0 { + shortID, _ = ids[0].(string) + } + reality := map[string]any{ + "serverName": in.ServerName, + "publicKey": base64.RawURLEncoding.EncodeToString(key.PublicKey().Bytes()), + "shortId": shortID, + "fingerprint": in.Hints.Fingerprint, + "spiderX": "/", + } + + if seed, _ := server["mldsa65Seed"].(string); seed != "" { + decoded, err := base64.RawURLEncoding.DecodeString(seed) + if err != nil || len(decoded) != mldsa65.SeedSize { + return nil, fail("mldsa65Seed", "want 32 bytes in base64url") + } + public, _ := mldsa65.NewKeyFromSeed((*[mldsa65.SeedSize]byte)(decoded)) + reality["mldsa65Verify"] = base64.RawURLEncoding.EncodeToString(public.Bytes()) + } + return reality, nil +} + +// deepCopy copies the maps and lists of a decoded JSON value, so a result never aliases Inbound.Raw. +func deepCopy(v any) any { + switch v := v.(type) { + case map[string]any: + out := make(map[string]any, len(v)) + for k, item := range v { + out[k] = deepCopy(item) + } + return out + case []any: + out := make([]any, len(v)) + for i, item := range v { + out[i] = deepCopy(item) + } + return out + default: + return v + } +} diff --git a/internal/xray/mirror_test.go b/internal/xray/mirror_test.go new file mode 100644 index 0000000..d787d7c --- /dev/null +++ b/internal/xray/mirror_test.go @@ -0,0 +1,568 @@ +package xray + +import ( + "crypto/ecdh" + "crypto/ecdsa" + "crypto/elliptic" + "crypto/rand" + "crypto/sha256" + "crypto/x509" + "crypto/x509/pkix" + "encoding/base64" + "encoding/hex" + "encoding/json" + "encoding/pem" + "math/big" + "reflect" + "strings" + "testing" + + "github.com/cloudflare/circl/sign/mldsa/mldsa65" + "github.com/google/uuid" + "github.com/xtls/xray-core/infra/conf" + + "github.com/mrcsin/usher/internal/state" +) + +const ( + testPrivateKey = "cEUEZw0y-gOyW9gPA93c1bt6TrCFqbsXu4o9dTC0Zt0" + testPassword = "user-password" +) + +var testID = uuid.MustParse("3f2504e0-4f89-41d3-9a0c-0305e82c3301") + +func testEntry(in Inbound) state.XrayEntry { + if protocols[in.Protocol].usesID { + return state.XrayEntry{ID: testID} + } + return state.XrayEntry{Password: testPassword} +} + +func mirrorOne(t *testing.T, in Inbound) map[string]any { + t.Helper() + if in.Err != nil { + t.Fatalf("inbound %q: %v", in.Tag, in.Err) + } + out, err := clientOutbound(in, "198.51.100.7", testEntry(in)) + if err != nil { + t.Fatalf("clientOutbound(%q): %v", in.Tag, err) + } + return out +} + +// wantJSON compares got with the JSON document want. +func wantJSON(t *testing.T, got map[string]any, want string) { + t.Helper() + encoded, err := json.Marshal(got) + if err != nil { + t.Fatal(err) + } + var gotAny, wantAny any + if err := json.Unmarshal(encoded, &gotAny); err != nil { + t.Fatal(err) + } + if err := json.Unmarshal([]byte(want), &wantAny); err != nil { + t.Fatalf("bad want: %v", err) + } + if !reflect.DeepEqual(gotAny, wantAny) { + pretty, _ := json.MarshalIndent(gotAny, "", " ") + t.Errorf("outbound mismatch, got:\n%s", pretty) + } +} + +func byTag(t *testing.T, inbounds []Inbound) map[string]Inbound { + t.Helper() + m := map[string]Inbound{} + for _, in := range inbounds { + m[in.Tag] = in + } + return m +} + +func TestClientOutboundShapes(t *testing.T) { + inbounds, err := ReadConfig("testdata/server.json") + if err != nil { + t.Fatal(err) + } + tags := byTag(t, inbounds) + + tests := []struct { + tag string + want string + }{ + {"vless-reality", `{"tag":"vless-reality","protocol":"vless", + "settings":{"address":"203.0.113.10","port":443,"id":"` + testID.String() + `", + "encryption":"none","flow":"xtls-rprx-vision"}, + "streamSettings":{"network":"raw","security":"reality","realitySettings":{ + "serverName":"www.example.com","publicKey":"` + publicKeyOf(t, testPrivateKey) + `", + "shortId":"ab12","fingerprint":"chrome","spiderX":"/"}}}`}, + {"xh", `{"tag":"xh","protocol":"vless", + "settings":{"address":"198.51.100.7","port":2053,"id":"` + testID.String() + `", + "encryption":"none","flow":""}, + "streamSettings":{"network":"xhttp","security":"tls", + "tlsSettings":{"serverName":"xh.example.com","fingerprint":"chrome"}, + "xhttpSettings":{"path":"/xh","mode":"auto","extra":{"xPaddingBytes":"2000-3000"}}}}`}, + {"xh-extra", `{"tag":"xh-extra","protocol":"vless", + "settings":{"address":"198.51.100.7","port":2054,"id":"` + testID.String() + `", + "encryption":"none","flow":""}, + "streamSettings":{"network":"xhttp","security":"tls", + "tlsSettings":{"serverName":"xh.example.com","fingerprint":"chrome"}, + "xhttpSettings":{"path":"/xh","extra":{"xPaddingBytes":"100-200"}}}}`}, + {"ws", `{"tag":"ws","protocol":"vless", + "settings":{"address":"198.51.100.7","port":2083,"id":"` + testID.String() + `", + "encryption":"none","flow":""}, + "streamSettings":{"network":"ws","security":"tls", + "tlsSettings":{"serverName":"ws.example.com","fingerprint":"chrome"}, + "wsSettings":{"path":"/ws"}}}`}, + {"grpc", `{"tag":"grpc","protocol":"vless", + "settings":{"address":"198.51.100.7","port":2087,"id":"` + testID.String() + `", + "encryption":"none","flow":""}, + "streamSettings":{"network":"grpc","security":"tls", + "tlsSettings":{"serverName":"g.example.com","fingerprint":"chrome"}, + "grpcSettings":{"serviceName":"svc"}}}`}, + {"hu", `{"tag":"hu","protocol":"vless", + "settings":{"address":"198.51.100.7","port":2088,"id":"` + testID.String() + `", + "encryption":"none","flow":""}, + "streamSettings":{"network":"httpupgrade","security":"tls", + "tlsSettings":{"serverName":"hu.example.com","fingerprint":"chrome"}, + "httpupgradeSettings":{"path":"/hu"}}}`}, + {"hy2", `{"tag":"hy2","protocol":"hysteria", + "settings":{"address":"198.51.100.7","port":8443,"version":2}, + "streamSettings":{"network":"hysteria","security":"tls", + "tlsSettings":{"serverName":"hy.example.com","alpn":["h3"],"fingerprint":"chrome", + "pinnedPeerCertSha256":"aabbccddeeff00112233445566778899aabbccddeeff00112233445566778899"}, + "hysteriaSettings":{"version":2,"auth":"` + testPassword + `"}, + "finalmask":{"udp":[ + {"type":"salamander","settings":{"password":"obfs"}}, + {"type":"udphop","settings":{"mode":"intervalLocal,intervalRemote", + "interval":30,"remotePorts":"20000-30000"}}]}}}`}, + {"vmess-ws", `{"tag":"vmess-ws","protocol":"vmess", + "settings":{"address":"198.51.100.7","port":2096,"id":"` + testID.String() + `","security":"auto"}, + "streamSettings":{"network":"ws","security":"none","wsSettings":{"path":"/vm"}}}`}, + {"trojan", `{"tag":"trojan","protocol":"trojan", + "settings":{"address":"198.51.100.7","port":8444,"password":"` + testPassword + `"}, + "streamSettings":{"network":"raw","security":"tls", + "tlsSettings":{"serverName":"t.example.com","fingerprint":"chrome"}}}`}, + } + for _, tt := range tests { + t.Run(tt.tag, func(t *testing.T) { + in, ok := tags[tt.tag] + if !ok { + t.Fatalf("testdata has no inbound %q", tt.tag) + } + out := mirrorOne(t, in) + wantJSON(t, out, tt.want) + assertBuilds(t, out) + }) + } +} + +// assertBuilds checks that Xray accepts the outbound as a client config. +func assertBuilds(t *testing.T, outbound map[string]any) { + t.Helper() + encoded, err := json.Marshal(outbound) + if err != nil { + t.Fatal(err) + } + var detour conf.OutboundDetourConfig + if err := json.Unmarshal(encoded, &detour); err != nil { + t.Fatalf("outbound does not decode: %v", err) + } + if _, err := detour.Build(); err != nil { + t.Fatalf("outbound does not build: %v", err) + } +} + +func publicKeyOf(t *testing.T, private string) string { + t.Helper() + raw, err := base64.RawURLEncoding.DecodeString(private) + if err != nil { + t.Fatal(err) + } + key, err := ecdh.X25519().NewPrivateKey(raw) + if err != nil { + t.Fatal(err) + } + return base64.RawURLEncoding.EncodeToString(key.PublicKey().Bytes()) +} + +func TestClientOutboundVlessFlow(t *testing.T) { + tests := []struct { + name, kind, path string + value any + want string + }{ + {"raw behind reality", "reality", "", nil, "xtls-rprx-vision"}, + {"raw behind tls", "vless-tls", "streamSettings.network", "raw", "xtls-rprx-vision"}, + {"raw without security", "vless-tls", "streamSettings.security", "none", ""}, + {"ws behind tls", "vless-tls", "", nil, ""}, + {"xhttp behind tls", "xhttp", "", nil, ""}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + raw := baseInbound(t, tt.kind) + if tt.kind == "reality" { + setPath(raw, "streamSettings.realitySettings.privateKey", testPrivateKey) + } + if tt.path != "" { + setPath(raw, tt.path, tt.value) + } + out := mirrorOne(t, readOne(t, raw)) + flow := object(out["settings"])["flow"] + if flow != tt.want { + t.Errorf("flow = %v, want %q", flow, tt.want) + } + }) + } +} + +func TestClientOutboundTrust(t *testing.T) { + cert, leafHash := selfSignedPEM(t) + lines := make([]any, 0) + for _, l := range strings.Split(strings.TrimSpace(cert), "\n") { + lines = append(lines, l) + } + const explicit = "00112233445566778899aabbccddeeff00112233445566778899aabbccddeeff" + + tests := []struct { + name string + kind string + set map[string]any + want string // expected pinnedPeerCertSha256, empty for none + fails string + }{ + {"ca by default", "vless-tls", nil, "", ""}, + {"ca ignores a certificate", "vless-tls", map[string]any{ + "streamSettings.tlsSettings.certificates": []any{map[string]any{"certificate": lines}}}, "", ""}, + {"pin from pin_sha256", "vless-tls", map[string]any{ + "x-usher.trust": "pin", "x-usher.pin_sha256": strings.ToUpper(explicit)}, explicit, ""}, + {"pin from inline lines", "vless-tls", map[string]any{ + "x-usher.trust": "pin", + "streamSettings.tlsSettings.certificates": []any{map[string]any{"certificate": lines}}}, + leafHash, ""}, + {"pin on hysteria", "hysteria", map[string]any{ + "x-usher.trust": "pin", "x-usher.pin_sha256": explicit}, explicit, ""}, + {"pin with a bad inline certificate", "vless-tls", map[string]any{ + "x-usher.trust": "pin", + "streamSettings.tlsSettings.certificates": []any{map[string]any{"certificate": []any{"not a pem"}}}}, + "", "x-usher.pin_sha256"}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + raw := baseInbound(t, tt.kind) + for path, v := range tt.set { + setPath(raw, path, v) + } + in := readOne(t, raw) + if in.Err != nil { + t.Fatal(in.Err) + } + out, err := clientOutbound(in, "h", testEntry(in)) + if tt.fails != "" { + if err == nil || !strings.Contains(err.Error(), tt.fails) { + t.Fatalf("err = %v, want it to name %q", err, tt.fails) + } + return + } + if err != nil { + t.Fatal(err) + } + tls := object(object(out["streamSettings"])["tlsSettings"]) + pin, present := tls["pinnedPeerCertSha256"] + if tt.want == "" && present { + t.Errorf("pin = %v, want none", pin) + } + if tt.want != "" && pin != tt.want { + t.Errorf("pin = %v, want %s", pin, tt.want) + } + assertBuilds(t, out) + }) + } +} + +func selfSignedPEM(t *testing.T) (certPEM, hash string) { + t.Helper() + key, err := ecdsa.GenerateKey(elliptic.P256(), rand.Reader) + if err != nil { + t.Fatal(err) + } + tmpl := &x509.Certificate{SerialNumber: big.NewInt(1), Subject: pkix.Name{CommonName: "a.example.com"}} + der, err := x509.CreateCertificate(rand.Reader, tmpl, tmpl, &key.PublicKey, key) + if err != nil { + t.Fatal(err) + } + sum := sha256.Sum256(der) + return string(pem.EncodeToMemory(&pem.Block{Type: "CERTIFICATE", Bytes: der})), hex.EncodeToString(sum[:]) +} + +func TestClientOutboundReality(t *testing.T) { + var seed [mldsa65.SeedSize]byte + for i := range seed { + seed[i] = byte(i) + } + public, _ := mldsa65.NewKeyFromSeed(&seed) + wantVerify := base64.RawURLEncoding.EncodeToString(public.Bytes()) + + realityOf := func(out map[string]any) map[string]any { + return object(object(out["streamSettings"])["realitySettings"]) + } + base := func(t *testing.T) map[string]any { + raw := baseInbound(t, "reality") + setPath(raw, "streamSettings.realitySettings.privateKey", testPrivateKey) + setPath(raw, "streamSettings.realitySettings.shortIds", []any{"ab12", "cd34"}) + return raw + } + + t.Run("public key short id and spiderX", func(t *testing.T) { + out := mirrorOne(t, readOne(t, base(t))) + r := realityOf(out) + if r["publicKey"] != publicKeyOf(t, testPrivateKey) || r["shortId"] != "ab12" || r["spiderX"] != "/" { + t.Errorf("reality = %v", r) + } + if _, ok := r["mldsa65Verify"]; ok { + t.Error("mldsa65Verify set without a seed") + } + assertBuilds(t, out) + }) + t.Run("verify key from seed", func(t *testing.T) { + raw := base(t) + setPath(raw, "streamSettings.realitySettings.mldsa65Seed", base64.RawURLEncoding.EncodeToString(seed[:])) + out := mirrorOne(t, readOne(t, raw)) + if got := realityOf(out)["mldsa65Verify"]; got != wantVerify { + t.Errorf("mldsa65Verify = %v, want the key of the seed", got) + } + assertBuilds(t, out) + }) + t.Run("no short ids", func(t *testing.T) { + raw := base(t) + setPath(raw, "streamSettings.realitySettings.shortIds", []any{}) + if got := realityOf(mirrorOne(t, readOne(t, raw)))["shortId"]; got != "" { + t.Errorf("shortId = %v, want empty", got) + } + }) + t.Run("overrides", func(t *testing.T) { + raw := base(t) + setPath(raw, "x-usher.sni", "front.example.com") + setPath(raw, "x-usher.fingerprint", "firefox") + r := realityOf(mirrorOne(t, readOne(t, raw))) + if r["serverName"] != "front.example.com" || r["fingerprint"] != "firefox" { + t.Errorf("reality = %v", r) + } + }) + + failures := []struct { + name, field string + value any + }{ + {"missing private key", "privateKey", nil}, + {"private key not base64", "privateKey", "!!!"}, + {"private key too short", "privateKey", "AAAA"}, + {"seed not base64", "mldsa65Seed", "!!!"}, + {"seed too short", "mldsa65Seed", "AAAA"}, + } + for _, tt := range failures { + t.Run(tt.name, func(t *testing.T) { + raw := base(t) + setPath(raw, "streamSettings.realitySettings."+tt.field, tt.value) + in := readOne(t, raw) + _, err := clientOutbound(in, "h", testEntry(in)) + if err == nil || !strings.Contains(err.Error(), tt.field) { + t.Fatalf("err = %v, want it to name %s", err, tt.field) + } + if tt.value != nil && strings.Contains(err.Error(), tt.value.(string)) { + t.Errorf("error quotes the value: %v", err) + } + }) + } +} + +func TestClientOutboundXHTTP(t *testing.T) { + tests := []struct { + name string + xhttp map[string]any + want string + }{ + {"plain", map[string]any{"path": "/x"}, `{"path":"/x"}`}, + {"other fields move into extra", map[string]any{ + "path": "/x", "host": "h.example.com", "mode": "packet-up", + "sessionIDPlacement": "header", "xPaddingBytes": "10-20", "noSSEHeader": true}, + `{"path":"/x","host":"h.example.com","mode":"packet-up", + "extra":{"sessionIDPlacement":"header","xPaddingBytes":"10-20","noSSEHeader":true}}`}, + {"server extra is verbatim and top-level fields are dropped", map[string]any{ + "path": "/x", "mode": "auto", "xPaddingBytes": "10-20", + "extra": map[string]any{"xPaddingObfsMode": true, "xmux": map[string]any{"maxConcurrency": "16"}}}, + `{"path":"/x","mode":"auto", + "extra":{"xPaddingObfsMode":true,"xmux":{"maxConcurrency":"16"}}}`}, + {"empty server extra falls back to the fields", map[string]any{ + "path": "/x", "xPaddingBytes": "10-20", "extra": map[string]any{}}, + `{"path":"/x","extra":{"xPaddingBytes":"10-20"}}`}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + raw := baseInbound(t, "xhttp") + setPath(raw, "streamSettings.xhttpSettings", tt.xhttp) + out := mirrorOne(t, readOne(t, raw)) + got := object(object(out["streamSettings"])["xhttpSettings"]) + wantJSON(t, got, tt.want) + assertBuilds(t, out) + }) + } + + t.Run("result does not alias the config", func(t *testing.T) { + raw := baseInbound(t, "xhttp") + setPath(raw, "streamSettings.xhttpSettings.extra", map[string]any{"xmux": map[string]any{"a": "1"}}) + in := readOne(t, raw) + out := mirrorOne(t, in) + extra := object(object(object(out["streamSettings"])["xhttpSettings"])["extra"]) + object(extra["xmux"])["a"] = "changed" + server := object(object(in.Raw["streamSettings"])["xhttpSettings"]) + if object(object(server["extra"])["xmux"])["a"] != "1" { + t.Error("the result shares memory with Inbound.Raw") + } + }) +} + +func TestClientOutboundTransports(t *testing.T) { + tests := []struct { + name, key string + server map[string]any + want string + }{ + {"ws path and host", "wsSettings", + map[string]any{"path": "/w", "host": "w.example.com", "headers": map[string]any{"X": "y"}, + "acceptProxyProtocol": true}, + `{"path":"/w","host":"w.example.com"}`}, + {"httpupgrade path and host", "httpupgradeSettings", + map[string]any{"path": "/h", "host": "h.example.com", "headers": map[string]any{"X": "y"}}, + `{"path":"/h","host":"h.example.com"}`}, + {"grpc service and authority", "grpcSettings", + map[string]any{"serviceName": "svc", "authority": "a.example.com", "multiMode": true}, + `{"serviceName":"svc","authority":"a.example.com"}`}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + raw := baseInbound(t, "vless-tls") + network := strings.TrimSuffix(tt.key, "Settings") + setPath(raw, "streamSettings.network", network) + setPath(raw, "streamSettings."+tt.key, tt.server) + out := mirrorOne(t, readOne(t, raw)) + wantJSON(t, object(object(out["streamSettings"])[tt.key]), tt.want) + assertBuilds(t, out) + }) + } +} + +func TestClientOutboundHysteria(t *testing.T) { + t.Run("no hop keeps salamander only", func(t *testing.T) { + raw := baseInbound(t, "hysteria") + setPath(raw, "streamSettings.finalmask.quicParams", map[string]any{"congestion": "bbr"}) + setPath(raw, "streamSettings.finalmask.udp", []any{map[string]any{ + "type": "salamander", "settings": map[string]any{"password": "obfs"}}}) + out := mirrorOne(t, readOne(t, raw)) + stream := object(out["streamSettings"]) + wantJSON(t, object(stream["finalmask"]), + `{"udp":[{"type":"salamander","settings":{"password":"obfs"}}]}`) + wantJSON(t, object(stream["hysteriaSettings"]), `{"version":2,"auth":"`+testPassword+`"}`) + assertBuilds(t, out) + }) + t.Run("hop without salamander", func(t *testing.T) { + raw := baseInbound(t, "hysteria") + setPath(raw, "streamSettings.finalmask", nil) + setPath(raw, "x-usher.hop_ports", "1000-2000") + out := mirrorOne(t, readOne(t, raw)) + wantJSON(t, object(object(out["streamSettings"])["finalmask"]), `{"udp":[{"type":"udphop", + "settings":{"mode":"intervalLocal,intervalRemote","interval":30, + "remotePorts":"1000-2000"}}]}`) + assertBuilds(t, out) + }) + t.Run("neither mask nor hop", func(t *testing.T) { + raw := baseInbound(t, "hysteria") + setPath(raw, "streamSettings.finalmask", nil) + out := mirrorOne(t, readOne(t, raw)) + if _, ok := object(out["streamSettings"])["finalmask"]; ok { + t.Error("finalmask set") + } + assertBuilds(t, out) + }) + t.Run("alpn is copied", func(t *testing.T) { + raw := baseInbound(t, "hysteria") + setPath(raw, "streamSettings.tlsSettings.alpn", []any{"h3"}) + tls := object(object(mirrorOne(t, readOne(t, raw))["streamSettings"])["tlsSettings"]) + if !reflect.DeepEqual(tls["alpn"], []any{"h3"}) { + t.Errorf("alpn = %v", tls["alpn"]) + } + }) +} + +func TestClientOutboundOverrides(t *testing.T) { + raw := baseInbound(t, "vless-tls") + setPath(raw, "x-usher", map[string]any{ + "address": "vpn.example.com", "port": 8443, "sni": "front.example.com", "fingerprint": "ios"}) + out := mirrorOne(t, readOne(t, raw)) + settings := object(out["settings"]) + if settings["address"] != "vpn.example.com" || settings["port"] != 8443 { + t.Errorf("settings = %v", settings) + } + tls := object(object(out["streamSettings"])["tlsSettings"]) + if tls["serverName"] != "front.example.com" || tls["fingerprint"] != "ios" { + t.Errorf("tlsSettings = %v", tls) + } + + t.Run("host argument without an address hint", func(t *testing.T) { + out := mirrorOne(t, readOne(t, baseInbound(t, "vless-tls"))) + if got := object(out["settings"])["address"]; got != "198.51.100.7" { + t.Errorf("address = %v", got) + } + }) +} + +// TestClientOutboundLeaksNothing feeds an inbound full of server-only material and checks that +// none of it reaches any outbound. +func TestClientOutboundLeaksNothing(t *testing.T) { + cert, _ := selfSignedPEM(t) + var seed [mldsa65.SeedSize]byte + seed[0] = 1 + mldsaSeed := base64.RawURLEncoding.EncodeToString(seed[:]) + const ( + keyPEM = "-----BEGIN PRIVATE KEY-----\nMIGHAgEAMBMGByqGSM49AgEGCCqGSM49AwEHBG0wawIBAQQg\n-----END PRIVATE KEY-----" + certFile = "/etc/xray/leaf.pem" + keyFile = "/etc/xray/leaf.key" + fallbackDest = "fallback.internal:8080" + ) + secrets := []string{testPrivateKey, mldsaSeed, certFile, keyFile, "MIGHAgEAMBMGByqGSM49AgEG", + "sockopt", fallbackDest, "fallbacks", "privateKey", "mldsa65Seed", "certificateFile", "keyFile"} + + kinds := []string{"reality", "vless-tls", "xhttp", "hysteria"} + for _, kind := range kinds { + for _, trust := range []string{"ca", "pin"} { + t.Run(kind+"/"+trust, func(t *testing.T) { + raw := baseInbound(t, kind) + setPath(raw, "settings.fallbacks", []any{map[string]any{"dest": fallbackDest}}) + setPath(raw, "streamSettings.sockopt", map[string]any{"mark": 255}) + setPath(raw, "streamSettings.tlsSettings.certificates", []any{map[string]any{ + "certificate": strings.Split(cert, "\n"), + "key": strings.Split(keyPEM, "\n"), + "certificateFile": certFile, + "keyFile": keyFile, + }}) + setPath(raw, "streamSettings.realitySettings.privateKey", testPrivateKey) + setPath(raw, "streamSettings.realitySettings.mldsa65Seed", mldsaSeed) + setPath(raw, "streamSettings.finalmask.quicParams", map[string]any{"congestion": "bbr"}) + if trust == "pin" && kind != "reality" { + setPath(raw, "x-usher.trust", "pin") + } + in := readOne(t, raw) + out := mirrorOne(t, in) + encoded, err := json.Marshal(out) + if err != nil { + t.Fatal(err) + } + for _, secret := range secrets { + if strings.Contains(string(encoded), secret) { + t.Errorf("outbound contains %q:\n%s", secret, encoded) + } + } + }) + } + } +} From 3cc7c06e27522e10a0bd28f4c650d824831e73ee Mon Sep 17 00:00:00 2001 From: EvaTheSalmon <18466855+EvaTheSalmon@users.noreply.github.com> Date: Wed, 14 Oct 2026 18:10:58 +0300 Subject: [PATCH 13/32] feat(xray): render client json and share links Co-Authored-By: Claude Sonnet 5.5 --- docs/plans/20261014-xray-config-backend.md | 10 +- go.mod | 6 +- go.sum | 11 +- internal/xray/config.go | 4 +- internal/xray/config_test.go | 1 + internal/xray/render.go | 95 ++++++++++++ internal/xray/render_test.go | 141 ++++++++++++++++++ internal/xray/session.go | 21 ++- internal/xray/session_test.go | 6 +- internal/xray/testdata/golden/hy2-hop.json | 66 ++++++++ internal/xray/testdata/golden/hy2-hop.txt | 1 + internal/xray/testdata/golden/hy2-pin.json | 49 ++++++ internal/xray/testdata/golden/hy2.json | 58 +++++++ internal/xray/testdata/golden/hy2.txt | 1 + internal/xray/testdata/golden/server.json | 92 ++++++++++++ internal/xray/testdata/golden/trojan-raw.json | 41 +++++ internal/xray/testdata/golden/trojan-raw.txt | 1 + .../xray/testdata/golden/vless-reality.json | 46 ++++++ .../xray/testdata/golden/vless-reality.txt | 1 + internal/xray/testdata/golden/vless-ws.json | 46 ++++++ internal/xray/testdata/golden/vless-ws.txt | 1 + .../testdata/golden/vless-xhttp-extra.json | 50 +++++++ .../testdata/golden/vless-xhttp-extra.txt | 1 + .../xray/testdata/golden/vless-xhttp.json | 55 +++++++ internal/xray/testdata/golden/vless-xhttp.txt | 1 + internal/xray/testdata/golden/vmess-ws.json | 45 ++++++ internal/xray/testdata/golden/vmess-ws.txt | 1 + 27 files changed, 834 insertions(+), 17 deletions(-) create mode 100644 internal/xray/render.go create mode 100644 internal/xray/render_test.go create mode 100644 internal/xray/testdata/golden/hy2-hop.json create mode 100644 internal/xray/testdata/golden/hy2-hop.txt create mode 100644 internal/xray/testdata/golden/hy2-pin.json create mode 100644 internal/xray/testdata/golden/hy2.json create mode 100644 internal/xray/testdata/golden/hy2.txt create mode 100644 internal/xray/testdata/golden/server.json create mode 100644 internal/xray/testdata/golden/trojan-raw.json create mode 100644 internal/xray/testdata/golden/trojan-raw.txt create mode 100644 internal/xray/testdata/golden/vless-reality.json create mode 100644 internal/xray/testdata/golden/vless-reality.txt create mode 100644 internal/xray/testdata/golden/vless-ws.json create mode 100644 internal/xray/testdata/golden/vless-ws.txt create mode 100644 internal/xray/testdata/golden/vless-xhttp-extra.json create mode 100644 internal/xray/testdata/golden/vless-xhttp-extra.txt create mode 100644 internal/xray/testdata/golden/vless-xhttp.json create mode 100644 internal/xray/testdata/golden/vless-xhttp.txt create mode 100644 internal/xray/testdata/golden/vmess-ws.json create mode 100644 internal/xray/testdata/golden/vmess-ws.txt diff --git a/docs/plans/20261014-xray-config-backend.md b/docs/plans/20261014-xray-config-backend.md index 907140f..38823db 100644 --- a/docs/plans/20261014-xray-config-backend.md +++ b/docs/plans/20261014-xray-config-backend.md @@ -437,13 +437,13 @@ reported by awg-grpc" and nothing reaches Xray. - Create: `internal/xray/render.go`, `internal/xray/render_test.go` - Create: `internal/xray/testdata/golden/` -- [ ] `.json`: complete client config with the mirrored outbound and a local proxy inbound; the +- [x] `.json`: complete client config with the mirrored outbound and a local proxy inbound; the outbound passes `conf.OutboundDetourConfig.Build()` or the inbound fails -- [ ] `.txt`: libxray per outbound; on error no `.txt` and a logged defect; none for a Hysteria2 +- [x] `.txt`: libxray per outbound; on error no `.txt` and a logged defect; none for a Hysteria2 pin inbound -- [ ] prepare step writes both files through the seam -- [ ] golden tests for the shapes in Acceptance Evidence item 2 -- [ ] run `go test -race ./...` - must pass before Task 9 +- [x] prepare step writes both files through the seam +- [x] golden tests for the shapes in Acceptance Evidence item 2 +- [x] run `go test -race ./...` - must pass before Task 9 ### Task 9: Wire the backend and the deploy example diff --git a/go.mod b/go.mod index 34764da..c8800ee 100644 --- a/go.mod +++ b/go.mod @@ -6,6 +6,7 @@ require ( github.com/cloudflare/circl v1.6.5 github.com/google/uuid v1.6.0 github.com/mrcsin/awg-grpc v0.1.0 + github.com/xtls/libxray v1.261010.0 github.com/xtls/xray-core v1.260327.1-0.20261010092107-701af60772cd google.golang.org/grpc v1.84.0 google.golang.org/protobuf v1.36.12 @@ -23,9 +24,12 @@ require ( github.com/klauspost/compress v1.20.1 // indirect github.com/klauspost/cpuid/v2 v2.4.0 // indirect github.com/koron/go-ssdp v0.9.1 // indirect - github.com/kr/text v0.2.0 // indirect github.com/libp2p/go-nat v1.0.1-0.20250821073202-01afc089f138 // indirect github.com/libp2p/go-netroute v0.4.0 // indirect + github.com/metacubex/age v0.0.0-20260603010618-28d156b4ea78 // indirect + github.com/metacubex/hkdf v0.1.0 // indirect + github.com/metacubex/hpke v0.1.0 // indirect + github.com/metacubex/mlkem v0.1.0 // indirect github.com/miekg/dns v1.1.73 // indirect github.com/pion/dtls/v3 v3.1.10 // indirect github.com/pion/logging v0.2.4 // indirect diff --git a/go.sum b/go.sum index 2063da2..d53e8b1 100644 --- a/go.sum +++ b/go.sum @@ -7,7 +7,6 @@ github.com/chzyer/readline v0.0.0-20180603132655-2972be24d48e/go.mod h1:nSuG5e5P github.com/chzyer/test v0.0.0-20180213035817-a1ea475d72b1/go.mod h1:Q3SI9o4m/ZMnBNeIyt5eFwwo7qiLfzFZmjNmxjkiQlU= github.com/cloudflare/circl v1.6.5 h1:O64F26HEqNhznd/hrC5KZXVKYuKM2rx4deZDTc4ihQA= github.com/cloudflare/circl v1.6.5/go.mod h1:h5LNyxAc5nTue9DS5jT+48en2PSDYt3zdGnz5OstK6c= -github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E= github.com/ghodss/yaml v1.0.1-0.20220118164431-d8423dcdf344 h1:Arcl6UOIS/kgO2nW3A65HN+7CMjSDP/gofXL4CZt1V4= github.com/ghodss/yaml v1.0.1-0.20220118164431-d8423dcdf344/go.mod h1:GIjDIg/heH5DOkXY3YJ/wNhfHsQHoXGjl8G8amsYQ1I= github.com/golang/mock v1.7.0-rc.1 h1:YojYx61/OLFsiv6Rw1Z96LpldJIy31o+UHmwAUMJ6/U= @@ -42,6 +41,14 @@ github.com/libp2p/go-nat v1.0.1-0.20250821073202-01afc089f138 h1:YohuNPT/1k3VcTh github.com/libp2p/go-nat v1.0.1-0.20250821073202-01afc089f138/go.mod h1:TXQg5tfSy+bUjnhT5728j5j/MBj7keIYqqZ1+8k/ui8= github.com/libp2p/go-netroute v0.4.0 h1:sZZx9hyANYUx9PZyqcgE/E1GUG3iEtTZHUEvdtXT7/Q= github.com/libp2p/go-netroute v0.4.0/go.mod h1:Nkd5ShYgSMS5MUKy/MU2T57xFoOKvvLR92Lic48LEyA= +github.com/metacubex/age v0.0.0-20260603010618-28d156b4ea78 h1:LqWr0vb9zDNuQS+jJd4fnRYk/SEI7KJ7TDe/L4WFK48= +github.com/metacubex/age v0.0.0-20260603010618-28d156b4ea78/go.mod h1:BTBG/iVY7rg3qq5WdVCg0GFk58CSvCDSbjy8I7kEx/c= +github.com/metacubex/hkdf v0.1.0 h1:fPA6VzXK8cU1foc/TOmGCDmSa7pZbxlnqhl3RNsthaA= +github.com/metacubex/hkdf v0.1.0/go.mod h1:3seEfds3smgTAXqUGn+tgEJH3uXdsUjOiduG/2EtvZ4= +github.com/metacubex/hpke v0.1.0 h1:gu2jUNhraehWi0P/z5HX2md3d7L1FhPQE6/Q0E9r9xQ= +github.com/metacubex/hpke v0.1.0/go.mod h1:vfDm6gfgrwlXUxKDkWbcE44hXtmc1uxLDm2BcR11b3U= +github.com/metacubex/mlkem v0.1.0 h1:wFClitonSFcmipzzQvax75beLQU+D7JuC+VK1RzSL8I= +github.com/metacubex/mlkem v0.1.0/go.mod h1:amhaXZVeYNShuy9BILcR7P0gbeo/QLZsnqCdL8U2PDQ= github.com/miekg/dns v1.1.73 h1:uhT8nJxmTrPJYClxVxTCX+CVn6qnzSiybRk72Z6DgrE= github.com/miekg/dns v1.1.73/go.mod h1:RW2Obtfd5NZHvOFe3zYG0W8koWOQtAzyHaLo8vASBuQ= github.com/mrcsin/awg-grpc v0.1.0 h1:hJHs0LijjnuaBuAAYgYO4bC3es9oWeXgIRwIDUwx7dE= @@ -78,6 +85,8 @@ github.com/vishvananda/netns v0.0.5 h1:DfiHV+j8bA32MFM7bfEunvT8IAqQ/NzSJHtcmW5zd github.com/vishvananda/netns v0.0.5/go.mod h1:SpkAiCQRtJ6TvvxPnOSyH3BMl6unz3xZlaprSwhNNJM= github.com/wlynxg/anet v0.0.5 h1:J3VJGi1gvo0JwZ/P1/Yc/8p63SoW98B5dHkYDmpgvvU= github.com/wlynxg/anet v0.0.5/go.mod h1:eay5PRQr7fIVAMbTbchTnO9gG65Hg/uYGdc7mguHxoA= +github.com/xtls/libxray v1.261010.0 h1:VeS2jXww0VS2MnTTW82H7vB9Kdv3bv3sqQ5vMKz/HcA= +github.com/xtls/libxray v1.261010.0/go.mod h1:vQ0Y7LFYBXEhEQiJhshJ7zn0nzbMJo49p9nQ6ffMojc= github.com/xtls/reality v0.0.0-20260921001439-3c98159dee38 h1:Q5KwiMm+WRyw2kwPW4+fIQxNio1CyrWo/eAEaAHxl/Q= github.com/xtls/reality v0.0.0-20260921001439-3c98159dee38/go.mod h1:/YQ6FwmAtkDuo8K3ujKQH1Br5eGNFauBipdl1gdiebk= github.com/xtls/xray-core v1.260327.1-0.20261010092107-701af60772cd h1:Am/9Pzw5PZr7+hMN1Vs5faeSXRoutnrwMAgtUDnh/J8= diff --git a/internal/xray/config.go b/internal/xray/config.go index 494d1e1..7c87017 100644 --- a/internal/xray/config.go +++ b/internal/xray/config.go @@ -480,7 +480,7 @@ func (in *Inbound) readHints(stream map[string]any) (field, reason string) { return "x-usher.hop_ports", "valid on hysteria only" } if !validHopPorts(hint.HopPorts) { - return "x-usher.hop_ports", `want "from-to" with 1 <= from <= to <= 65535` + return "x-usher.hop_ports", `want "from-to" with 1 <= from < to <= 65535` } } @@ -554,7 +554,7 @@ func validHopPorts(s string) bool { } from, _ := strconv.Atoi(m[1]) to, _ := strconv.Atoi(m[2]) - return from >= 1 && from <= to && to <= 65535 + return from >= 1 && from < to && to <= 65535 } // object returns v as a JSON object, or nil when it is not one. A read from nil is safe. diff --git a/internal/xray/config_test.go b/internal/xray/config_test.go index 1666971..dd457d1 100644 --- a/internal/xray/config_test.go +++ b/internal/xray/config_test.go @@ -235,6 +235,7 @@ func TestReadConfigRejects(t *testing.T) { "x-usher.pin_sha256"}, {"hop on vless", "vless-tls", map[string]any{"x-usher.hop_ports": "20000-30000"}, "x-usher.hop_ports"}, {"hop format", "hysteria", map[string]any{"x-usher.hop_ports": "30000-20000"}, "x-usher.hop_ports"}, + {"hop equal range", "hysteria", map[string]any{"x-usher.hop_ports": "20000-20000"}, "x-usher.hop_ports"}, {"hop single port", "hysteria", map[string]any{"x-usher.hop_ports": "20000"}, "x-usher.hop_ports"}, {"xhttp downloadSettings", "xhttp", map[string]any{ "streamSettings.xhttpSettings.downloadSettings": map[string]any{}}, diff --git a/internal/xray/render.go b/internal/xray/render.go new file mode 100644 index 0000000..2ea5670 --- /dev/null +++ b/internal/xray/render.go @@ -0,0 +1,95 @@ +package xray + +import ( + "encoding/json" + "fmt" + "log/slog" + + "github.com/xtls/libxray/share" + "github.com/xtls/xray-core/infra/conf" + + "github.com/mrcsin/usher/internal/clients" + "github.com/mrcsin/usher/internal/state" +) + +const ( + // JSONSuffix is the suffix of the client config usher writes for an inbound. + JSONSuffix = ".json" + // LinkSuffix is the suffix of the share link usher writes for an inbound. + LinkSuffix = ".txt" + + localSocksPort = 10808 + localHTTPPort = 10809 +) + +// renderJSON returns the complete client config for the outbound: the outbound plus a local socks +// and http proxy. The outbound must build as an Xray outbound, so a shape Xray itself refuses never +// reaches a client. +func renderJSON(outbound map[string]any) ([]byte, error) { + config := map[string]any{ + "log": map[string]any{"loglevel": "warning"}, + "inbounds": []any{ + map[string]any{ + "tag": "socks-in", "listen": "127.0.0.1", "port": localSocksPort, + "protocol": "socks", "settings": map[string]any{"udp": true}, + }, + map[string]any{ + "tag": "http-in", "listen": "127.0.0.1", "port": localHTTPPort, + "protocol": "http", + }, + }, + "outbounds": []any{outbound}, + } + encoded, err := json.Marshal(outbound) + if err != nil { + return nil, fmt.Errorf("encoding outbound: %w", err) + } + var detour conf.OutboundDetourConfig + if err := json.Unmarshal(encoded, &detour); err != nil { + return nil, fmt.Errorf("decoding outbound: %w", err) + } + if _, err := detour.Build(); err != nil { + return nil, fmt.Errorf("building outbound: %w", err) + } + out, err := json.MarshalIndent(config, "", " ") + if err != nil { + return nil, fmt.Errorf("encoding client config: %w", err) + } + return append(out, '\n'), nil +} + +// renderLink returns the share link of the client config, one line with a trailing newline. +func renderLink(clientJSON []byte) ([]byte, error) { + link, err := share.ConvertXrayJsonToShareLinks(clientJSON) + if err != nil { + return nil, err + } + return []byte(link + "\n"), nil +} + +// renderFiles returns the client files of one user on the inbound, keyed by clients.Path: always +// the .json, and the .txt unless the inbound cannot have a link. A link libxray refuses is a +// defect of usher, which builds only shapes libxray knows; it is logged and the .json stays. +func renderFiles(log *slog.Logger, in Inbound, host, user string, entry state.XrayEntry) (map[string][]byte, error) { + outbound, err := clientOutbound(in, host, entry) + if err != nil { + return nil, err + } + clientJSON, err := renderJSON(outbound) + if err != nil { + return nil, fmt.Errorf("inbound %q: %w", in.Tag, err) + } + files := map[string][]byte{clients.Path(user, in.Tag, JSONSuffix): clientJSON} + + if in.Protocol == "hysteria" && in.Hints.Trust == TrustPin { + return files, nil // v2rayNG and Hiddify do not read pcs for hysteria2:// + } + link, err := renderLink(clientJSON) + if err != nil { + log.Error("libxray refused a share link; this is a defect in usher", + "inbound", in.Tag, "user", user, "error", err) + return files, nil + } + files[clients.Path(user, in.Tag, LinkSuffix)] = link + return files, nil +} diff --git a/internal/xray/render_test.go b/internal/xray/render_test.go new file mode 100644 index 0000000..60928db --- /dev/null +++ b/internal/xray/render_test.go @@ -0,0 +1,141 @@ +package xray + +import ( + "bytes" + "encoding/json" + "flag" + "log/slog" + "os" + "path/filepath" + "strings" + "testing" + + "github.com/xtls/libxray/share" + "github.com/xtls/xray-core/infra/conf" + + "github.com/mrcsin/usher/internal/clients" +) + +var update = flag.Bool("update", false, "rewrite the golden files") + +const goldenDir = "testdata/golden" + +func TestGolden(t *testing.T) { + inbounds, err := ReadConfig(filepath.Join(goldenDir, "server.json")) + if err != nil { + t.Fatal(err) + } + for _, in := range inbounds { + t.Run(in.Tag, func(t *testing.T) { + if in.Err != nil { + t.Fatal(in.Err) + } + files, err := renderFiles(slog.New(slog.DiscardHandler), in, "198.51.100.7", "alice", testEntry(in)) + if err != nil { + t.Fatal(err) + } + checkGolden(t, files, in.Tag, JSONSuffix, true) + checkGolden(t, files, in.Tag, LinkSuffix, in.Tag != "hy2-pin") + + var parsed conf.Config + if err := json.Unmarshal(files[clients.Path("alice", in.Tag, JSONSuffix)], &parsed); err != nil { + t.Fatalf("client config does not decode: %v", err) + } + for _, outbound := range parsed.OutboundConfigs { + if _, err := outbound.Build(); err != nil { + t.Errorf("outbound does not build: %v", err) + } + } + if link, ok := files[clients.Path("alice", in.Tag, LinkSuffix)]; ok { + back, err := share.ConvertShareLinksToXrayJson(string(link), "") + if err != nil { + t.Fatalf("link does not parse back: %v", err) + } + var round conf.Config + if err := json.Unmarshal(back, &round); err != nil || len(round.OutboundConfigs) != 1 { + t.Errorf("link parsed back to %d outbounds, err %v, want 1", len(round.OutboundConfigs), err) + } + } + }) + } +} + +// checkGolden compares the file for suffix with testdata/golden/, or checks that it +// is absent when want is false. +func checkGolden(t *testing.T, files map[string][]byte, tag, suffix string, want bool) { + t.Helper() + got, present := files[clients.Path("alice", tag, suffix)] + path := filepath.Join(goldenDir, tag+suffix) + if !want { + if present { + t.Errorf("%s is present, want none", suffix) + } + return + } + if !present { + t.Fatalf("no %s file", suffix) + } + if *update { + if err := os.WriteFile(path, got, 0o644); err != nil { + t.Fatal(err) + } + } + golden, err := os.ReadFile(path) + if err != nil { + t.Fatal(err) + } + if !bytes.Equal(got, golden) { + t.Errorf("%s differs from %s, got:\n%s", suffix, path, got) + } +} + +func TestRenderJSONRejectsBadOutbound(t *testing.T) { + tests := []struct { + name string + outbound map[string]any + }{ + {"unknown protocol", map[string]any{"protocol": "nope", "settings": map[string]any{}}}, + {"bad stream network", map[string]any{ + "protocol": "trojan", + "settings": map[string]any{"address": "198.51.100.7", "port": 443, "password": "x"}, + "streamSettings": map[string]any{"network": "carrier-pigeon"}, + }}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + if _, err := renderJSON(tt.outbound); err == nil { + t.Error("renderJSON succeeded") + } + }) + } +} + +func TestRenderFilesLinkDefect(t *testing.T) { + // libxray refuses a single-port hop while the outbound still builds: the .json stays and the + // defect is logged without a secret. + inbounds, err := ReadConfig(filepath.Join(goldenDir, "server.json")) + if err != nil { + t.Fatal(err) + } + var hop Inbound + for _, in := range inbounds { + if in.Tag == "hy2-hop" { + hop = in + } + } + hop.Hints.HopPorts = "20000-20000" + logs := &bytes.Buffer{} + files, err := renderFiles(slog.New(slog.NewTextHandler(logs, nil)), hop, "198.51.100.7", "alice", testEntry(hop)) + if err != nil { + t.Fatal(err) + } + if _, ok := files[clients.Path("alice", "hy2-hop", JSONSuffix)]; !ok { + t.Error("no .json after a link defect") + } + if _, ok := files[clients.Path("alice", "hy2-hop", LinkSuffix)]; ok { + t.Error(".txt written although libxray refused the link") + } + if !strings.Contains(logs.String(), "defect") || strings.Contains(logs.String(), testPassword) { + t.Errorf("log = %q, want the defect and no secret", logs.String()) + } +} diff --git a/internal/xray/session.go b/internal/xray/session.go index 39f9291..bb0cf93 100644 --- a/internal/xray/session.go +++ b/internal/xray/session.go @@ -32,13 +32,15 @@ type Session struct { inbounds []Inbound byTag map[string]Inbound drift map[string]error + host string log *slog.Logger } // Open reads the Xray config at configPath, connects to the API and checks every inbound of the -// file against the running Xray. When socketPath is a stale socket of a killed Xray, Open removes -// it and fails, so the pass reports Xray down and the restart policy can start Xray again. -func Open(ctx context.Context, dial Dial, socketPath, configPath string, log *slog.Logger) (*Session, error) { +// file against the running Xray. host is the address clients dial unless an inbound sets one. When +// socketPath is a stale socket of a killed Xray, Open removes it and fails, so the pass reports Xray +// down and the restart policy can start Xray again. +func Open(ctx context.Context, dial Dial, socketPath, configPath, host string, log *slog.Logger) (*Session, error) { inbounds, err := ReadConfig(configPath) if err != nil { return nil, err @@ -72,6 +74,7 @@ func Open(ctx context.Context, dial Dial, socketPath, configPath string, log *sl inbounds: inbounds, byTag: byTag, drift: drift, + host: host, log: log, }, nil } @@ -86,7 +89,7 @@ func (s *Session) Interfaces() []string { return tags } -// Prepare enrolls the users on the inbound in st. It fails an inbound that did not validate. +// Prepare enrolls the users on the inbound in st and renders their client files. It fails an inbound that did not validate. func (s *Session) Prepare(st *state.State, name string, users []string) (map[string][]byte, bool, error) { in := s.byTag[name] if in.Err != nil { @@ -96,7 +99,15 @@ func (s *Session) Prepare(st *state.State, name string, users []string) (map[str if err != nil { return nil, false, err } - return map[string][]byte{}, changed, nil + files := make(map[string][]byte, 2*len(users)) + for _, user := range users { + userFiles, err := renderFiles(s.log, in, s.host, user, st.Xray[state.EntryName(name, user)]) + if err != nil { + return nil, false, fmt.Errorf("user %s: %w", user, err) + } + maps.Copy(files, userFiles) + } + return files, changed, nil } // Apply makes the inbound hold exactly the users, then returns the drift error of the tag, if diff --git a/internal/xray/session_test.go b/internal/xray/session_test.go index a2af8d7..aabda25 100644 --- a/internal/xray/session_test.go +++ b/internal/xray/session_test.go @@ -116,7 +116,7 @@ func newSessionFixture(t *testing.T, change func(c *driftConfig)) sessionFixture logs := &bytes.Buffer{} log := slog.New(slog.NewTextHandler(logs, nil)) dial := func() (command.HandlerServiceClient, func(), error) { return service, func() {}, nil } - session, err := Open(t.Context(), dial, filepath.Join(t.TempDir(), "xray.sock"), path, log) + session, err := Open(t.Context(), dial, filepath.Join(t.TempDir(), "xray.sock"), path, "198.51.100.7", log) if err != nil { t.Fatal(err) } @@ -286,7 +286,7 @@ func TestSessionApplyUnderDrift(t *testing.T) { f.cfg.realityPort = 8444 path := f.cfg.write(t) dial := func() (command.HandlerServiceClient, func(), error) { return f.service, func() {}, nil } - session, err := Open(t.Context(), dial, filepath.Join(t.TempDir(), "xray.sock"), path, slog.New(slog.DiscardHandler)) + session, err := Open(t.Context(), dial, filepath.Join(t.TempDir(), "xray.sock"), path, "198.51.100.7", slog.New(slog.DiscardHandler)) if err != nil { t.Fatal(err) } @@ -414,7 +414,7 @@ func TestOpenStaleSocketReportsDown(t *testing.T) { dialed = true return nil, func() {}, nil } - if _, err := Open(t.Context(), dial, socket, path, slog.New(slog.DiscardHandler)); err == nil { + if _, err := Open(t.Context(), dial, socket, path, "198.51.100.7", slog.New(slog.DiscardHandler)); err == nil { t.Fatal("want an error for a stale socket") } if dialed { diff --git a/internal/xray/testdata/golden/hy2-hop.json b/internal/xray/testdata/golden/hy2-hop.json new file mode 100644 index 0000000..caf4f5c --- /dev/null +++ b/internal/xray/testdata/golden/hy2-hop.json @@ -0,0 +1,66 @@ +{ + "inbounds": [ + { + "listen": "127.0.0.1", + "port": 10808, + "protocol": "socks", + "settings": { + "udp": true + }, + "tag": "socks-in" + }, + { + "listen": "127.0.0.1", + "port": 10809, + "protocol": "http", + "tag": "http-in" + } + ], + "log": { + "loglevel": "warning" + }, + "outbounds": [ + { + "protocol": "hysteria", + "settings": { + "address": "198.51.100.7", + "port": 8444, + "version": 2 + }, + "streamSettings": { + "finalmask": { + "udp": [ + { + "settings": { + "password": "obfs" + }, + "type": "salamander" + }, + { + "settings": { + "interval": 30, + "mode": "intervalLocal,intervalRemote", + "remotePorts": "20000-30000" + }, + "type": "udphop" + } + ] + }, + "hysteriaSettings": { + "auth": "user-password", + "version": 2 + }, + "network": "hysteria", + "security": "tls", + "tlsSettings": { + "alpn": [ + "h3" + ], + "fingerprint": "chrome", + "serverName": "hy.example.com" + } + }, + "tag": "hy2-hop" + } + ] +} diff --git a/internal/xray/testdata/golden/hy2-hop.txt b/internal/xray/testdata/golden/hy2-hop.txt new file mode 100644 index 0000000..0ecff89 --- /dev/null +++ b/internal/xray/testdata/golden/hy2-hop.txt @@ -0,0 +1 @@ +hysteria2://user-password@198.51.100.7:20000-30000?alpn=h3&fp=chrome&obfs=salamander&obfs-password=obfs&sni=hy.example.com#hy2-hop diff --git a/internal/xray/testdata/golden/hy2-pin.json b/internal/xray/testdata/golden/hy2-pin.json new file mode 100644 index 0000000..154d795 --- /dev/null +++ b/internal/xray/testdata/golden/hy2-pin.json @@ -0,0 +1,49 @@ +{ + "inbounds": [ + { + "listen": "127.0.0.1", + "port": 10808, + "protocol": "socks", + "settings": { + "udp": true + }, + "tag": "socks-in" + }, + { + "listen": "127.0.0.1", + "port": 10809, + "protocol": "http", + "tag": "http-in" + } + ], + "log": { + "loglevel": "warning" + }, + "outbounds": [ + { + "protocol": "hysteria", + "settings": { + "address": "198.51.100.7", + "port": 8445, + "version": 2 + }, + "streamSettings": { + "hysteriaSettings": { + "auth": "user-password", + "version": 2 + }, + "network": "hysteria", + "security": "tls", + "tlsSettings": { + "alpn": [ + "h3" + ], + "fingerprint": "chrome", + "pinnedPeerCertSha256": "aabbccddeeff00112233445566778899aabbccddeeff00112233445566778899", + "serverName": "hy.example.com" + } + }, + "tag": "hy2-pin" + } + ] +} diff --git a/internal/xray/testdata/golden/hy2.json b/internal/xray/testdata/golden/hy2.json new file mode 100644 index 0000000..c56ead0 --- /dev/null +++ b/internal/xray/testdata/golden/hy2.json @@ -0,0 +1,58 @@ +{ + "inbounds": [ + { + "listen": "127.0.0.1", + "port": 10808, + "protocol": "socks", + "settings": { + "udp": true + }, + "tag": "socks-in" + }, + { + "listen": "127.0.0.1", + "port": 10809, + "protocol": "http", + "tag": "http-in" + } + ], + "log": { + "loglevel": "warning" + }, + "outbounds": [ + { + "protocol": "hysteria", + "settings": { + "address": "198.51.100.7", + "port": 8443, + "version": 2 + }, + "streamSettings": { + "finalmask": { + "udp": [ + { + "settings": { + "password": "obfs" + }, + "type": "salamander" + } + ] + }, + "hysteriaSettings": { + "auth": "user-password", + "version": 2 + }, + "network": "hysteria", + "security": "tls", + "tlsSettings": { + "alpn": [ + "h3" + ], + "fingerprint": "chrome", + "serverName": "hy.example.com" + } + }, + "tag": "hy2" + } + ] +} diff --git a/internal/xray/testdata/golden/hy2.txt b/internal/xray/testdata/golden/hy2.txt new file mode 100644 index 0000000..94b71ee --- /dev/null +++ b/internal/xray/testdata/golden/hy2.txt @@ -0,0 +1 @@ +hysteria2://user-password@198.51.100.7:8443?alpn=h3&fp=chrome&obfs=salamander&obfs-password=obfs&sni=hy.example.com#hy2 diff --git a/internal/xray/testdata/golden/server.json b/internal/xray/testdata/golden/server.json new file mode 100644 index 0000000..4a0dbca --- /dev/null +++ b/internal/xray/testdata/golden/server.json @@ -0,0 +1,92 @@ +// Golden inputs: one inbound per supported shape. The expected client files sit beside this file +// as .json and .txt. +{ + "inbounds": [ + { + "tag": "vless-reality", "protocol": "vless", "port": 443, + "settings": {"clients": [], "decryption": "none"}, + "streamSettings": { + "network": "raw", "security": "reality", + "realitySettings": { + "privateKey": "cEUEZw0y-gOyW9gPA93c1bt6TrCFqbsXu4o9dTC0Zt0", + "serverNames": ["www.example.com"], "shortIds": ["ab12"], "target": "www.example.com:443" + } + } + }, + { + "tag": "vless-xhttp", "protocol": "vless", "port": 2053, + "settings": {"clients": [], "decryption": "none"}, + "streamSettings": { + "network": "xhttp", "security": "tls", + "tlsSettings": {"serverName": "xh.example.com", "alpn": ["h2"], "certificates": [{"certificateFile": "/c/xh.pem", "keyFile": "/c/xh.key"}]}, + "xhttpSettings": { + "path": "/xh", "host": "xh.example.com", "mode": "auto", + "xPaddingBytes": "2000-3000", "sessionIDPlacement": "header" + } + } + }, + { + "tag": "vless-xhttp-extra", "protocol": "vless", "port": 2054, + "settings": {"clients": [], "decryption": "none"}, + "streamSettings": { + "network": "xhttp", "security": "tls", + "tlsSettings": {"serverName": "xh.example.com"}, + "xhttpSettings": {"path": "/xh", "mode": "auto", "extra": {"xPaddingBytes": "100-200"}} + } + }, + { + "tag": "vless-ws", "protocol": "vless", "port": 2083, + "settings": {"clients": [], "decryption": "none"}, + "streamSettings": { + "network": "ws", "security": "tls", + "tlsSettings": {"serverName": "ws.example.com"}, + "wsSettings": {"path": "/ws"} + } + }, + { + "tag": "hy2", "protocol": "hysteria", "port": 8443, + "settings": {"version": 2, "clients": []}, + "streamSettings": { + "network": "hysteria", "security": "tls", + "tlsSettings": {"serverName": "hy.example.com", "alpn": ["h3"]}, + "hysteriaSettings": {"version": 2}, + "finalmask": {"udp": [{"type": "salamander", "settings": {"password": "obfs"}}]} + } + }, + { + "tag": "hy2-hop", "protocol": "hysteria", "port": 8444, + "settings": {"version": 2, "clients": []}, + "streamSettings": { + "network": "hysteria", "security": "tls", + "tlsSettings": {"serverName": "hy.example.com", "alpn": ["h3"]}, + "hysteriaSettings": {"version": 2}, + "finalmask": {"udp": [{"type": "salamander", "settings": {"password": "obfs"}}]} + }, + "x-usher": {"hop_ports": "20000-30000"} + }, + { + "tag": "hy2-pin", "protocol": "hysteria", "port": 8445, + "settings": {"version": 2, "clients": []}, + "streamSettings": { + "network": "hysteria", "security": "tls", + "tlsSettings": {"serverName": "hy.example.com", "alpn": ["h3"]}, + "hysteriaSettings": {"version": 2} + }, + "x-usher": {"trust": "pin", "pin_sha256": "aabbccddeeff00112233445566778899aabbccddeeff00112233445566778899"} + }, + { + "tag": "vmess-ws", "protocol": "vmess", "port": 2096, + "settings": {"clients": []}, + "streamSettings": { + "network": "ws", "security": "tls", + "tlsSettings": {"serverName": "vm.example.com"}, + "wsSettings": {"path": "/vm"} + } + }, + { + "tag": "trojan-raw", "protocol": "trojan", "port": 8446, + "settings": {"clients": []}, + "streamSettings": {"network": "raw", "security": "tls", "tlsSettings": {"serverName": "t.example.com"}} + } + ] +} diff --git a/internal/xray/testdata/golden/trojan-raw.json b/internal/xray/testdata/golden/trojan-raw.json new file mode 100644 index 0000000..99c79be --- /dev/null +++ b/internal/xray/testdata/golden/trojan-raw.json @@ -0,0 +1,41 @@ +{ + "inbounds": [ + { + "listen": "127.0.0.1", + "port": 10808, + "protocol": "socks", + "settings": { + "udp": true + }, + "tag": "socks-in" + }, + { + "listen": "127.0.0.1", + "port": 10809, + "protocol": "http", + "tag": "http-in" + } + ], + "log": { + "loglevel": "warning" + }, + "outbounds": [ + { + "protocol": "trojan", + "settings": { + "address": "198.51.100.7", + "password": "user-password", + "port": 8446 + }, + "streamSettings": { + "network": "raw", + "security": "tls", + "tlsSettings": { + "fingerprint": "chrome", + "serverName": "t.example.com" + } + }, + "tag": "trojan-raw" + } + ] +} diff --git a/internal/xray/testdata/golden/trojan-raw.txt b/internal/xray/testdata/golden/trojan-raw.txt new file mode 100644 index 0000000..8f37e10 --- /dev/null +++ b/internal/xray/testdata/golden/trojan-raw.txt @@ -0,0 +1 @@ +trojan://user-password@198.51.100.7:8446?fp=chrome&security=tls&sni=t.example.com&type=tcp#trojan-raw diff --git a/internal/xray/testdata/golden/vless-reality.json b/internal/xray/testdata/golden/vless-reality.json new file mode 100644 index 0000000..45b4688 --- /dev/null +++ b/internal/xray/testdata/golden/vless-reality.json @@ -0,0 +1,46 @@ +{ + "inbounds": [ + { + "listen": "127.0.0.1", + "port": 10808, + "protocol": "socks", + "settings": { + "udp": true + }, + "tag": "socks-in" + }, + { + "listen": "127.0.0.1", + "port": 10809, + "protocol": "http", + "tag": "http-in" + } + ], + "log": { + "loglevel": "warning" + }, + "outbounds": [ + { + "protocol": "vless", + "settings": { + "address": "198.51.100.7", + "encryption": "none", + "flow": "xtls-rprx-vision", + "id": "3f2504e0-4f89-41d3-9a0c-0305e82c3301", + "port": 443 + }, + "streamSettings": { + "network": "raw", + "realitySettings": { + "fingerprint": "chrome", + "publicKey": "VKrK-MBnuOdef7qi0Kp3c7UvISj_SrynyMJowLZmXBc", + "serverName": "www.example.com", + "shortId": "ab12", + "spiderX": "/" + }, + "security": "reality" + }, + "tag": "vless-reality" + } + ] +} diff --git a/internal/xray/testdata/golden/vless-reality.txt b/internal/xray/testdata/golden/vless-reality.txt new file mode 100644 index 0000000..0b5f031 --- /dev/null +++ b/internal/xray/testdata/golden/vless-reality.txt @@ -0,0 +1 @@ +vless://3f2504e0-4f89-41d3-9a0c-0305e82c3301@198.51.100.7:443?encryption=none&flow=xtls-rprx-vision&fp=chrome&pbk=VKrK-MBnuOdef7qi0Kp3c7UvISj_SrynyMJowLZmXBc&security=reality&sid=ab12&sni=www.example.com&spx=%2F&type=tcp#vless-reality diff --git a/internal/xray/testdata/golden/vless-ws.json b/internal/xray/testdata/golden/vless-ws.json new file mode 100644 index 0000000..1df1c90 --- /dev/null +++ b/internal/xray/testdata/golden/vless-ws.json @@ -0,0 +1,46 @@ +{ + "inbounds": [ + { + "listen": "127.0.0.1", + "port": 10808, + "protocol": "socks", + "settings": { + "udp": true + }, + "tag": "socks-in" + }, + { + "listen": "127.0.0.1", + "port": 10809, + "protocol": "http", + "tag": "http-in" + } + ], + "log": { + "loglevel": "warning" + }, + "outbounds": [ + { + "protocol": "vless", + "settings": { + "address": "198.51.100.7", + "encryption": "none", + "flow": "", + "id": "3f2504e0-4f89-41d3-9a0c-0305e82c3301", + "port": 2083 + }, + "streamSettings": { + "network": "ws", + "security": "tls", + "tlsSettings": { + "fingerprint": "chrome", + "serverName": "ws.example.com" + }, + "wsSettings": { + "path": "/ws" + } + }, + "tag": "vless-ws" + } + ] +} diff --git a/internal/xray/testdata/golden/vless-ws.txt b/internal/xray/testdata/golden/vless-ws.txt new file mode 100644 index 0000000..35c83a2 --- /dev/null +++ b/internal/xray/testdata/golden/vless-ws.txt @@ -0,0 +1 @@ +vless://3f2504e0-4f89-41d3-9a0c-0305e82c3301@198.51.100.7:2083?encryption=none&fp=chrome&path=%2Fws&security=tls&sni=ws.example.com&type=ws#vless-ws diff --git a/internal/xray/testdata/golden/vless-xhttp-extra.json b/internal/xray/testdata/golden/vless-xhttp-extra.json new file mode 100644 index 0000000..dc57c0a --- /dev/null +++ b/internal/xray/testdata/golden/vless-xhttp-extra.json @@ -0,0 +1,50 @@ +{ + "inbounds": [ + { + "listen": "127.0.0.1", + "port": 10808, + "protocol": "socks", + "settings": { + "udp": true + }, + "tag": "socks-in" + }, + { + "listen": "127.0.0.1", + "port": 10809, + "protocol": "http", + "tag": "http-in" + } + ], + "log": { + "loglevel": "warning" + }, + "outbounds": [ + { + "protocol": "vless", + "settings": { + "address": "198.51.100.7", + "encryption": "none", + "flow": "", + "id": "3f2504e0-4f89-41d3-9a0c-0305e82c3301", + "port": 2054 + }, + "streamSettings": { + "network": "xhttp", + "security": "tls", + "tlsSettings": { + "fingerprint": "chrome", + "serverName": "xh.example.com" + }, + "xhttpSettings": { + "extra": { + "xPaddingBytes": "100-200" + }, + "mode": "auto", + "path": "/xh" + } + }, + "tag": "vless-xhttp-extra" + } + ] +} diff --git a/internal/xray/testdata/golden/vless-xhttp-extra.txt b/internal/xray/testdata/golden/vless-xhttp-extra.txt new file mode 100644 index 0000000..8c2eb56 --- /dev/null +++ b/internal/xray/testdata/golden/vless-xhttp-extra.txt @@ -0,0 +1 @@ +vless://3f2504e0-4f89-41d3-9a0c-0305e82c3301@198.51.100.7:2054?encryption=none&extra=%7B%0A%20%20%20%20%20%20%20%20%20%20%20%20%22xPaddingBytes%22%3A%20%22100-200%22%0A%20%20%20%20%20%20%20%20%20%20%7D&fp=chrome&mode=auto&path=%2Fxh&security=tls&sni=xh.example.com&type=xhttp#vless-xhttp-extra diff --git a/internal/xray/testdata/golden/vless-xhttp.json b/internal/xray/testdata/golden/vless-xhttp.json new file mode 100644 index 0000000..b640d7c --- /dev/null +++ b/internal/xray/testdata/golden/vless-xhttp.json @@ -0,0 +1,55 @@ +{ + "inbounds": [ + { + "listen": "127.0.0.1", + "port": 10808, + "protocol": "socks", + "settings": { + "udp": true + }, + "tag": "socks-in" + }, + { + "listen": "127.0.0.1", + "port": 10809, + "protocol": "http", + "tag": "http-in" + } + ], + "log": { + "loglevel": "warning" + }, + "outbounds": [ + { + "protocol": "vless", + "settings": { + "address": "198.51.100.7", + "encryption": "none", + "flow": "", + "id": "3f2504e0-4f89-41d3-9a0c-0305e82c3301", + "port": 2053 + }, + "streamSettings": { + "network": "xhttp", + "security": "tls", + "tlsSettings": { + "alpn": [ + "h2" + ], + "fingerprint": "chrome", + "serverName": "xh.example.com" + }, + "xhttpSettings": { + "extra": { + "sessionIDPlacement": "header", + "xPaddingBytes": "2000-3000" + }, + "host": "xh.example.com", + "mode": "auto", + "path": "/xh" + } + }, + "tag": "vless-xhttp" + } + ] +} diff --git a/internal/xray/testdata/golden/vless-xhttp.txt b/internal/xray/testdata/golden/vless-xhttp.txt new file mode 100644 index 0000000..275a030 --- /dev/null +++ b/internal/xray/testdata/golden/vless-xhttp.txt @@ -0,0 +1 @@ +vless://3f2504e0-4f89-41d3-9a0c-0305e82c3301@198.51.100.7:2053?alpn=h2&encryption=none&extra=%7B%0A%20%20%20%20%20%20%20%20%20%20%20%20%22sessionIDPlacement%22%3A%20%22header%22%2C%0A%20%20%20%20%20%20%20%20%20%20%20%20%22xPaddingBytes%22%3A%20%222000-3000%22%0A%20%20%20%20%20%20%20%20%20%20%7D&fp=chrome&host=xh.example.com&mode=auto&path=%2Fxh&security=tls&sni=xh.example.com&type=xhttp#vless-xhttp diff --git a/internal/xray/testdata/golden/vmess-ws.json b/internal/xray/testdata/golden/vmess-ws.json new file mode 100644 index 0000000..c021dd1 --- /dev/null +++ b/internal/xray/testdata/golden/vmess-ws.json @@ -0,0 +1,45 @@ +{ + "inbounds": [ + { + "listen": "127.0.0.1", + "port": 10808, + "protocol": "socks", + "settings": { + "udp": true + }, + "tag": "socks-in" + }, + { + "listen": "127.0.0.1", + "port": 10809, + "protocol": "http", + "tag": "http-in" + } + ], + "log": { + "loglevel": "warning" + }, + "outbounds": [ + { + "protocol": "vmess", + "settings": { + "address": "198.51.100.7", + "id": "3f2504e0-4f89-41d3-9a0c-0305e82c3301", + "port": 2096, + "security": "auto" + }, + "streamSettings": { + "network": "ws", + "security": "tls", + "tlsSettings": { + "fingerprint": "chrome", + "serverName": "vm.example.com" + }, + "wsSettings": { + "path": "/vm" + } + }, + "tag": "vmess-ws" + } + ] +} diff --git a/internal/xray/testdata/golden/vmess-ws.txt b/internal/xray/testdata/golden/vmess-ws.txt new file mode 100644 index 0000000..4097402 --- /dev/null +++ b/internal/xray/testdata/golden/vmess-ws.txt @@ -0,0 +1 @@ +vmess://3f2504e0-4f89-41d3-9a0c-0305e82c3301@198.51.100.7:2096?encryption=auto&fp=chrome&path=%2Fvm&security=tls&sni=vm.example.com&type=ws#vmess-ws From 398be301d3745f09b967b201c837766080f14c1e Mon Sep 17 00:00:00 2001 From: EvaTheSalmon <18466855+EvaTheSalmon@users.noreply.github.com> Date: Wed, 14 Oct 2026 18:13:25 +0300 Subject: [PATCH 14/32] feat(usher): wire the Xray backend Co-Authored-By: Claude Sonnet 5.5 --- cmd/usher/env.go | 23 +++++++-- cmd/usher/env_test.go | 53 ++++++++++++++++++-- cmd/usher/main.go | 34 ++++++++++++- cmd/usher/main_test.go | 39 +++++++++++++++ deploy/compose.example.yml | 22 ++++++++ deploy/xray/config.json | 58 ++++++++++++++++++++++ docs/plans/20261014-xray-config-backend.md | 8 +-- 7 files changed, 222 insertions(+), 15 deletions(-) create mode 100644 deploy/xray/config.json diff --git a/cmd/usher/env.go b/cmd/usher/env.go index 08b2de8..11686a9 100644 --- a/cmd/usher/env.go +++ b/cmd/usher/env.go @@ -21,7 +21,10 @@ type environment struct { pass.Settings Host netip.Addr // USHER_HOST DNS []netip.Addr // USHER_DNS - AWGSocket string // USHER_AWG_SOCKET + AWGSocket string // USHER_AWG_SOCKET, empty without the awg backend + + XraySocket string // USHER_XRAY_SOCKET, empty without the Xray backend + XrayConfig string // USHER_XRAY_CONFIG, set with XraySocket } func environmentFrom(getenv func(string) string) (environment, error) { @@ -30,7 +33,7 @@ func environmentFrom(getenv func(string) string) (environment, error) { return environment{}, err } - var awgSocket string + var awgSocket, xraySocket, xrayConfig string var dns []netip.Addr if getenv("USHER_AWG_SOCKET") != "" { if awgSocket, err = parseSocketPath("USHER_AWG_SOCKET", getenv("USHER_AWG_SOCKET")); err != nil { @@ -40,8 +43,16 @@ func environmentFrom(getenv func(string) string) (environment, error) { return environment{}, err } } - if awgSocket == "" { - return environment{}, errors.New("USHER_AWG_SOCKET is required") + if getenv("USHER_XRAY_SOCKET") != "" { + if xraySocket, err = parseSocketPath("USHER_XRAY_SOCKET", getenv("USHER_XRAY_SOCKET")); err != nil { + return environment{}, err + } + if xrayConfig, err = parseSocketPath("USHER_XRAY_CONFIG", getenv("USHER_XRAY_CONFIG")); err != nil { + return environment{}, err + } + } + if awgSocket == "" && xraySocket == "" { + return environment{}, errors.New("USHER_AWG_SOCKET or USHER_XRAY_SOCKET is required") } return environment{ @@ -49,6 +60,9 @@ func environmentFrom(getenv func(string) string) (environment, error) { Host: host, DNS: dns, AWGSocket: awgSocket, + + XraySocket: xraySocket, + XrayConfig: xrayConfig, }, nil } @@ -76,6 +90,7 @@ func parseIPv4(name, value string) (netip.Addr, error) { return addr, nil } +// parseSocketPath checks that value is an absolute path; it serves the socket and config paths. func parseSocketPath(name, value string) (string, error) { if value == "" { return "", fmt.Errorf("%s is required", name) diff --git a/cmd/usher/env_test.go b/cmd/usher/env_test.go index 718564c..0158402 100644 --- a/cmd/usher/env_test.go +++ b/cmd/usher/env_test.go @@ -63,9 +63,52 @@ func TestEnvironmentFrom(t *testing.T) { wantErr: "USHER_AWG_SOCKET", }, { - name: "missing awg socket does not ask for DNS", + name: "no backend socket does not ask for DNS", env: map[string]string{"USHER_HOST": "203.0.113.10"}, - wantErr: "USHER_AWG_SOCKET is required", + wantErr: "USHER_AWG_SOCKET or USHER_XRAY_SOCKET is required", + }, + { + name: "xray only needs no DNS", + env: map[string]string{"USHER_HOST": "203.0.113.10", "USHER_XRAY_SOCKET": "/run/xray/api.sock", "USHER_XRAY_CONFIG": "/etc/xray/config.json"}, + want: environment{ + Host: netip.MustParseAddr("203.0.113.10"), + XraySocket: "/run/xray/api.sock", + XrayConfig: "/etc/xray/config.json", + }, + }, + { + name: "both backends", + env: map[string]string{ + "USHER_HOST": "203.0.113.10", "USHER_AWG_SOCKET": "/run/awg.sock", "USHER_DNS": "1.1.1.1", + "USHER_XRAY_SOCKET": "/run/xray/api.sock", "USHER_XRAY_CONFIG": "/etc/xray/config.json", + }, + want: environment{ + Host: netip.MustParseAddr("203.0.113.10"), + DNS: []netip.Addr{netip.MustParseAddr("1.1.1.1")}, + AWGSocket: "/run/awg.sock", + XraySocket: "/run/xray/api.sock", + XrayConfig: "/etc/xray/config.json", + }, + }, + { + name: "xray socket without config", + env: map[string]string{"USHER_HOST": "203.0.113.10", "USHER_XRAY_SOCKET": "/run/xray/api.sock"}, + wantErr: "USHER_XRAY_CONFIG", + }, + { + name: "relative xray socket", + env: map[string]string{"USHER_HOST": "203.0.113.10", "USHER_XRAY_SOCKET": "api.sock", "USHER_XRAY_CONFIG": "/etc/xray/config.json"}, + wantErr: "USHER_XRAY_SOCKET", + }, + { + name: "relative xray config", + env: map[string]string{"USHER_HOST": "203.0.113.10", "USHER_XRAY_SOCKET": "/run/xray/api.sock", "USHER_XRAY_CONFIG": "config.json"}, + wantErr: "USHER_XRAY_CONFIG", + }, + { + name: "xray config without socket", + env: map[string]string{"USHER_HOST": "203.0.113.10", "USHER_XRAY_CONFIG": "/etc/xray/config.json"}, + wantErr: "USHER_AWG_SOCKET or USHER_XRAY_SOCKET is required", }, { name: "empty DNS", @@ -100,9 +143,9 @@ func TestEnvironmentFrom(t *testing.T) { if err != nil { t.Fatalf("unexpected error: %v", err) } - if got.Host != tt.want.Host || !slices.Equal(got.DNS, tt.want.DNS) || got.AWGSocket != tt.want.AWGSocket { - t.Errorf("host/dns/socket = %v %v %q, want %v %v %q", - got.Host, got.DNS, got.AWGSocket, tt.want.Host, tt.want.DNS, tt.want.AWGSocket) + if got.Host != tt.want.Host || !slices.Equal(got.DNS, tt.want.DNS) || got.AWGSocket != tt.want.AWGSocket || + got.XraySocket != tt.want.XraySocket || got.XrayConfig != tt.want.XrayConfig { + t.Errorf("got %+v, want %+v", got, tt.want) } }) } diff --git a/cmd/usher/main.go b/cmd/usher/main.go index 793c43a..8ed9570 100644 --- a/cmd/usher/main.go +++ b/cmd/usher/main.go @@ -1,4 +1,4 @@ -// Command usher keeps AmneziaWG peers and client configs in step with config/usher.yml. +// Command usher keeps AmneziaWG and Xray users and client configs in step with config/usher.yml. package main import ( @@ -12,12 +12,14 @@ import ( "time" awgv1 "github.com/mrcsin/awg-grpc/gen/awg/v1" + "github.com/xtls/xray-core/app/proxyman/command" "google.golang.org/grpc" "google.golang.org/grpc/credentials/insecure" "github.com/mrcsin/usher/internal/awg" "github.com/mrcsin/usher/internal/pass" "github.com/mrcsin/usher/internal/watch" + "github.com/mrcsin/usher/internal/xray" ) // version is set at build time with -X main.version. @@ -41,7 +43,13 @@ func run(ctx context.Context, args []string, getenv func(string) string, stderr } log := slog.New(slog.NewTextHandler(stderr, nil)) - backends := []pass.Backend{awgBackend(env, log)} + var backends []pass.Backend + if env.AWGSocket != "" { + backends = append(backends, awgBackend(env, log)) + } + if env.XraySocket != "" { + backends = append(backends, xrayBackend(env, log)) + } return serve(ctx, env.Settings, backends, log) } @@ -65,6 +73,17 @@ func awgBackend(env environment, log *slog.Logger) pass.Backend { } } +func xrayBackend(env environment, log *slog.Logger) pass.Backend { + dial := dialXraySocket(env.XraySocket) + return pass.Backend{ + Name: "xray", + Suffixes: []string{xray.JSONSuffix, xray.LinkSuffix}, + Open: func(ctx context.Context) (pass.Session, error) { + return asSession(xray.Open(ctx, dial, env.XraySocket, env.XrayConfig, env.Host.String(), log)) + }, + } +} + // asSession converts a backend session to pass.Session. On error it returns a nil interface, not // an interface that holds a nil pointer. func asSession[S pass.Session](session S, err error) (pass.Session, error) { @@ -84,3 +103,14 @@ func dialAWGSocket(path string) awg.Dial { return awgv1.NewManagementServiceClient(conn), func() { _ = conn.Close() }, nil } } + +func dialXraySocket(path string) xray.Dial { + target := "unix://" + path + return func() (command.HandlerServiceClient, func(), error) { + conn, err := grpc.NewClient(target, grpc.WithTransportCredentials(insecure.NewCredentials())) + if err != nil { + return nil, nil, fmt.Errorf("opening %s: %w", target, err) + } + return command.NewHandlerServiceClient(conn), func() { _ = conn.Close() }, nil + } +} diff --git a/cmd/usher/main_test.go b/cmd/usher/main_test.go index cb8b69c..e5cdda3 100644 --- a/cmd/usher/main_test.go +++ b/cmd/usher/main_test.go @@ -6,11 +6,13 @@ import ( "errors" "log/slog" "path/filepath" + "slices" "strings" "testing" "time" "github.com/mrcsin/usher/internal/pass" + "github.com/mrcsin/usher/internal/xray" ) func TestRun(t *testing.T) { @@ -54,6 +56,29 @@ func TestRun(t *testing.T) { } } +func TestBackends(t *testing.T) { + log := slog.New(slog.NewTextHandler(&bytes.Buffer{}, nil)) + env := environment{ + AWGSocket: "/run/awg.sock", + XraySocket: "/run/xray/api.sock", + XrayConfig: filepath.Join(t.TempDir(), "missing.json"), + } + + awg := awgBackend(env, log) + if awg.Name != "awg-grpc" || !slices.Equal(awg.Suffixes, []string{".conf"}) { + t.Errorf("awg backend = %q %v", awg.Name, awg.Suffixes) + } + + xr := xrayBackend(env, log) + if xr.Name != "xray" || !slices.Equal(xr.Suffixes, []string{".json", ".txt"}) { + t.Errorf("xray backend = %q %v", xr.Name, xr.Suffixes) + } + session, err := xr.Open(context.Background()) + if err == nil || session != nil { + t.Errorf("Open = %v, %v, want a nil session and an error for the missing config", session, err) + } +} + func TestServe(t *testing.T) { root := t.TempDir() settings := pass.Settings{ @@ -89,3 +114,17 @@ func TestServe(t *testing.T) { } } } + +func TestDeployExampleConfig(t *testing.T) { + inbounds, err := xray.ReadConfig(filepath.Join("..", "..", "deploy", "xray", "config.json")) + if err != nil { + t.Fatal(err) + } + var tags []string + for _, in := range inbounds { + tags = append(tags, in.Tag) + } + if want := []string{"vless-reality", "hysteria2"}; !slices.Equal(tags, want) { + t.Errorf("tags = %v, want %v", tags, want) + } +} diff --git a/deploy/compose.example.yml b/deploy/compose.example.yml index 891102c..9371dca 100644 --- a/deploy/compose.example.yml +++ b/deploy/compose.example.yml @@ -21,6 +21,21 @@ services: environment: # Must equal usher's group. AWG_GRPC_SOCKET_GID: "1000" + xray: + # The tag follows the xray-core version in usher's go.mod. A different version can change how + # Xray builds an inbound, and usher then reports every inbound as drifted. + image: ghcr.io/xtls/xray-core:26.10.10 + restart: unless-stopped + # Must equal usher's group, so that usher can open the API socket. + user: "0:1000" + command: ["run", "-config", "/etc/xray/config.json"] + ports: + - "443:443/tcp" + - "8443:8443/udp" + volumes: + # The directory, not the file: usher and Xray then both see a replaced config. + - ./xray:/etc/xray:ro + - xray-sock:/run/xray usher: image: ghcr.io/mrcsin/usher:v0.1.0 restart: unless-stopped @@ -30,16 +45,23 @@ services: depends_on: awg: condition: service_healthy + xray: + condition: service_started volumes: # The host directories belong to uid 1000. - ./usher/config:/srv/usher/config:ro - ./usher/clients:/srv/usher/clients - usher-state:/srv/usher/state - awg-sock:/run/awg-grpc + - ./xray:/etc/xray:ro + - xray-sock:/run/xray environment: USHER_HOST: "203.0.113.10" USHER_AWG_SOCKET: /run/awg-grpc/awg.sock USHER_DNS: "1.1.1.1,1.0.0.1" + USHER_XRAY_SOCKET: /run/xray/api.sock + USHER_XRAY_CONFIG: /etc/xray/config.json volumes: awg-sock: + xray-sock: usher-state: diff --git a/deploy/xray/config.json b/deploy/xray/config.json new file mode 100644 index 0000000..07a62bd --- /dev/null +++ b/deploy/xray/config.json @@ -0,0 +1,58 @@ +{ + // Placeholders below make Xray refuse to start until the operator replaces them. + "log": { "loglevel": "warning" }, + "api": { + "tag": "api", + "listen": "/run/xray/api.sock,0660", + "services": ["HandlerService"] + }, + "inbounds": [ + { + "tag": "vless-reality", + "listen": "0.0.0.0", + "port": 443, + "protocol": "vless", + "settings": { "clients": [], "decryption": "none" }, + "streamSettings": { + "network": "raw", + "security": "reality", + "realitySettings": { + // Generate with `xray x25519`. + "privateKey": "REPLACE_WITH_X25519_PRIVATE_KEY", + "target": "example.com:443", + "serverNames": ["example.com"], + "shortIds": ["REPLACE_WITH_HEX_SHORT_ID"] + } + }, + "x-usher": { "address": "203.0.113.10" } + }, + { + "tag": "hysteria2", + "listen": "0.0.0.0", + "port": 8443, + "protocol": "hysteria", + "settings": { "version": 2, "clients": [] }, + "streamSettings": { + "network": "hysteria", + "security": "tls", + "tlsSettings": { + "serverName": "vpn.example.com", + "alpn": ["h3"], + // A certificate that a public CA issued for the domain above. + "certificates": [ + { + "certificateFile": "/etc/xray/certs/REPLACE_WITH_FULLCHAIN.pem", + "keyFile": "/etc/xray/certs/REPLACE_WITH_PRIVKEY.pem" + } + ] + }, + "hysteriaSettings": { "version": 2 }, + "finalmask": { + "udp": [{ "type": "salamander", "settings": { "password": "REPLACE_WITH_OBFS_PASSWORD" } }] + } + }, + "x-usher": { "address": "203.0.113.10", "sni": "vpn.example.com" } + } + ], + "outbounds": [{ "protocol": "freedom", "tag": "direct" }] +} diff --git a/docs/plans/20261014-xray-config-backend.md b/docs/plans/20261014-xray-config-backend.md index 38823db..ef42a79 100644 --- a/docs/plans/20261014-xray-config-backend.md +++ b/docs/plans/20261014-xray-config-backend.md @@ -451,13 +451,13 @@ reported by awg-grpc" and nothing reaches Xray. - Modify: `cmd/usher/env.go`, `cmd/usher/main.go`, tests - Modify: `deploy/compose.example.yml`; Create: `deploy/xray/config.json` -- [ ] read `USHER_XRAY_SOCKET` and `USHER_XRAY_CONFIG`; build the Xray backend -- [ ] example config: one VLESS+Reality and one Hysteria2 inbound with a `certificateFile` and +- [x] read `USHER_XRAY_SOCKET` and `USHER_XRAY_CONFIG`; build the Xray backend +- [x] example config: one VLESS+Reality and one Hysteria2 inbound with a `certificateFile` and `x-usher`, placeholders that Xray refuses to start with; compose mounts the config directory into both containers (usher read-only), a socket directory usher owns, Xray with `restart: unless-stopped` and image `26.10.10` with a comment that it follows `go.mod` -- [ ] write env and wiring tests -- [ ] run `go test -race ./...` - must pass before Task 10 +- [x] write env and wiring tests +- [x] run `go test -race ./...` - must pass before Task 10 ### Task 10: Xray end-to-end suite From e06d0f2103151d80177937ac6982b1b25b50a85e Mon Sep 17 00:00:00 2001 From: EvaTheSalmon <18466855+EvaTheSalmon@users.noreply.github.com> Date: Wed, 14 Oct 2026 18:44:02 +0300 Subject: [PATCH 15/32] fix(xray): open the socket lock read-only Xray creates the lock as root with mode 0644, so the read-write open failed for usher and every pass reported Xray down. The example compose file now binds a host socket directory that usher can write. Co-Authored-By: Claude Sonnet 5.5 --- deploy/compose.example.yml | 8 +++++--- internal/xray/socket.go | 23 ++++++++++++++--------- internal/xray/socket_test.go | 20 ++++++++++++++++++++ 3 files changed, 39 insertions(+), 12 deletions(-) diff --git a/deploy/compose.example.yml b/deploy/compose.example.yml index 9371dca..24ce986 100644 --- a/deploy/compose.example.yml +++ b/deploy/compose.example.yml @@ -35,7 +35,10 @@ services: volumes: # The directory, not the file: usher and Xray then both see a replaced config. - ./xray:/etc/xray:ro - - xray-sock:/run/xray + # A host directory that uid 1000 owns (create it before the first start), not a named volume, + # which belongs to root: after a crash usher unlinks the stale API socket, and that needs + # write access to the directory. + - ./xray-sock:/run/xray usher: image: ghcr.io/mrcsin/usher:v0.1.0 restart: unless-stopped @@ -54,7 +57,7 @@ services: - usher-state:/srv/usher/state - awg-sock:/run/awg-grpc - ./xray:/etc/xray:ro - - xray-sock:/run/xray + - ./xray-sock:/run/xray environment: USHER_HOST: "203.0.113.10" USHER_AWG_SOCKET: /run/awg-grpc/awg.sock @@ -63,5 +66,4 @@ services: USHER_XRAY_CONFIG: /etc/xray/config.json volumes: awg-sock: - xray-sock: usher-state: diff --git a/internal/xray/socket.go b/internal/xray/socket.go index 5eaa4a7..eb0796c 100644 --- a/internal/xray/socket.go +++ b/internal/xray/socket.go @@ -17,18 +17,23 @@ func removeStaleSocket(socketPath string) (bool, error) { } else if err != nil { return false, fmt.Errorf("checking xray socket: %w", err) } - lock, err := os.OpenFile(socketPath+".lock", os.O_RDWR|os.O_CREATE, 0o600) - if err != nil { + // Xray creates the lock as root with mode 0644, so usher opens it read-only; flock needs no + // write access. A socket without a lock file has no live owner either. + lock, err := os.Open(socketPath + ".lock") + switch { + case errors.Is(err, fs.ErrNotExist): + case err != nil: return false, fmt.Errorf("opening xray socket lock: %w", err) - } - defer lock.Close() - if err := syscall.Flock(int(lock.Fd()), syscall.LOCK_EX|syscall.LOCK_NB); err != nil { - if errors.Is(err, syscall.EWOULDBLOCK) { - return false, nil + default: + defer lock.Close() + if err := syscall.Flock(int(lock.Fd()), syscall.LOCK_EX|syscall.LOCK_NB); err != nil { + if errors.Is(err, syscall.EWOULDBLOCK) { + return false, nil + } + return false, fmt.Errorf("locking xray socket: %w", err) } - return false, fmt.Errorf("locking xray socket: %w", err) + defer syscall.Flock(int(lock.Fd()), syscall.LOCK_UN) //nolint:errcheck // closing the file releases it } - defer syscall.Flock(int(lock.Fd()), syscall.LOCK_UN) //nolint:errcheck // closing the file releases it if err := os.Remove(socketPath); err != nil && !errors.Is(err, fs.ErrNotExist) { return false, fmt.Errorf("removing stale xray socket: %w", err) } diff --git a/internal/xray/socket_test.go b/internal/xray/socket_test.go index 434c798..1cc3ce9 100644 --- a/internal/xray/socket_test.go +++ b/internal/xray/socket_test.go @@ -39,6 +39,26 @@ func TestRemoveStaleSocket(t *testing.T) { t.Fatalf("socket still there: %v", err) } }) + t.Run("read-only lock file is enough", func(t *testing.T) { + path := staleSocket(t) + if err := os.WriteFile(path+".lock", nil, 0o444); err != nil { + t.Fatal(err) + } + removed, err := removeStaleSocket(path) + if err != nil || !removed { + t.Fatalf("removed=%v err=%v", removed, err) + } + }) + t.Run("missing lock file removes the socket", func(t *testing.T) { + path := staleSocket(t) + removed, err := removeStaleSocket(path) + if err != nil || !removed { + t.Fatalf("removed=%v err=%v", removed, err) + } + if _, err := os.Lstat(path + ".lock"); !os.IsNotExist(err) { + t.Fatalf("lock file created: %v", err) + } + }) t.Run("held lock keeps the socket", func(t *testing.T) { path := staleSocket(t) lock, err := os.OpenFile(path+".lock", os.O_RDWR|os.O_CREATE, 0o600) From f06d531ec5ba8cdfb6b26cdb061ae02f789ed280 Mon Sep 17 00:00:00 2001 From: EvaTheSalmon <18466855+EvaTheSalmon@users.noreply.github.com> Date: Wed, 14 Oct 2026 18:44:14 +0300 Subject: [PATCH 16/32] test(xray): add the Xray e2e suite Co-Authored-By: Claude Sonnet 5.5 --- .github/workflows/ci.yml | 3 + .gitignore | 1 + deploy/compose.e2e-xray.yml | 93 +++++ deploy/e2e/xray/nginx.conf | 22 ++ deploy/e2e/xray/target.crt | 10 + deploy/e2e/xray/target.key | 5 + docs/plans/20261014-xray-config-backend.md | 11 +- test/e2e/xray/client_test.go | 237 +++++++++++ test/e2e/xray/pki_test.go | 87 ++++ test/e2e/xray/server_test.go | 184 +++++++++ test/e2e/xray/xray_test.go | 440 +++++++++++++++++++++ 11 files changed, 1088 insertions(+), 5 deletions(-) create mode 100644 deploy/compose.e2e-xray.yml create mode 100644 deploy/e2e/xray/nginx.conf create mode 100644 deploy/e2e/xray/target.crt create mode 100644 deploy/e2e/xray/target.key create mode 100644 test/e2e/xray/client_test.go create mode 100644 test/e2e/xray/pki_test.go create mode 100644 test/e2e/xray/server_test.go create mode 100644 test/e2e/xray/xray_test.go diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 294e0d7..3348385 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -60,3 +60,6 @@ jobs: - name: e2e test run: go test -tags e2e -count=1 -v ./test/e2e/ + + - name: xray e2e test + run: go test -tags e2e -count=1 -v ./test/e2e/xray/ diff --git a/.gitignore b/.gitignore index 776e188..bd6d940 100644 --- a/.gitignore +++ b/.gitignore @@ -5,3 +5,4 @@ Thumbs.db .idea/ .vs/ .e2e/ +.e2e-xray/ diff --git a/deploy/compose.e2e-xray.yml b/deploy/compose.e2e-xray.yml new file mode 100644 index 0000000..62e1b32 --- /dev/null +++ b/deploy/compose.e2e-xray.yml @@ -0,0 +1,93 @@ +name: ushere2exray +services: + # Reality target and plain HTTP page that the proxied requests fetch. + target: + image: nginx:1.27-alpine + networks: + e2e: + ipv4_address: 172.30.98.3 + volumes: + - ./e2e/xray/nginx.conf:/etc/nginx/conf.d/default.conf:ro + - ./e2e/xray/target.crt:/etc/nginx/certs/target.crt:ro + - ./e2e/xray/target.key:/etc/nginx/certs/target.key:ro + healthcheck: + test: ["CMD", "wget", "-q", "-O", "/dev/null", "http://127.0.0.1/"] + interval: 1s + retries: 30 + xray: + # The tag follows the xray-core version in go.mod. + image: ghcr.io/xtls/xray-core:26.10.10 + restart: unless-stopped + # With Docker's init as PID 1 the test can SIGKILL the Xray process alone. Killing PID 1 from + # inside is ignored, and "docker kill" marks the container as stopped by hand, which disables + # the restart policy. + init: true + # Root uid for the ports and the socket; the group equals usher's group. + user: "0:${E2E_GID:-0}" + command: ["run", "-config", "/etc/xray/config.json"] + depends_on: + target: + condition: service_healthy + networks: + e2e: + ipv4_address: 172.30.98.2 + volumes: + # The directory, not the file: the test replaces config.json and the certificate files. + - ../.e2e-xray/xray:/etc/xray:ro + # A bind mount that the test user owns: after a hard kill usher unlinks the stale socket, and + # that needs write access to the directory. + - ../.e2e-xray/sock:/run/xray + usher: + build: + context: .. + image: usher:e2e + user: ${E2E_UID:-0}:${E2E_GID:-0} + cap_drop: [ALL] + read_only: true + security_opt: [no-new-privileges:true] + depends_on: + xray: + condition: service_started + networks: [e2e] + volumes: + - ../.e2e-xray/config:/srv/usher/config:ro + - ../.e2e-xray/clients:/srv/usher/clients + - ../.e2e-xray/state:/srv/usher/state + - ../.e2e-xray/xray:/etc/xray:ro + - ../.e2e-xray/sock:/run/xray + environment: + USHER_HOST: 172.30.98.2 + USHER_XRAY_SOCKET: /run/xray/api.sock + USHER_XRAY_CONFIG: /etc/xray/config.json + # Client of one user; the test writes its config to ../.e2e-xray/client//config.json. The + # config has one HTTP proxy inbound per tested link, and the test reaches them at the fixed + # address. SSL_CERT_FILE stands in for the public CA that a real certificate chains to. + client-alice: + profiles: [client] + image: ghcr.io/xtls/xray-core:26.10.10 + command: ["run", "-config", "/etc/xray/config.json"] + networks: + e2e: + ipv4_address: 172.30.98.11 + volumes: + - ../.e2e-xray/client/alice:/etc/xray:ro + - ../.e2e-xray/client/ca.pem:/ca.pem:ro + environment: + SSL_CERT_FILE: /ca.pem + client-bob: + profiles: [client] + image: ghcr.io/xtls/xray-core:26.10.10 + command: ["run", "-config", "/etc/xray/config.json"] + networks: + e2e: + ipv4_address: 172.30.98.12 + volumes: + - ../.e2e-xray/client/bob:/etc/xray:ro + - ../.e2e-xray/client/ca.pem:/ca.pem:ro + environment: + SSL_CERT_FILE: /ca.pem +networks: + e2e: + ipam: + config: + - subnet: 172.30.98.0/24 diff --git a/deploy/e2e/xray/nginx.conf b/deploy/e2e/xray/nginx.conf new file mode 100644 index 0000000..c5c7293 --- /dev/null +++ b/deploy/e2e/xray/nginx.conf @@ -0,0 +1,22 @@ +# Test-only Reality target of the Xray e2e suite. The certificate and key in this directory are +# committed on purpose and protect nothing. +server { + listen 80; + default_type text/plain; + location / { + return 200 "usher-e2e-target\n"; + } +} + +server { + listen 443 ssl; + http2 on; + server_name target.e2e; + ssl_certificate /etc/nginx/certs/target.crt; + ssl_certificate_key /etc/nginx/certs/target.key; + ssl_protocols TLSv1.3; + default_type text/plain; + location / { + return 200 "usher-e2e-target\n"; + } +} diff --git a/deploy/e2e/xray/target.crt b/deploy/e2e/xray/target.crt new file mode 100644 index 0000000..5585436 --- /dev/null +++ b/deploy/e2e/xray/target.crt @@ -0,0 +1,10 @@ +-----BEGIN CERTIFICATE----- +MIIBeDCCAR2gAwIBAgIUTCI1efLW4GtS4+IlTcNaeybofbMwCgYIKoZIzj0EAwIw +FTETMBEGA1UEAwwKdGFyZ2V0LmUyZTAgFw0yNjEwMDkxMzQ4MDFaGA8yMTI2MDkx +NTEzNDgwMVowFTETMBEGA1UEAwwKdGFyZ2V0LmUyZTBZMBMGByqGSM49AgEGCCqG +SM49AwEHA0IABOqvJ3ybmrhGOcyOiShkBnfxvBHCfknrSn/9hfS4U56aN0sNSyGJ +MdgAEgrX3mM1VgmpV3vvONLDs3JhfIuQYIujSTBHMB0GA1UdDgQWBBQa9oksEEtx +GIw26VZAP7MAgbjq2TAPBgNVHRMBAf8EBTADAQH/MBUGA1UdEQQOMAyCCnRhcmdl +dC5lMmUwCgYIKoZIzj0EAwIDSQAwRgIhAKEExRx9zApYyGWqprdrWgpcs9D7pAOi +dkP3F+n48w+yAiEA5SBpddEgzsv1L53iOIQZ3OiSxjAzfip+xjYmnYfz8qc= +-----END CERTIFICATE----- diff --git a/deploy/e2e/xray/target.key b/deploy/e2e/xray/target.key new file mode 100644 index 0000000..74d7ce4 --- /dev/null +++ b/deploy/e2e/xray/target.key @@ -0,0 +1,5 @@ +-----BEGIN PRIVATE KEY----- +MIGHAgEAMBMGByqGSM49AgEGCCqGSM49AwEHBG0wawIBAQQgIljW3PlDFVEPLt2i +ME1RXkTqfByoBrcJIHqAA7zolBWhRANCAATqryd8m5q4RjnMjokoZAZ38bwRwn5J +60p//YX0uFOemjdLDUshiTHYABIK195jNVYJqVd77zjSw7NyYXyLkGCL +-----END PRIVATE KEY----- diff --git a/docs/plans/20261014-xray-config-backend.md b/docs/plans/20261014-xray-config-backend.md index ef42a79..d097bfe 100644 --- a/docs/plans/20261014-xray-config-backend.md +++ b/docs/plans/20261014-xray-config-backend.md @@ -466,14 +466,15 @@ reported by awg-grpc" and nothing reaches Xray. `c5a9f2c`, `e47b4cc`, `0993e07`, adapted) - Modify: `.github/workflows/ci.yml` -- [ ] server config with the four inbounds of Acceptance Evidence item 3, a test CA and leaves +- [x] server config with the four inbounds of Acceptance Evidence item 3, a test CA and leaves generated by the test, `x-usher` blocks, freedom `finalRules` allowing the target; config directory mount; Xray with `restart: unless-stopped`; Hysteria2 case first -- [ ] Xray client with `SSL_CERT_FILE` and `useSystemCA` -- [ ] each client connects with the rendered `.json` and with the `.txt` via +- [x] Xray client with `SSL_CERT_FILE` and `useSystemCA` +- [x] each client connects with the rendered `.json` and with the `.txt` via `ConvertShareLinksToXrayJson` -- [ ] assert the outcomes of Acceptance Evidence item 3 -- [ ] run the suite in the CI `e2e` job - must pass before Task 11 +- [x] assert the outcomes of Acceptance Evidence item 3 +- [x] run the suite in the CI `e2e` job - must pass before Task 11 (CI job added, not run: no + push; the suite passed on the Linux test host) ### Task 11: Verify acceptance criteria diff --git a/test/e2e/xray/client_test.go b/test/e2e/xray/client_test.go new file mode 100644 index 0000000..9fa6bd2 --- /dev/null +++ b/test/e2e/xray/client_test.go @@ -0,0 +1,237 @@ +//go:build e2e + +package xray + +import ( + "encoding/json" + "errors" + "fmt" + "io" + "net" + "net/http" + "net/url" + "os" + "path/filepath" + "strconv" + "strings" + "sync" + "testing" + "time" + + "github.com/xtls/libxray/share" +) + +const ( + targetURL = "http://" + targetHost + "/" + targetBody = "usher-e2e-target" + firstProxyPort = 8000 + requestTimeout = 5 * time.Second +) + +// proxyHosts maps each user to the address of that user's client container. +var proxyHosts = map[string]string{ + "alice": "172.30.98.11", + "bob": "172.30.98.12", +} + +// route is one way to reach the target: the client file of a user for an inbound, either the +// rendered .json config or the .txt share link. Each route has its own proxy port on the client. +type route struct { + tag string + form string + port int +} + +func (r route) String() string { return r.tag + r.formSuffix() } + +func (r route) formSuffix() string { return "." + r.form } + +// routes lists a route for every inbound and form, in the port order of the client config. +func routes() []route { + var all []route + for _, tag := range tags { + for _, form := range []string{"json", "txt"} { + all = append(all, route{tag: tag, form: form, port: firstProxyPort + len(all)}) + } + } + return all +} + +// routesOf returns the routes of one inbound. +func routesOf(tag string) []route { + var among []route + for _, r := range routes() { + if r.tag == tag { + among = append(among, r) + } + } + return among +} + +// writeClientConfig builds the Xray client config of a user from the rendered files of every +// inbound: one outbound and one HTTP proxy inbound per route. The test only adds useSystemCA to +// the TLS settings, so the client trusts SSL_CERT_FILE. +func writeClientConfig(t *testing.T, user string) { + t.Helper() + var inbounds, outbounds, rules []any + for _, r := range routes() { + outbound, err := routeOutbound(user, r) + if err != nil { + t.Fatalf("outbound of %s for %s: %v", r, user, err) + } + outboundTag := "out-" + strconv.Itoa(r.port) + inboundTag := "in-" + strconv.Itoa(r.port) + outbound["tag"] = outboundTag + if tls := nested(outbound, "streamSettings", "tlsSettings"); tls != nil { + tls["useSystemCA"] = true + } + inbounds = append(inbounds, map[string]any{ + "tag": inboundTag, "listen": "0.0.0.0", "port": r.port, "protocol": "http", + }) + outbounds = append(outbounds, outbound) + rules = append(rules, map[string]any{ + "type": "field", "inboundTag": []string{inboundTag}, "outboundTag": outboundTag, + }) + } + outbounds = append(outbounds, map[string]any{"tag": "block", "protocol": "blackhole"}) + config := map[string]any{ + "log": map[string]any{"loglevel": "warning"}, + "inbounds": inbounds, + "outbounds": outbounds, + "routing": map[string]any{"rules": rules}, + } + data, err := json.MarshalIndent(config, "", " ") + if err != nil { + t.Fatalf("encoding the client config of %s: %v", user, err) + } + if err := os.WriteFile(project.path(clientDir, user, "config.json"), data, 0o644); err != nil { + t.Fatalf("writing the client config of %s: %v", user, err) + } +} + +// routeOutbound returns the outbound that the rendered file of the route defines. +func routeOutbound(user string, r route) (map[string]any, error) { + data, err := os.ReadFile(project.path(clientsDir, user, r.tag+r.formSuffix())) + if err != nil { + return nil, err + } + if r.form == "txt" { + data, err = share.ConvertShareLinksToXrayJson(strings.TrimSpace(string(data)), "") + if err != nil { + return nil, fmt.Errorf("converting the share link: %w", err) + } + } + var config struct { + Outbounds []map[string]any `json:"outbounds"` + } + if err := json.Unmarshal(data, &config); err != nil { + return nil, fmt.Errorf("parsing the client file: %w", err) + } + if len(config.Outbounds) == 0 { + return nil, errors.New("the client file has no outbound") + } + return config.Outbounds[0], nil +} + +// nested returns the object at the path of keys, or nil when any step is missing. +func nested(object map[string]any, keys ...string) map[string]any { + for _, key := range keys { + next, ok := object[key].(map[string]any) + if !ok { + return nil + } + object = next + } + return object +} + +// waitForClientFiles waits until usher has written the .json and .txt of the user for every +// inbound. +func waitForClientFiles(t *testing.T, user string, timeout time.Duration) { + t.Helper() + waitFor(t, timeout, "client files of "+user, func() error { + for _, r := range routes() { + path := project.path(clientsDir, user, r.tag+r.formSuffix()) + info, err := os.Stat(path) + if err != nil { + return err + } + if info.Mode().Perm() != 0o600 { + return fmt.Errorf("mode of %s = %v, want 0600", filepath.Base(path), info.Mode().Perm()) + } + } + return nil + }) +} + +// fetch requests the target page through the proxy of the route and reports why the request did +// not return the page with status 200. +func fetch(user string, r route) error { + proxy, err := url.Parse("http://" + net.JoinHostPort(proxyHosts[user], strconv.Itoa(r.port))) + if err != nil { + return err + } + client := &http.Client{ + Timeout: requestTimeout, + Transport: &http.Transport{Proxy: http.ProxyURL(proxy), DisableKeepAlives: true}, + } + resp, err := client.Get(targetURL) + if err != nil { + return err + } + defer resp.Body.Close() + body, err := io.ReadAll(resp.Body) + if err != nil { + return err + } + if resp.StatusCode != http.StatusOK { + return fmt.Errorf("status %d, want 200", resp.StatusCode) + } + if !strings.Contains(string(body), targetBody) { + return fmt.Errorf("body %q does not come from the target", body) + } + return nil +} + +// fetchAll fetches through the given routes of a user at once and returns the error of each. +func fetchAll(user string, among []route) map[route]error { + var mu sync.Mutex + var wg sync.WaitGroup + results := make(map[route]error, len(among)) + for _, r := range among { + wg.Add(1) + go func() { + defer wg.Done() + err := fetch(user, r) + mu.Lock() + results[r] = err + mu.Unlock() + }() + } + wg.Wait() + return results +} + +// allWork returns nil when every route of the list returns the page. +func allWork(user string, among []route) error { + results := fetchAll(user, among) + var errs []error + for _, r := range among { + if err := results[r]; err != nil { + errs = append(errs, fmt.Errorf("%s of %s: %w", r, user, err)) + } + } + return errors.Join(errs...) +} + +// noneWorks returns nil when every route of the list fails to return the page. +func noneWorks(user string, among []route) error { + results := fetchAll(user, among) + var errs []error + for _, r := range among { + if results[r] == nil { + errs = append(errs, fmt.Errorf("%s of %s still works", r, user)) + } + } + return errors.Join(errs...) +} diff --git a/test/e2e/xray/pki_test.go b/test/e2e/xray/pki_test.go new file mode 100644 index 0000000..053f691 --- /dev/null +++ b/test/e2e/xray/pki_test.go @@ -0,0 +1,87 @@ +//go:build e2e + +package xray + +import ( + "crypto/ecdsa" + "crypto/elliptic" + "crypto/rand" + "crypto/x509" + "crypto/x509/pkix" + "encoding/pem" + "fmt" + "math/big" + "time" +) + +// authority is a throwaway CA that signs the server leaves. The client containers trust it through +// SSL_CERT_FILE, as they would trust the public CA of a real certificate. +type authority struct { + cert *x509.Certificate + key *ecdsa.PrivateKey + pem []byte +} + +func newAuthority() (*authority, error) { + key, err := ecdsa.GenerateKey(elliptic.P256(), rand.Reader) + if err != nil { + return nil, fmt.Errorf("generating the CA key: %w", err) + } + template := &x509.Certificate{ + SerialNumber: big.NewInt(1), + Subject: pkix.Name{CommonName: "usher e2e CA"}, + NotBefore: time.Now().Add(-time.Hour), + NotAfter: time.Now().Add(24 * time.Hour), + IsCA: true, + BasicConstraintsValid: true, + KeyUsage: x509.KeyUsageCertSign, + } + der, err := x509.CreateCertificate(rand.Reader, template, template, &key.PublicKey, key) + if err != nil { + return nil, fmt.Errorf("signing the CA certificate: %w", err) + } + cert, err := x509.ParseCertificate(der) + if err != nil { + return nil, fmt.Errorf("parsing the CA certificate: %w", err) + } + return &authority{cert: cert, key: key, pem: pem.EncodeToMemory(&pem.Block{Type: "CERTIFICATE", Bytes: der})}, nil +} + +// leaf is a certificate and its key, both PEM encoded. +type leaf struct { + cert, key []byte +} + +// issue signs a new leaf for the DNS name. Every call yields a different certificate and key from +// the same CA. +func (a *authority) issue(name string) (leaf, error) { + key, err := ecdsa.GenerateKey(elliptic.P256(), rand.Reader) + if err != nil { + return leaf{}, fmt.Errorf("generating the key of %s: %w", name, err) + } + serial, err := rand.Int(rand.Reader, new(big.Int).Lsh(big.NewInt(1), 62)) + if err != nil { + return leaf{}, fmt.Errorf("drawing a serial number: %w", err) + } + template := &x509.Certificate{ + SerialNumber: serial, + Subject: pkix.Name{CommonName: name}, + DNSNames: []string{name}, + NotBefore: time.Now().Add(-time.Hour), + NotAfter: time.Now().Add(24 * time.Hour), + KeyUsage: x509.KeyUsageDigitalSignature, + ExtKeyUsage: []x509.ExtKeyUsage{x509.ExtKeyUsageServerAuth}, + } + der, err := x509.CreateCertificate(rand.Reader, template, a.cert, &key.PublicKey, a.key) + if err != nil { + return leaf{}, fmt.Errorf("signing the certificate of %s: %w", name, err) + } + keyDER, err := x509.MarshalECPrivateKey(key) + if err != nil { + return leaf{}, fmt.Errorf("encoding the key of %s: %w", name, err) + } + return leaf{ + cert: pem.EncodeToMemory(&pem.Block{Type: "CERTIFICATE", Bytes: der}), + key: pem.EncodeToMemory(&pem.Block{Type: "EC PRIVATE KEY", Bytes: keyDER}), + }, nil +} diff --git a/test/e2e/xray/server_test.go b/test/e2e/xray/server_test.go new file mode 100644 index 0000000..bdde970 --- /dev/null +++ b/test/e2e/xray/server_test.go @@ -0,0 +1,184 @@ +//go:build e2e + +package xray + +import ( + "crypto/ecdh" + "crypto/rand" + "encoding/base64" + "encoding/json" + "fmt" + "os" + "path/filepath" + "strings" +) + +// The inbounds of the server config; usher.yml names them as interfaces. +const ( + tagHysteria = "hy2" + tagReality = "vless-reality" + tagXHTTP = "vless-xhttp" + tagWS = "vless-ws" +) + +// tags lists the inbounds in the order the suite connects to them, Hysteria2 first. +var tags = []string{tagHysteria, tagReality, tagXHTTP, tagWS} + +const ( + serverAddress = "172.30.98.2" + targetHost = "target" + targetAddress = "172.30.98.3" + // realityShortID is the shortIds entry of the config before the drift step changes it. + realityShortID = "ab12" + driftShortID = "cd34" +) + +// Where the server config and the files it names sit, relative to the config directory that +// Xray mounts at /etc/xray. +const ( + wsCertFile = "certs/ws.pem" + wsKeyFile = "certs/ws.key" +) + +// server holds what the generated server config needs: the Reality key and the leaves of the +// inline certificates. +type server struct { + realityPrivateKey string + hysteria, xhttp leaf + ws leaf +} + +func newServer(ca *authority) (*server, error) { + key, err := ecdh.X25519().GenerateKey(rand.Reader) + if err != nil { + return nil, fmt.Errorf("generating the Reality key: %w", err) + } + s := &server{realityPrivateKey: base64.RawURLEncoding.EncodeToString(key.Bytes())} + for name, target := range map[string]*leaf{"hy.e2e": &s.hysteria, "xh.e2e": &s.xhttp, "ws.e2e": &s.ws} { + if *target, err = ca.issue(name); err != nil { + return nil, err + } + } + return s, nil +} + +// pemLines splits a PEM block into the line list that Xray takes for an inline certificate. +func pemLines(data []byte) []string { + return strings.Split(strings.TrimSpace(string(data)), "\n") +} + +func inlineCertificate(l leaf, serverName string, alpn ...string) map[string]any { + return map[string]any{ + "serverName": serverName, + "alpn": alpn, + "certificates": []any{map[string]any{"certificate": pemLines(l.cert), "key": pemLines(l.key)}}, + } +} + +// config renders the server config with the four inbounds and the given Reality shortId. +func (s *server) config(shortID string) ([]byte, error) { + usherBlock := map[string]any{"address": serverAddress} + inbound := func(tag string, port int, protocol string, settings, stream map[string]any) map[string]any { + return map[string]any{ + "tag": tag, "listen": "0.0.0.0", "port": port, "protocol": protocol, + "settings": settings, "streamSettings": stream, "x-usher": usherBlock, + } + } + config := map[string]any{ + "log": map[string]any{"loglevel": "warning"}, + "api": map[string]any{ + "tag": "api", "listen": "/run/xray/api.sock,0660", "services": []string{"HandlerService"}, + }, + "inbounds": []any{ + inbound(tagHysteria, 4443, "hysteria", + map[string]any{"version": 2, "clients": []any{}}, + map[string]any{ + "network": "hysteria", + "security": "tls", + "tlsSettings": inlineCertificate(s.hysteria, "hy.e2e", "h3"), + "hysteriaSettings": map[string]any{"version": 2}, + "finalmask": map[string]any{"udp": []any{map[string]any{ + "type": "salamander", "settings": map[string]any{"password": "e2e-salamander"}, + }}}, + }), + inbound(tagReality, 443, "vless", + map[string]any{"clients": []any{}, "decryption": "none"}, + map[string]any{ + "network": "raw", + "security": "reality", + "realitySettings": map[string]any{ + "target": targetHost + ":443", + "serverNames": []string{"target.e2e"}, + "privateKey": s.realityPrivateKey, + "shortIds": []string{shortID}, + }, + }), + inbound(tagXHTTP, 2053, "vless", + map[string]any{"clients": []any{}, "decryption": "none"}, + map[string]any{ + "network": "xhttp", + "security": "tls", + "tlsSettings": inlineCertificate(s.xhttp, "xh.e2e", "h2", "http/1.1"), + "xhttpSettings": map[string]any{ + "mode": "auto", + "path": "/xh", + "sessionIDKey": "X-Sid", + "sessionIDPlacement": "header", + "xPaddingBytes": "2000-3000", + }, + }), + inbound(tagWS, 2057, "vless", + map[string]any{"clients": []any{}, "decryption": "none"}, + map[string]any{ + "network": "ws", + "security": "tls", + "tlsSettings": map[string]any{ + "serverName": "ws.e2e", + "alpn": []string{"http/1.1"}, + "certificates": []any{map[string]any{ + "certificateFile": "/etc/xray/" + wsCertFile, + "keyFile": "/etc/xray/" + wsKeyFile, + }}, + }, + "wsSettings": map[string]any{"path": "/ws"}, + }), + }, + // Freedom blocks private destinations unless a final rule allows them. + "outbounds": []any{map[string]any{ + "tag": "direct", "protocol": "freedom", + "settings": map[string]any{"finalRules": []any{ + map[string]any{"action": "allow", "ip": []string{targetAddress + "/32"}}, + }}, + }}, + } + return json.MarshalIndent(config, "", " ") +} + +// writeConfig replaces config.json in the mounted config directory. +func (s *server) writeConfig(dir, shortID string) error { + data, err := s.config(shortID) + if err != nil { + return fmt.Errorf("encoding the server config: %w", err) + } + return replaceFile(filepath.Join(dir, "config.json"), data) +} + +// writeWSCertificate replaces the certificate and key files of the ws inbound. +func (s *server) writeWSCertificate(dir string) error { + if err := os.MkdirAll(filepath.Join(dir, filepath.Dir(wsCertFile)), 0o755); err != nil { + return err + } + if err := replaceFile(filepath.Join(dir, wsKeyFile), s.ws.key); err != nil { + return err + } + return replaceFile(filepath.Join(dir, wsCertFile), s.ws.cert) +} + +// replaceFile writes data next to path and renames it over path, so no reader sees half a file. +func replaceFile(path string, data []byte) error { + tmp := path + ".tmp" + if err := os.WriteFile(tmp, data, 0o644); err != nil { + return err + } + return os.Rename(tmp, path) +} diff --git a/test/e2e/xray/xray_test.go b/test/e2e/xray/xray_test.go new file mode 100644 index 0000000..d1af9a8 --- /dev/null +++ b/test/e2e/xray/xray_test.go @@ -0,0 +1,440 @@ +//go:build e2e + +// Package xray drives usher against the official Xray image through deploy/compose.e2e-xray.yml. +// It needs Docker with compose and no kernel module; AGENTS.md, Test, runs it. +package xray + +import ( + "bytes" + "flag" + "fmt" + "os" + "os/exec" + "path/filepath" + "regexp" + "strconv" + "strings" + "testing" + "time" +) + +const ( + // repoDir is the repository root relative to this package; go test runs in the package dir. + repoDir = "../../.." + composeFile = "deploy/compose.e2e-xray.yml" + clientProfile = "client" + // helperImage is the image of the compose file that has a shell, kill and pidof. + helperImage = "nginx:1.27-alpine" + + // The deadlines follow the production constants: 3 s file poll, 500 ms settle, 30 s refill. + changeTimeout = 10 * time.Second + connectTimeout = 40 * time.Second + driftTimeout = 60 * time.Second + // restartTimeout and killTimeout are the recovery bounds of the acceptance evidence. + restartTimeout = 45 * time.Second + killTimeout = 90 * time.Second + // refillTick is a little longer than the 30 s refill interval. + refillTick = 35 * time.Second + // failedRounds is the number of consecutive failed rounds that prove a user is off. + failedRounds = 3 + pollInterval = 500 * time.Millisecond +) + +// The bind-mount directories of the compose project, under scratchRoot. +const ( + scratchRoot = ".e2e-xray" + configDir = "config" + clientsDir = "clients" + stateDir = "state" + sockDir = "sock" + xrayDir = "xray" + clientDir = "client" +) + +type composeProject struct{} + +var ( + project composeProject + // ca and srv are the fixtures that TestMain generates before the containers start. + ca *authority + srv *server +) + +// path returns a path under the scratch root. +func (composeProject) path(elem ...string) string { + return filepath.Join(append([]string{repoDir, scratchRoot}, elem...)...) +} + +// cmd builds docker compose on the project, run from the repository root with the uid and gid of +// the test user, so usher can write the bind mounts and open the socket. +func (composeProject) cmd(args ...string) *exec.Cmd { + cmd := exec.Command("docker", append([]string{"compose", "-f", composeFile}, args...)...) + cmd.Dir = repoDir + cmd.Env = append(os.Environ(), + "E2E_UID="+strconv.Itoa(os.Getuid()), + "E2E_GID="+strconv.Itoa(os.Getgid())) + return cmd +} + +// compose runs docker compose on the project and returns its stdout; a failure ends the test. +func (p composeProject) compose(t *testing.T, args ...string) string { + t.Helper() + var stdout, stderr bytes.Buffer + cmd := p.cmd(args...) + cmd.Stdout = &stdout + cmd.Stderr = &stderr + if err := cmd.Run(); err != nil { + t.Fatalf("docker compose %s: %v\nstdout:\n%s\nstderr:\n%s", + strings.Join(args, " "), err, stdout.String(), stderr.String()) + } + return stdout.String() +} + +func (p composeProject) down() { + out, err := p.cmd("--profile", clientProfile, "down", "-v", "--remove-orphans", "--timeout", "10").CombinedOutput() + if err != nil { + fmt.Fprintf(os.Stderr, "docker compose down: %v\n%s\n", err, out) + } +} + +func (p composeProject) printLogs() { + out, err := p.cmd("--profile", clientProfile, "logs", "--no-color").CombinedOutput() + fmt.Fprintf(os.Stderr, "docker compose logs (err %v):\n%s\n", err, out) +} + +func TestMain(m *testing.M) { + os.Exit(runSuite(m)) +} + +// runSuite creates the bind-mount directories as the test user, so dockerd does not create them as +// root and usher, which runs as the test user, can write them. It writes the server config and the +// CA before the containers start. A run that was killed leaves the project up, so it takes the +// project down first. +func runSuite(m *testing.M) int { + flag.Parse() + if count := flag.Lookup("test.count").Value.String(); count != "1" { + fmt.Fprintf(os.Stderr, "-count=%s: the suite restarts a service, so it runs once per process; use -count=1\n", count) + return 1 + } + project.down() + if err := os.RemoveAll(project.path()); err != nil { + fmt.Fprintf(os.Stderr, "removing %s: %v\n", scratchRoot, err) + return 1 + } + dirs := []string{configDir, clientsDir, stateDir, sockDir, xrayDir, + filepath.Join(clientDir, "alice"), filepath.Join(clientDir, "bob")} + for _, dir := range dirs { + if err := os.MkdirAll(project.path(dir), 0o755); err != nil { + fmt.Fprintf(os.Stderr, "creating %s: %v\n", dir, err) + return 1 + } + } + if err := writeFixtures(); err != nil { + fmt.Fprintln(os.Stderr, "writing fixtures:", err) + return 1 + } + defer project.down() + + out, err := project.cmd("up", "-d", "--build", "--wait", "usher").CombinedOutput() + fmt.Fprintf(os.Stderr, "docker compose up:\n%s\n", out) + if err != nil { + fmt.Fprintf(os.Stderr, "docker compose up: %v\n", err) + project.printLogs() + return 1 + } + code := m.Run() + if code != 0 { + project.printLogs() + } + return code +} + +func writeFixtures() error { + var err error + if ca, err = newAuthority(); err != nil { + return err + } + if srv, err = newServer(ca); err != nil { + return err + } + if err := os.WriteFile(project.path(clientDir, "ca.pem"), ca.pem, 0o644); err != nil { + return err + } + if err := srv.writeWSCertificate(project.path(xrayDir)); err != nil { + return err + } + return srv.writeConfig(project.path(xrayDir), realityShortID) +} + +// waitFor runs check until it returns nil or the timeout passes; then it ends the test with the +// last error. +func waitFor(t *testing.T, timeout time.Duration, what string, check func() error) { + t.Helper() + deadline := time.Now().Add(timeout) + var err error + for time.Now().Before(deadline) { + if err = check(); err == nil { + return + } + time.Sleep(pollInterval) + } + t.Fatalf("no %s within %v: %v", what, timeout, err) +} + +func usherLogs(t *testing.T) string { + t.Helper() + out, err := project.cmd("logs", "--no-color", "usher").CombinedOutput() + if err != nil { + t.Fatalf("docker compose logs usher: %v\n%s", err, out) + } + return string(out) +} + +// logMark is a position in the usher log, so a step can look at what usher logged after it. +type logMark int + +func markLogs(t *testing.T) logMark { + t.Helper() + return logMark(len(usherLogs(t))) +} + +// logsSince returns the usher log after the mark. +func logsSince(t *testing.T, mark logMark) string { + t.Helper() + logs := usherLogs(t) + if int(mark) > len(logs) { + return logs + } + return logs[mark:] +} + +// waitForLog waits until a line of the usher log after the mark matches the pattern. +func waitForLog(t *testing.T, mark logMark, timeout time.Duration, pattern string) { + t.Helper() + re := regexp.MustCompile(pattern) + waitFor(t, timeout, "usher log line "+pattern, func() error { + if !re.MatchString(logsSince(t, mark)) { + return fmt.Errorf("no line matches yet") + } + return nil + }) +} + +func usherYAML(users ...string) string { + var b strings.Builder + for _, user := range users { + fmt.Fprintf(&b, "%s: [%s]\n", user, strings.Join(tags, ", ")) + } + return b.String() +} + +// writeUsherConfig replaces config/usher.yml atomically. +func writeUsherConfig(t *testing.T, users ...string) { + t.Helper() + if err := replaceFile(project.path(configDir, "usher.yml"), []byte(usherYAML(users...))); err != nil { + t.Fatalf("writing usher.yml: %v", err) + } +} + +// startClient writes the client config of the user from the rendered files and starts or restarts +// the user's container, so no earlier connection survives. +func startClient(t *testing.T, user string) { + t.Helper() + waitForClientFiles(t, user, changeTimeout) + writeClientConfig(t, user) + project.compose(t, "--profile", clientProfile, "up", "-d", "--force-recreate", "client-"+user) +} + +// assertOffRepeatedly requires several consecutive rounds in which no route of the user works, so +// one transient error cannot stand for a removed user. +func assertOffRepeatedly(t *testing.T, user string, among []route) { + t.Helper() + for range failedRounds { + if err := noneWorks(user, among); err != nil { + t.Fatal(err) + } + time.Sleep(time.Second) + } +} + +// removedLine matches the usher log line that reports the removal of a user from an inbound. +func removedLine(tag, user string) string { + return fmt.Sprintf(`users changed" inbound=%s .*removed="?\[[^\]]*%s/%s[ \]]`, tag, tag, user) +} + +// addedLine matches the usher log line that reports the addition of a user to an inbound. +func addedLine(tag, user string) string { + return fmt.Sprintf(`users changed" inbound=%s added="?\[[^\]]*%s/%s[ \]]`, tag, tag, user) +} + +// TestXray runs its subtests in order; each one starts from the state the previous one left. +func TestXray(t *testing.T) { + steps := []struct { + name string + run func(t *testing.T) + }{ + {"connect", testConnect}, + {"switch off", testSwitchOff}, + {"restart", testRestart}, + {"hard kill", testHardKill}, + {"drift", testDrift}, + {"certificate rewrite", testCertificateRewrite}, + } + for _, step := range steps { + if !t.Run(step.name, step.run) { + t.FailNow() + } + } +} + +// testConnect writes a config of two users, builds a client from the rendered files of each and +// fetches the target page through every .json and .txt. +func testConnect(t *testing.T) { + waitFor(t, changeTimeout, "the API socket of xray", func() error { + _, err := os.Stat(project.path(sockDir, "api.sock")) + return err + }) + writeUsherConfig(t, "alice", "bob") + for _, user := range []string{"alice", "bob"} { + startClient(t, user) + } + for _, user := range []string{"alice", "bob"} { + waitFor(t, connectTimeout, "pages through the files of "+user, func() error { + return allWork(user, routes()) + }) + } +} + +// testSwitchOff removes bob from usher.yml. Once usher logs the removal from every inbound, a new +// connection through any file of bob must keep failing while alice still works. +func testSwitchOff(t *testing.T) { + mark := markLogs(t) + writeUsherConfig(t, "alice") + for _, tag := range tags { + waitForLog(t, mark, changeTimeout, removedLine(tag, "bob")) + } + // Restarting the client drops connections that bob opened before the removal; the removal + // holds for new connections. + project.compose(t, "--profile", clientProfile, "restart", "client-bob") + assertOffRepeatedly(t, "bob", routes()) + waitFor(t, changeTimeout, "pages through the files of alice after bob was removed", func() error { + return allWork("alice", routes()) + }) +} + +// testRestart restarts xray, which empties its user list. usher must put alice back on its refill +// tick and leave bob out. +func testRestart(t *testing.T) { + project.compose(t, "restart", "xray") + waitFor(t, restartTimeout, "pages through the files of alice after the restart", func() error { + return allWork("alice", routes()) + }) + assertOffRepeatedly(t, "bob", routes()) +} + +// testHardKill kills the xray process with SIGKILL, which leaves its socket behind. The restart policy starts +// xray again, it cannot bind the stale socket, and usher unlinks the socket at its next tick. +func testHardKill(t *testing.T) { + mark := markLogs(t) + killXray(t) + waitFor(t, killTimeout, "pages through the files of alice after the kill", func() error { + return allWork("alice", routes()) + }) + if !strings.Contains(logsSince(t, mark), "removed stale xray socket") { + t.Fatal("usher did not log that it removed the stale xray socket") + } + assertOffRepeatedly(t, "bob", routes()) +} + +// killXray sends SIGKILL to the Xray process from a helper container in the PID namespace of the +// xray container. The container exits with status 137, as after a crash or an OOM kill, so the +// restart policy applies. +func killXray(t *testing.T) { + t.Helper() + id := strings.TrimSpace(project.compose(t, "ps", "-q", "xray")) + if id == "" { + t.Fatal("no xray container") + } + out, err := exec.Command("docker", "run", "--rm", "--pid=container:"+id, + "--entrypoint", "sh", helperImage, "-c", "kill -9 $(pidof xray)").CombinedOutput() + if err != nil { + t.Fatalf("killing xray: %v\n%s", err, out) + } +} + +// testDrift changes shortIds in the mounted config without restarting xray. The reality inbound +// must fail with a drift reason at the next tick, the other inbounds must not, and a user removed +// from usher.yml while the drift is present must still be removed from xray. +func testDrift(t *testing.T) { + mark := markLogs(t) + writeUsherConfig(t, "alice", "bob") + for _, tag := range tags { + waitForLog(t, mark, restartTimeout, addedLine(tag, "bob")) + } + startClient(t, "bob") + waitFor(t, connectTimeout, "pages through the files of bob after he came back", func() error { + return allWork("bob", routes()) + }) + + mark = markLogs(t) + if err := srv.writeConfig(project.path(xrayDir), driftShortID); err != nil { + t.Fatal(err) + } + waitForLog(t, mark, driftTimeout, + `interface failed" backend=xray interface=`+tagReality+` .*differs from the running Xray in`) + for _, tag := range failedInterfaces(logsSince(t, mark)) { + if tag != tagReality { + t.Fatalf("inbound %s failed, only %s has drift", tag, tagReality) + } + } + + mark = markLogs(t) + writeUsherConfig(t, "alice") + waitForLog(t, mark, changeTimeout, removedLine(tagReality, "bob")) + project.compose(t, "--profile", clientProfile, "restart", "client-bob") + assertOffRepeatedly(t, "bob", routesOf(tagReality)) + waitFor(t, changeTimeout, "pages through the reality files of alice under drift", func() error { + return allWork("alice", routesOf(tagReality)) + }) + + // With the file back in step with xray, the next tick has no failure. + if err := srv.writeConfig(project.path(xrayDir), realityShortID); err != nil { + t.Fatal(err) + } + time.Sleep(2 * time.Second) + mark = markLogs(t) + time.Sleep(refillTick) + if failed := logsSince(t, mark); strings.Contains(failed, "interface failed") { + t.Fatalf("a tick failed after the drift was undone:\n%s", failed) + } +} + +// failedInterfaces returns the interface of every "interface failed" line of the log. +func failedInterfaces(logs string) []string { + var failed []string + for _, match := range regexp.MustCompile(`interface failed" backend=xray interface=(\S+)`).FindAllStringSubmatch(logs, -1) { + failed = append(failed, match[1]) + } + return failed +} + +// testCertificateRewrite replaces the certificateFile of the ws inbound with a new leaf of the +// same CA. That is no drift, and the inbound keeps serving. +func testCertificateRewrite(t *testing.T) { + fresh, err := ca.issue("ws.e2e") + if err != nil { + t.Fatal(err) + } + srv.ws = fresh + mark := markLogs(t) + if err := srv.writeWSCertificate(project.path(xrayDir)); err != nil { + t.Fatal(err) + } + time.Sleep(refillTick) + if failed := logsSince(t, mark); strings.Contains(failed, "interface failed") { + t.Fatalf("a tick failed after the certificate was rewritten:\n%s", failed) + } + waitFor(t, restartTimeout, "a page through ws after the rewrite", func() error { + return allWork("alice", routesOf(tagWS)) + }) +} From 0155ff4d58cba005bf33355671295b05f3518802 Mon Sep 17 00:00:00 2001 From: EvaTheSalmon <18466855+EvaTheSalmon@users.noreply.github.com> Date: Wed, 14 Oct 2026 18:45:47 +0300 Subject: [PATCH 17/32] docs(plans): verify xray backend acceptance Co-Authored-By: Claude Sonnet 5.5 --- docs/plans/20261014-xray-config-backend.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/docs/plans/20261014-xray-config-backend.md b/docs/plans/20261014-xray-config-backend.md index d097bfe..708974d 100644 --- a/docs/plans/20261014-xray-config-backend.md +++ b/docs/plans/20261014-xray-config-backend.md @@ -478,12 +478,12 @@ reported by awg-grpc" and nothing reaches Xray. ### Task 11: Verify acceptance criteria -- [ ] verify all requirements from Overview are implemented -- [ ] verify edge cases are handled -- [ ] run full test suite: `go test -race -count=1 ./...` -- [ ] run `scripts/format.sh`, shellcheck and actionlint -- [ ] run e2e tests: `go test -tags e2e -count=1 -v ./test/e2e/...` (CI) -- [ ] verify test coverage meets project standard +- [x] verify all requirements from Overview are implemented +- [x] verify edge cases are handled +- [x] run full test suite: `go test -race -count=1 ./...` +- [x] run `scripts/format.sh`, shellcheck and actionlint +- [x] run e2e tests: `go test -tags e2e -count=1 -v ./test/e2e/...` (CI) (awg e2e deferred to CI, not run locally; Xray e2e passed on ger in Task 10, code unchanged since) +- [x] verify test coverage meets project standard (go test -cover: xray 90.9%, awg 96.6%, config 97.6%, pass 97.9%, state 91.0%, watch 100%, clients 78.9%, cmd/usher 73.2%, atomicfile 61.5%) ### Task 12: [Final] Update documentation From 0ccfeeeca708ece874f11702e05501c69e3b253a Mon Sep 17 00:00:00 2001 From: EvaTheSalmon <18466855+EvaTheSalmon@users.noreply.github.com> Date: Wed, 14 Oct 2026 18:48:32 +0300 Subject: [PATCH 18/32] docs(xray): document the Xray backend Co-Authored-By: Claude Sonnet 5.5 --- AGENTS.md | 36 ++++-- README.md | 133 +++++++++++++++++++-- docs/architecture/overview.md | 115 ++++++++++++++++-- docs/plans/20261014-xray-config-backend.md | 8 +- 4 files changed, 259 insertions(+), 33 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 6bbf520..047fcea 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,8 +1,8 @@ # usher -Container that reads `config/usher.yml`, a map of user name to AmneziaWG interface names, and keeps -the peers of those interfaces and the client configs in `clients/` in step with it through the -`awg-grpc` unix socket; `README.md` describes it. Go module `github.com/mrcsin/usher`; entry point +Container that reads `config/usher.yml`, a map of user name to interface names, and keeps the +AmneziaWG peers, the Xray users and the client files in `clients/` in step with it through the +`awg-grpc` and Xray API unix sockets; `README.md` describes it. Go module `github.com/mrcsin/usher`; entry point `cmd/usher`. All commands run from the repository root. ## Build @@ -24,7 +24,8 @@ go test -race ./... Table-driven tests sit beside the source as `*_test.go`; filesystem tests use `t.TempDir()`. The `awg-grpc` API is faked by a struct with func fields that implements -`awgv1.ManagementServiceClient`. +`awgv1.ManagementServiceClient`; the Xray API by a struct that implements +`command.HandlerServiceClient`. The e2e suite in `test/e2e` carries `//go:build e2e`. It needs Docker and a host with the `amneziawg` module loaded, so it runs on a Linux host, not on macOS. It uses the published @@ -40,6 +41,17 @@ The suite needs `docker compose` without `sudo`. It wipes and recreates the bind under `.e2e/` (gitignored) at start, and passes `E2E_UID` and `E2E_GID` to compose so usher writes them as the test user. +The Xray suite in `test/e2e/xray` (`deploy/compose.e2e-xray.yml`) needs only Docker and no kernel +module. The test generates a CA, the server certificates and the Reality key, and starts the +official Xray image. It also kills Xray's process from a helper container in its PID namespace, +because `docker kill` bypasses the restart policy: + +```sh +go test -tags e2e -count=1 -v ./test/e2e/xray/ +``` + +It uses `.e2e-xray/` (gitignored) for its bind mounts. CI runs it in the `e2e` job. + To copy the tree from macOS to a Linux host, use `COPYFILE_DISABLE=1 tar --no-xattrs`: plain macOS tar adds `._*` AppleDouble files. A run killed by a signal leaves containers up; remove them with: @@ -82,6 +94,11 @@ the notes of the annotated tag. - A private key, a preshared key or a `client_params` value is never logged, and an error message never quotes one. A public key without an entry is logged as base64. +- A Reality `privateKey` or `mldsa65Seed`, a `certificateFile` or `keyFile`, a PEM key, `sockopt`, + a server `fallbacks` entry and a drift value never reach `clients/`, a log line or an error. A + drift reason names the differing part only. +- usher never writes the Xray server config. It owns the users of the managed inbounds and + `clients/`. - `state/users.json` is the only source of secrets. The kernel and `clients/` derive from it, and usher saves it before `ApplyPeers`. - A failure of one interface never stops the daemon; the pass logs it and goes on. @@ -94,11 +111,14 @@ cmd/usher/ entry point: environment, signals, run loop wiring internal/config/ usher.yml loader and validation internal/state/ users.json load and save internal/atomicfile/ atomic file replace shared by state and the clients mirror -internal/pass/ settings, enrollment, address allocation, the reconcile pass -internal/clients/ client config rendering and the clients/ mirror +internal/pass/ settings, the Backend/Session seam, the reconcile pass +internal/awg/ awg backend: enrollment, address allocation, .conf rendering +internal/xray/ Xray backend: config reader, drift guard, users, mirror, rendering, stale socket +internal/clients/ client file paths and the clients/ mirror internal/watch/ file poll, refill ticker and the trigger loop -deploy/ example compose file, e2e compose file and interface config -test/e2e/ end-to-end test through docker compose (tag e2e) +deploy/ example compose file, example Xray config, e2e compose files +test/e2e/ awg end-to-end test through docker compose (tag e2e) +test/e2e/xray/ Xray end-to-end test (tag e2e) scripts/format.sh the Format chain (see Format) scripts/git-hooks/ pre-commit hook: branch guard, then scripts/format.sh scripts/ci/module.sh builds and loads the pinned kernel module in the CI e2e job diff --git a/README.md b/README.md index 08e0fe4..02e8fa2 100644 --- a/README.md +++ b/README.md @@ -10,12 +10,12 @@ User management for the VPN services of one node. You list users and the interfaces each may use in one file; within seconds usher creates their keys, puts them into the running services and writes a ready client config per user and interface. Addresses and keys stay internal: you type -names only. +names only. An interface is an AmneziaWG interface or an inbound of your Xray config. usher holds no capabilities and is never in the packet path. Every VPN service runs in its own container with a small management API on a unix socket; usher calls that API and nothing else, so a crash or an upgrade of usher leaves every session running. AmneziaWG is supported through -[awg-grpc](https://github.com/mrcsin/awg-grpc); Xray is next. +[awg-grpc](https://github.com/mrcsin/awg-grpc), Xray through its own gRPC API. ## Users @@ -33,25 +33,134 @@ bob: [] - A broken file changes nothing: usher keeps the last valid one and logs the error with its line. - A service that restarts empty is refilled within 30 s without any change to the file. -Client configs appear in `clients//.conf`. Secrets live in `state/users.json`, -the only copy of every key: keep the state volume in the backup set. +Client files appear in `clients//`: `.conf` for AmneziaWG, `.json` and `.txt` +for Xray. Secrets live in `state/users.json`, the only copy of every key: keep the state volume in +the backup set. ## Running -[`deploy/compose.example.yml`](deploy/compose.example.yml) runs usher next to awg-grpc. Images are -`ghcr.io/mrcsin/usher:`, one per release. +[`deploy/compose.example.yml`](deploy/compose.example.yml) runs usher next to awg-grpc and Xray. +Images are `ghcr.io/mrcsin/usher:`, one per release. Drop the service you do not use. -| Variable | Example | Meaning | -| ------------ | ----------------- | ----------------------------------------------- | -| `USHER_HOST` | `203.0.113.10` | IPv4 address clients connect to; no host names | -| `USHER_DNS` | `1.1.1.1,1.0.0.1` | DNS servers written into client configs | +| Variable | Example | Meaning | +| -------------------- | ------------------------ | ---------------------------------------------------- | +| `USHER_HOST` | `203.0.113.10` | IPv4 address clients connect to; no host names | +| `USHER_AWG_SOCKET` | `/run/awg-grpc/awg.sock` | awg-grpc socket; enables the AmneziaWG backend | +| `USHER_DNS` | `1.1.1.1,1.0.0.1` | DNS servers in AmneziaWG configs; needs the awg socket | +| `USHER_XRAY_SOCKET` | `/run/xray/api.sock` | Xray API socket; enables the Xray backend | +| `USHER_XRAY_CONFIG` | `/etc/xray/config.json` | Xray server config, read-only; needs the Xray socket | -Both are required. Publish each interface port 1:1, and give usher the socket group of the -service containers. +`USHER_HOST` is always required. Set at least one socket. Publish each interface port 1:1, and give +usher the socket group of the service containers. To issue new keys for a user, stop usher, delete its `/` entry from `users.json` and start it again. +## Xray + +You write the Xray server config and usher fills its inbounds with users. An inbound tag is the +interface name in `usher.yml`, up to 32 characters. usher reads the config file and puts users +into the running inbounds through Xray's `HandlerService`. It never writes the config. Xray keeps +users in memory only, so usher refills them after a restart within 30 s. + +Per user and inbound usher writes `clients//.json`, a full client config with a local +socks proxy on 127.0.0.1:10808 and an http proxy on 127.0.0.1:10809, and `clients//.txt`, +a share link. + +### Server config + +- One JSON file; comments are allowed. No `-confdir`. +- Mount the directory that holds the file into both containers, not the file. usher then sees a + replaced file at its next refill tick. +- `api.services` is `["HandlerService"]` and `api.listen` is a unix path with `,0660`, such as + `/run/xray/api.sock,0660`. The Xray process group must equal usher's group. +- The socket directory is a bind mount that usher can write. After a hard kill Xray leaves a stale + socket and fails to start on it. usher unlinks the socket at its next tick, and Xray's restart + policy (`restart: unless-stopped`) starts it again. +- The Xray image follows `xray-core` in `go.mod` (now `ghcr.io/xtls/xray-core:26.10.10`). Another + version can change how Xray builds an inbound and put every inbound in drift. +- Every managed inbound has a tag, a single integer `port`, no unix `listen` and an empty + `clients` list. A Hysteria2 inbound also has an empty `hysteriaSettings.auth`; usher gives each + user an auth. VLESS `decryption` is `none`. + +[`deploy/xray/config.json`](deploy/xray/config.json) is an example with placeholders that Xray +refuses to start with. + +### Supported shapes + +| Part | Values | +| --------- | --------------------------------------------------------------- | +| Protocol | VLESS, Hysteria2, VMess, Trojan | +| Transport | raw (header none), ws, grpc, httpupgrade, xhttp, hysteria | +| Security | none, tls, reality | +| Mask | salamander (Hysteria2, without `packetSize`), `quicParams` | + +Shadowsocks and WireGuard inbounds are ignored. VMess and Trojan are covered by unit tests only. +An inbound outside this table fails; the log names the field. Its existing client files stay. + +### The x-usher block + +Add `x-usher` to an inbound to tell usher what clients need. Xray ignores the key. Every field is +optional, and an unknown key fails the inbound. + +```json +"x-usher": { + "address": "203.0.113.10", + "port": 443, + "sni": "example.com", + "fingerprint": "chrome", + "trust": "ca", + "pin_sha256": "…", + "hop_ports": "20000-30000" +} +``` + +| Field | Default | Meaning | +| ------------- | ------------------------------------ | ---------------------------------------------------- | +| `address` | `USHER_HOST` | address clients dial | +| `port` | the inbound `port` | port clients dial | +| `sni` | `tlsSettings.serverName`, or the first Reality `serverNames` entry without `*` | name clients send | +| `fingerprint` | `chrome` | uTLS fingerprint; Reality allows `chrome`, `firefox`, `safari` | +| `trust` | `ca` | `ca` or `pin`, see below | +| `pin_sha256` | hash of the inline certificate | hex SHA-256 of the leaf; only with `trust: pin` | +| `hop_ports` | none | Hysteria2 port range `from-to`, from < to | + +A TLS inbound with neither `sni` nor `tlsSettings.serverName` fails. An explicit `sni` also +replaces the Reality `serverNames` rule, for a config that lists only wildcard names. + +### TLS trust + +- `ca`: the client verifies the server certificate against its CA roots and gets only the + server name, the ALPN list and the fingerprint. Use a certificate that a public CA issued for + your domain. +- `pin`: the client also gets `pinnedPeerCertSha256`, from `pin_sha256` or from an inline + `certificates[].certificate`. An inbound with neither fails. A Hysteria2 pin inbound gets no + `.txt`, because v2rayNG and Hiddify do not read the pin from a `hysteria2://` link. + +Reality needs no certificate. usher derives the public key from `privateKey`, takes `shortIds[0]` +and sets `spiderX` to `/`. With `mldsa65Seed` it adds `mldsa65Verify`. Private keys, seeds, key +files and fallbacks never reach `clients/` or the log. + +### Client compatibility + +Measured on 2026-10-14 against Xray 26.10.10. + +- Reality links work in Xray-core clients only. sing-box 1.14.3 fails the server's + `X25519MLKEM768` requirement. Only `chrome`, `firefox` and `safari` carry it. +- xhttp is not in upstream sing-box. +- VLESS ws, Trojan, VMess and Hysteria2 with salamander work in sing-box. +- A user in `usher.yml` that loses an interface loses new connections at once. An open Hysteria2 + QUIC connection stays up until the client reconnects. +- `hop_ports` needs a DNAT on the host from the range to the inbound port. usher does not set it. + +### Config edits + +usher compares each inbound with the running Xray at every pass. After you edit a managed inbound +in the file, restart Xray. Until then usher logs a drift reason that names the part (port, proxy +settings, stream settings), keeps the old client files and still removes users who left +`usher.yml`. Certificate files that the config names are reloaded by Xray itself and cause no +drift. + ## License [MIT](LICENSE) diff --git a/docs/architecture/overview.md b/docs/architecture/overview.md index a92157e..b190986 100644 --- a/docs/architecture/overview.md +++ b/docs/architecture/overview.md @@ -2,29 +2,40 @@ ## Purpose -usher keeps AmneziaWG peers and client configs in step with `config/usher.yml`, a map of user name -to interface names. The operator types names only. Addresses, keys and the subnet are internal. +usher keeps AmneziaWG peers, Xray users and client files in step with `config/usher.yml`, a map of +user name to interface names. An interface is an AmneziaWG interface or a tagged inbound of the +operator's Xray config. The operator types names only. Addresses and keys are internal. ## Stack Go, one binary: `usher run`. It talks gRPC to `awg-grpc` (`github.com/mrcsin/awg-grpc` v0.1.0) -over the unix socket `/run/awg-grpc/awg.sock`. It runs in a container with no capabilities, a -read-only root file system and uid 1000. Config parsing uses `gopkg.in/yaml.v3`. +over the socket in `USHER_AWG_SOCKET`, and to Xray's `HandlerService` over the socket in +`USHER_XRAY_SOCKET`. At least one is set. It runs in a container with no capabilities, a read-only +root file system and uid 1000. Config parsing uses `gopkg.in/yaml.v3`. The Xray backend imports +`xray-core` (`infra/conf`, `app/proxyman/command`) and `libxray/share`. ## Components | Package | Responsibility | |---|---| -| `cmd/usher` | Reads `USHER_HOST` and `USHER_DNS`, wires signals, the watcher and the pass. | +| `cmd/usher` | Reads the `USHER_*` environment, builds the backends, wires signals, the watcher and the pass. | | `internal/config` | Loads and validates `usher.yml`. | -| `internal/state` | Loads and saves `state/users.json`. | +| `internal/state` | Loads and saves `state/users.json`, version 2. | | `internal/atomicfile` | Replaces a file atomically for `state` and the `clients/` mirror. | -| `internal/pass` | Settings, enrollment, address allocation and the reconcile pass. | -| `internal/clients` | Renders client configs and mirrors the `clients/` directory. | +| `internal/pass` | Settings, the `Backend` and `Session` seam and the reconcile pass. | +| `internal/awg` | The awg backend: enrollment, address allocation, `.conf` rendering. | +| `internal/xray` | The Xray backend: config reader, drift guard, users, mirror, `.json` and `.txt` rendering, stale socket recovery. | +| `internal/clients` | Paths of client files and the mirror of the `clients/` directory. | | `internal/watch` | Polls the file, runs the refill ticker and serializes passes. | ## The pass +A backend is a `Backend{Name, Suffixes, Open}`. `Open` returns a `Session` with `Interfaces`, +`Prepare` (enroll users, render files), `Apply` and `Close`. `Suffixes` lists the file suffixes +the backend owns (`.conf`; `.json` and `.txt`). A backend that fails to open is down for the pass: +usher keeps every file with one of its suffixes. An interface name that two backends report fails +in both. The steps below run per backend; the text names the awg calls. + 1. Load `usher.yml`. When it is invalid, use the last valid file of this process. With none, log and stop. Load `users.json`. On error, log and stop. 2. Call `GetStatus`. On error, log and stop. @@ -61,7 +72,7 @@ pass and every call has a 10 s deadline. ## Locked decisions -- The scope is AmneziaWG over IPv4. The `users.json` grouping leaves room for Xray. +- The scope is AmneziaWG over IPv4 and the Xray shapes in the README. - usher owns every interface the wrapper reports, and an unreferenced interface gets an empty list. - A broken `usher.yml` never changes state. The last valid file is kept in memory only. @@ -94,3 +105,89 @@ pass and every call has a 10 s deadline. `FORWARD` drops between clients in `PostUp`; a resolver on the tunnel address (for example cloudflared with a DoH upstream in the `awg` network namespace) with `USHER_DNS` set to that address, and optionally a `DNAT` of port 53 in `PostUp`. + +## Xray backend + +### Ownership + +- The operator owns one Xray server config: inbounds, outbounds, routing, `api`, certificates, + Reality keys, and a per-inbound `x-usher` block with client hints. Xray reads it at start. usher + reads the same file read-only at every pass. +- usher owns the users inside the inbounds, through the API, and everything under `clients/`. + Xray keeps users in memory, so the refill tick restores them after a restart. +- A managed inbound has a tag that matches the interface name pattern (32 characters), one integer + port, no unix `listen`, empty `clients` and a protocol of vless, hysteria, vmess or trojan. + Other inbounds are skipped. An inbound that fails validation carries its error: `Prepare` + returns it, the pass skips its `Apply` and keeps its files. + +### Reading the config + +`ReadConfig` reads the file with Xray's comment-aware reader into `infra/conf` types plus raw +maps. It normalizes the aliases (`method`, `splithttp`, `websocket`, `tcp`, `splithttpSettings`, +`tcpSettings`, Hysteria `users`) before validation and mirroring. Every error names the field and +never a value. `x-usher` decodes with unknown keys rejected. An explicit `x-usher.sni` relaxes the +Reality `serverNames` rule. `hop_ports` requires from < to. ReadConfig registers a discard handler +for Xray's global logger once, because `Build()` prints deprecation warnings. + +### Users + +- The email is `tag/user`, level 0. State keeps `id` (UUID v4) for VLESS and VMess, `password` + (base64url of 32 random bytes) for Trojan and Hysteria2, under the `xray` group. enroll never + changes an existing entry. +- `Apply` lists all users of the inbound with `GetInboundUsers` and compares the credential only: + VLESS `id` and `flow`, VMess `id`, Trojan `password`, Hysteria `auth`. It removes foreign and + changed emails and adds missing and changed ones. It tries every operation, joins the errors and + logs `users changed`. The next pass converges from the listed state. +- Removal stops new connections. An open Hysteria2 QUIC connection stays until the client + reconnects. + +### Drift guard + +`Open` calls `ListInbounds` once and compares each valid inbound with `Build()` of its file entry: +proxy type, proxy settings, port, stream settings and receiver settings. Certificate entries that +name a file are dropped on both sides, because usher cannot read the key files and Xray reloads +them. A tag missing at runtime counts as drift. The reason names the differing parts. `Apply` +reconciles users first and then returns the drift error, so revocation works while the old client +files stay. A running managed inbound that the file does not define is logged each pass. + +### Stale socket + +A killed Xray leaves its API socket, and a new start fails on it. Xray holds `flock(LOCK_EX)` on +`.lock` while it listens. Before dialing, usher opens the lock read-only and tries +`LOCK_EX|LOCK_NB`. When it gets the lock, no Xray listens: usher unlinks the socket and reports +Xray down for this pass. The restart policy starts Xray at its next attempt. The socket directory +is a bind mount that usher can write. + +### Client mirror + +- Taken from the server: `network`, `security`, path and host of ws and httpupgrade, + `serviceName` and `authority` of grpc. Everything else of the stream settings is dropped. +- xhttp: path, host and mode stay at the top. A server `extra` is copied verbatim; otherwise every + other field moves into `extra`. `.json` and `.txt` carry the same form. `downloadSettings` fails + the inbound. +- VLESS `flow` is `xtls-rprx-vision` when the network is raw and the security is tls or reality. +- `tlsSettings` is rebuilt: server name, ALPN, fingerprint and, with `trust: pin`, + `pinnedPeerCertSha256`. +- `realitySettings` is rebuilt: public key from `privateKey` (`crypto/ecdh`), `shortIds[0]`, + `spiderX` `/`, `mldsa65Verify` from `mldsa65Seed`. The fingerprint is `chrome`, `firefox` or + `safari`. +- Hysteria2: the client `auth` is the user's password. From salamander only `password` is copied; + `packetSize` fails the inbound. `hop_ports` adds a client `udphop` mask (mode + `intervalLocal,intervalRemote`, interval 30, in libxray's shape). The range needs a host DNAT to + the inbound port. + +### Outputs + +- `clients//.json` is a complete client config: the outbound plus a socks inbound on + 127.0.0.1:10808 and an http inbound on 127.0.0.1:10809. The outbound must pass + `conf.OutboundDetourConfig.Build()` before any file is written, or the inbound fails. +- `clients//.txt` comes from libxray `ConvertXrayJsonToShareLinks`. usher builds only + shapes libxray knows, so a refusal is a usher defect: it is logged with the tag and no `.txt` is + written. A Hysteria2 inbound with `trust: pin` gets no `.txt`. + +## State + +`state/users.json` is version 2: the `awg` group keyed `interface/user` and the `xray` group keyed +`tag/user`. An `xray` entry holds exactly one of `id` and `password`. `Load` accepts versions 1 +and 2. Before the first save of a version 1 file usher writes `users.json.v1` beside it, so a +rollback is a file rename. diff --git a/docs/plans/20261014-xray-config-backend.md b/docs/plans/20261014-xray-config-backend.md index 708974d..93b39bb 100644 --- a/docs/plans/20261014-xray-config-backend.md +++ b/docs/plans/20261014-xray-config-backend.md @@ -487,14 +487,14 @@ reported by awg-grpc" and nothing reaches Xray. ### Task 12: [Final] Update documentation -- [ ] README: delivery contract (image follows `go.mod`, directory mount, restart policy, socket +- [x] README: delivery contract (image follows `go.mod`, directory mount, restart policy, socket directory), `x-usher` block, TLS trust rule, outputs, supported shapes, client compatibility (Reality in Xray-core clients only, xhttp not in upstream sing-box), the Hysteria2 removal limit, the hop DNAT prerequisite, config edits seen at the refill tick -- [ ] `docs/architecture/overview.md`: who owns which JSON, the seam, the mirror, drift guard, +- [x] `docs/architecture/overview.md`: who owns which JSON, the seam, the mirror, drift guard, state v2, the stale socket rule -- [ ] `AGENTS.md`: layout, the Xray e2e, never-logged fields -- [ ] move this plan to `docs/plans/completed/` +- [x] `AGENTS.md`: layout, the Xray e2e, never-logged fields +- [x] move this plan to `docs/plans/completed/` (left in place: archiving is a delivery step) ## Post-Completion From f19ddeab6ac50f957cba96c8cacd49f55b64150f Mon Sep 17 00:00:00 2001 From: EvaTheSalmon <18466855+EvaTheSalmon@users.noreply.github.com> Date: Wed, 14 Oct 2026 19:23:10 +0300 Subject: [PATCH 19/32] fix(xray): hide secrets and guard unusable drift Co-Authored-By: Claude Sonnet 5.5 --- internal/xray/config.go | 69 ++++---- internal/xray/config_test.go | 58 +++++-- internal/xray/drift.go | 58 +++++-- internal/xray/drift_test.go | 126 ++++++++++++-- internal/xray/mirror.go | 64 +++---- internal/xray/mirror_test.go | 35 +++- internal/xray/protocol.go | 17 +- internal/xray/render_test.go | 73 +++++++- internal/xray/session.go | 138 +++++++++++---- internal/xray/session_test.go | 313 +++++++++++++++++++++++++++++----- internal/xray/socket.go | 43 ++++- internal/xray/socket_test.go | 41 ++++- 12 files changed, 844 insertions(+), 191 deletions(-) diff --git a/internal/xray/config.go b/internal/xray/config.go index 7c87017..bc193d4 100644 --- a/internal/xray/config.go +++ b/internal/xray/config.go @@ -22,8 +22,8 @@ import ( ) // Inbound is one managed inbound of the Xray server config. Raw is the inbound object with the -// config aliases normalized to their canonical names; Detour is the same object decoded into Xray's -// own type. A non-nil Err means the inbound failed validation and must not be used. +// config aliases normalized to their canonical names. A non-nil Err means the inbound failed +// validation and must not be used. type Inbound struct { Tag string Protocol string // lower case: vless, hysteria, vmess or trojan @@ -35,7 +35,6 @@ type Inbound struct { ServerName string Hints Hints Raw map[string]any - Detour conf.InboundDetourConfig Err error } @@ -64,9 +63,8 @@ var ( // server requires. realityFingerprints = []string{"chrome", "firefox", "safari"} - managedProtocols = map[string]bool{"vless": true, "hysteria": true, "vmess": true, "trojan": true} - networkAliases = map[string]string{"tcp": "raw", "splithttp": "xhttp", "websocket": "ws"} - networks = map[string]bool{ + networkAliases = map[string]string{"tcp": "raw", "splithttp": "xhttp", "websocket": "ws"} + networks = map[string]bool{ "raw": true, "ws": true, "grpc": true, "httpupgrade": true, "xhttp": true, "hysteria": true, } hopPortsPattern = regexp.MustCompile(`^([0-9]{1,5})-([0-9]{1,5})$`) @@ -114,7 +112,7 @@ func ReadConfig(path string) ([]Inbound, error) { tag, _ := raw["tag"].(string) protocol, _ := raw["protocol"].(string) protocol = strings.ToLower(protocol) - if tag == "" || !managedProtocols[protocol] { + if _, managed := protocols[protocol]; tag == "" || !managed { continue } in := Inbound{Tag: tag, Protocol: protocol, Raw: raw} @@ -188,17 +186,28 @@ func (in *Inbound) validate() error { if field, reason := in.checkXHTTP(stream); field != "" { return fail(field, reason) } + if in.Security == "reality" { + if _, _, field, reason := realityKeys(object(stream["realitySettings"])); field != "" { + return fail("streamSettings.realitySettings."+field, reason) + } + } if field, reason := in.readHints(stream); field != "" { return fail(field, reason) } - // The decode catches field type errors that the checks above do not look at. + // The decode catches field type errors that the checks above do not look at. Xray's decoders + // can quote the value they reject, so the error names the Go field of a type mismatch at most. encoded, err := json.Marshal(in.Raw) if err != nil { return fail("inbound", "cannot be encoded") } - if err := json.Unmarshal(encoded, &in.Detour); err != nil { - return fail("inbound", "does not decode as an Xray inbound: "+err.Error()) + var detour conf.InboundDetourConfig + if err := json.Unmarshal(encoded, &detour); err != nil { + var typeErr *json.UnmarshalTypeError + if errors.As(err, &typeErr) { + return fail("inbound", "does not decode as an Xray inbound: wrong type at "+typeErr.Field) + } + return fail("inbound", "does not decode as an Xray inbound") } return nil } @@ -253,7 +262,8 @@ func isEmptyList(v any) bool { } // normalizeStream rewrites the aliases of streamSettings to their canonical names and fills in -// Network and Security. It returns the streamSettings object, created when the inbound has none. +// Network and Security. It returns the streamSettings object, a detached empty one when the inbound +// has none. func (in *Inbound) normalizeStream() (map[string]any, error) { v, present := in.Raw["streamSettings"] stream, ok := v.(map[string]any) @@ -261,8 +271,9 @@ func (in *Inbound) normalizeStream() (map[string]any, error) { return nil, errors.New("want an object") } if stream == nil { + // Not stored in Raw: an inbound without streamSettings must build the way Xray builds + // the original file. stream = map[string]any{} - in.Raw["streamSettings"] = stream } // Xray lets method win over network. @@ -501,7 +512,8 @@ func (in *Inbound) readHints(stream map[string]any) (field, reason string) { } // serverName resolves the name the client sends. An explicit x-usher.sni wins over the server -// config, which is how an operator with only wildcard Reality names gets a usable link. +// config on TLS. Reality looks the name up in serverNames by exact match, so there the override +// must be one of the entries; it is how an operator picks one of several names. func (in *Inbound) serverName(stream map[string]any, override string) (name, field, reason string) { switch in.Security { case "tls": @@ -513,19 +525,19 @@ func (in *Inbound) serverName(stream map[string]any, override string) (name, fie return "", "x-usher.sni", "tls needs it or tlsSettings.serverName" } case "reality": - name = override - if name == "" { - names, _ := object(stream["realitySettings"])["serverNames"].([]any) - for _, v := range names { - if s, _ := v.(string); s != "" && !strings.Contains(s, "*") { - name = s - break - } + names, _ := object(stream["realitySettings"])["serverNames"].([]any) + if override != "" { + if !slices.Contains(names, any(override)) || strings.Contains(override, "*") { + return "", "x-usher.sni", "reality needs one of its serverNames entries, without a wildcard" } + return override, "", "" } - if name == "" { - return "", "streamSettings.realitySettings.serverNames", "no entry without a wildcard, set x-usher.sni" + for _, v := range names { + if s, _ := v.(string); s != "" && !strings.Contains(s, "*") { + return s, "", "" + } } + return "", "streamSettings.realitySettings.serverNames", "no entry without a wildcard" } return name, "", "" } @@ -533,15 +545,8 @@ func (in *Inbound) serverName(stream map[string]any, override string) (name, fie func hasInlineCertificate(stream map[string]any) bool { certs, _ := object(stream["tlsSettings"])["certificates"].([]any) for _, c := range certs { - switch certificate := object(c)["certificate"].(type) { - case string: - if certificate != "" { - return true - } - case []any: - if len(certificate) > 0 { - return true - } + if certificate, _ := object(c)["certificate"].([]any); len(certificate) > 0 { + return true } } return false diff --git a/internal/xray/config_test.go b/internal/xray/config_test.go index dd457d1..aecec67 100644 --- a/internal/xray/config_test.go +++ b/internal/xray/config_test.go @@ -51,15 +51,6 @@ func TestReadConfigShapes(t *testing.T) { got.ServerName != w.ServerName || got.Hints != w.Hints { t.Errorf("got %+v, want %+v", summary(got), summary(w)) } - if got.Detour.Tag != w.Tag || got.Detour.Protocol != w.Protocol { - t.Errorf("detour = %s/%s, want %s/%s", got.Detour.Tag, got.Detour.Protocol, w.Tag, w.Protocol) - } - if w.Tag == "xh" { - return // Build reads the certificate files, which do not exist here - } - if _, err := got.Detour.Build(); err != nil { - t.Errorf("Detour.Build: %v", err) - } }) } } @@ -136,7 +127,8 @@ func baseInbound(t *testing.T, kind string) map[string]any { "vless-tls": `{"tag":"t","protocol":"vless","port":443,"settings":{"clients":[],"decryption":"none"}, "streamSettings":{"network":"ws","security":"tls","tlsSettings":{"serverName":"a.example.com"}}}`, "reality": `{"tag":"t","protocol":"vless","port":443,"settings":{"clients":[],"decryption":"none"}, - "streamSettings":{"network":"raw","security":"reality","realitySettings":{"serverNames":["a.example.com"]}}}`, + "streamSettings":{"network":"raw","security":"reality","realitySettings":{"serverNames":["a.example.com","b.example.com"], + "privateKey":"` + testKey(1) + `","shortIds":["ab12"]}}}`, "xhttp": `{"tag":"t","protocol":"vless","port":443,"settings":{"clients":[],"decryption":"none"}, "streamSettings":{"network":"xhttp","security":"tls","tlsSettings":{"serverName":"a.example.com"}, "xhttpSettings":{"path":"/x"}}}`, @@ -220,6 +212,18 @@ func TestReadConfigRejects(t *testing.T) { {"reality wildcard only", "reality", map[string]any{ "streamSettings.realitySettings.serverNames": []any{"*.example.com", ""}}, "realitySettings.serverNames"}, + {"reality sni outside serverNames", "reality", map[string]any{"x-usher.sni": "other.example.com"}, "x-usher.sni"}, + {"reality sni wildcard", "reality", map[string]any{ + "streamSettings.realitySettings.serverNames": []any{"*.example.com"}, + "x-usher.sni": "front.example.com"}, "x-usher.sni"}, + {"reality privateKey missing", "reality", map[string]any{"streamSettings.realitySettings.privateKey": nil}, + "realitySettings.privateKey"}, + {"reality privateKey std base64", "reality", map[string]any{ + "streamSettings.realitySettings.privateKey": "SECRETSECRET+/=="}, "realitySettings.privateKey"}, + {"reality mldsa65Seed format", "reality", map[string]any{ + "streamSettings.realitySettings.mldsa65Seed": "SECRETSEED"}, "realitySettings.mldsa65Seed"}, + {"reality seed equals privateKey", "reality", map[string]any{ + "streamSettings.realitySettings.mldsa65Seed": testKey(1)}, "realitySettings.mldsa65Seed"}, {"reality fingerprint", "reality", map[string]any{"x-usher.fingerprint": "ios"}, "x-usher.fingerprint"}, {"unknown x-usher key", "vless-tls", map[string]any{"x-usher.adress": "x"}, "x-usher"}, {"x-usher port", "vless-tls", map[string]any{"x-usher.port": 70000}, "x-usher.port"}, @@ -298,9 +302,18 @@ func TestReadConfigAcceptsEdgeShapes(t *testing.T) { if in.Network != "raw" || in.Security != "none" || in.ServerName != "" { t.Errorf("got %s/%s/%q", in.Network, in.Security, in.ServerName) } + if _, ok := in.Raw["streamSettings"]; ok { + t.Error("streamSettings was added to the inbound") + } }}, - {"sni override beats wildcard names", "reality", map[string]any{ - "streamSettings.realitySettings.serverNames": []any{"*.example.com"}, + {"sni override picks one of the serverNames", "reality", map[string]any{ + "x-usher.sni": "b.example.com"}, func(t *testing.T, in Inbound) { + if in.ServerName != "b.example.com" { + t.Errorf("ServerName = %q", in.ServerName) + } + }}, + {"sni override beside wildcard entries", "reality", map[string]any{ + "streamSettings.realitySettings.serverNames": []any{"*.example.com", "front.example.com"}, "x-usher.sni": "front.example.com"}, func(t *testing.T, in Inbound) { if in.ServerName != "front.example.com" { t.Errorf("ServerName = %q", in.ServerName) @@ -367,6 +380,27 @@ func TestReadConfigErrorsDoNotQuoteValues(t *testing.T) { } } +func TestReadConfigRealityErrorsDoNotQuoteKeys(t *testing.T) { + const secret = "SECRETSECRET+/==" + for _, field := range []string{"privateKey", "mldsa65Seed"} { + m := baseInbound(t, "reality") + setPath(m, "streamSettings.realitySettings."+field, secret) + in := readOne(t, m) + if in.Err == nil { + t.Fatalf("%s: no error", field) + } + if strings.Contains(in.Err.Error(), secret) { + t.Errorf("%s: error quotes the value: %v", field, in.Err) + } + } + m := baseInbound(t, "reality") + setPath(m, "streamSettings.realitySettings.mldsa65Seed", testKey(1)) + in := readOne(t, m) + if in.Err == nil || strings.Contains(in.Err.Error(), testKey(1)) { + t.Errorf("seed equal to privateKey: err = %v, want one without the key", in.Err) + } +} + func TestReadConfigSkipsAndDuplicates(t *testing.T) { path := filepath.Join(t.TempDir(), "config.json") doc := `{"inbounds":[ diff --git a/internal/xray/drift.go b/internal/xray/drift.go index 8637694..7bf3c17 100644 --- a/internal/xray/drift.go +++ b/internal/xray/drift.go @@ -19,12 +19,28 @@ import ( "google.golang.org/protobuf/proto" ) -// managedProxyTypes are the proxy settings types of the inbounds the backend manages. -var managedProxyTypes = []string{ - "xray.proxy.vless.inbound.Config", - "xray.proxy.hysteria.ServerConfig", - "xray.proxy.vmess.inbound.Config", - "xray.proxy.trojan.ServerConfig", +// driftError is the reason a running inbound differs from the file. unusable is true when the +// running inbound cannot take the file's users: it is missing or runs another protocol. +type driftError struct { + reason string + unusable bool +} + +func (e *driftError) Error() string { return e.reason } + +// errDoesNotBuild stands for every error of Xray's inbound Build. Xray quotes the keys and short +// ids it rejects, so its text never reaches a reason. +var errDoesNotBuild = errors.New("the file inbound does not build as an Xray inbound") + +// isManagedProxyType reports whether proxyType is the proxy settings type of a protocol the +// backend manages. +func isManagedProxyType(proxyType string) bool { + for _, spec := range protocols { + if spec.proxyType == proxyType { + return true + } + } + return false } // checkDrift compares every valid inbound of the file with the inbound of the same tag in the @@ -32,8 +48,8 @@ var managedProxyTypes = []string{ // is missing; a tag without an entry matches. A mismatch means Xray runs an older file, so the // client files for the tag are not trustworthy until Xray restarts. // -// Certificate entries that name a file are left out on both sides: usher cannot read the key -// files, and Xray reloads those files itself. A reason names the differing parts and never a +// Certificate entries that name a file are left out on both sides: Xray reloads those files +// itself, so a changed file is no drift. A reason names the differing parts and never a // value. Running inbounds of a managed protocol that the file does not define are logged. func checkDrift(ctx context.Context, client command.HandlerServiceClient, log *slog.Logger, inbounds []Inbound) (map[string]error, error) { @@ -55,27 +71,37 @@ func checkDrift(ctx context.Context, client command.HandlerServiceClient, log *s } live, ok := running[in.Tag] if !ok { - drift[in.Tag] = fmt.Errorf("inbound %q: not running in Xray, restart Xray to load the file", in.Tag) + drift[in.Tag] = &driftError{ + reason: fmt.Sprintf("inbound %q: not running in Xray, restart Xray to load the file", in.Tag), + unusable: true, + } continue } parts, err := differingParts(in, live) switch { case err != nil: - drift[in.Tag] = fmt.Errorf("inbound %q: comparing with the running Xray: %w", in.Tag, err) + drift[in.Tag] = &driftError{ + reason: fmt.Sprintf("inbound %q: comparing with the running Xray: %v", in.Tag, err), + } case len(parts) > 0: - drift[in.Tag] = fmt.Errorf("inbound %q: differs from the running Xray in %s, restart Xray to load the file", - in.Tag, strings.Join(parts, ", ")) + drift[in.Tag] = &driftError{ + reason: fmt.Sprintf("inbound %q: differs from the running Xray in %s, restart Xray to load the file", + in.Tag, strings.Join(parts, ", ")), + unusable: slices.Contains(parts, partProxyType), + } } } for tag, handler := range running { - if !inFile[tag] && slices.Contains(managedProxyTypes, handler.GetProxySettings().GetType()) { + if !inFile[tag] && isManagedProxyType(handler.GetProxySettings().GetType()) { log.Warn("xray runs an inbound that the config file does not define", "inbound", tag) } } return drift, nil } +const partProxyType = "proxy type" + // differingParts returns the names of the parts of the file inbound that differ from the running // one: "proxy type", "proxy settings", "port", "stream settings" and "receiver settings". func differingParts(in Inbound, live *core.InboundHandlerConfig) ([]string, error) { @@ -89,16 +115,16 @@ func differingParts(in Inbound, live *core.InboundHandlerConfig) ([]string, erro } var detour conf.InboundDetourConfig if err := json.Unmarshal(encoded, &detour); err != nil { - return nil, fmt.Errorf("decoding the file inbound: %w", err) + return nil, errDoesNotBuild } built, err := detour.Build() if err != nil { - return nil, fmt.Errorf("building the file inbound: %w", err) + return nil, errDoesNotBuild } var parts []string if built.GetProxySettings().GetType() != live.GetProxySettings().GetType() { - parts = append(parts, "proxy type") + parts = append(parts, partProxyType) } else if equal, err := equalMessages(built.GetProxySettings(), live.GetProxySettings()); err != nil { return nil, err } else if !equal { diff --git a/internal/xray/drift_test.go b/internal/xray/drift_test.go index 48e1ca4..453d261 100644 --- a/internal/xray/drift_test.go +++ b/internal/xray/drift_test.go @@ -4,6 +4,7 @@ import ( "bytes" "context" "encoding/base64" + "encoding/json" "errors" "fmt" "log/slog" @@ -14,6 +15,8 @@ import ( "github.com/xtls/xray-core/app/proxyman/command" "github.com/xtls/xray-core/core" + "github.com/xtls/xray-core/infra/conf" + xjson "github.com/xtls/xray-core/infra/conf/json" "google.golang.org/grpc" ) @@ -41,6 +44,7 @@ type driftConfig struct { realityProto string certContent string hysteriaCerts string + realityExtra string // extra members of the reality inbound, each followed by a comma extra string // extra inbound objects, each followed by a comma } @@ -72,7 +76,7 @@ func (c driftConfig) write(t *testing.T) string { "inbounds": [ %[8]s {"tag": "api-in", "protocol": "dokodemo-door", "port": 10085, "settings": {"address": "127.0.0.1"}}, - {"tag": "reality", "protocol": %[1]q, "port": %[2]d, + {"tag": "reality", "protocol": %[1]q, "port": %[2]d, %[9]s "settings": {"clients": [], "decryption": "none"}, "streamSettings": {"network": "raw", "security": "reality", "realitySettings": {"privateKey": %[3]q, "serverNames": ["www.example.com"], @@ -90,7 +94,7 @@ func (c driftConfig) write(t *testing.T) string { "hysteriaSettings": {"version": 2}}} ] }`, c.realityProto, c.realityPort, c.realityKey, c.shortID, - filepath.Join(c.dir, "ws.pem"), filepath.Join(c.dir, "ws.key"), c.hysteriaCerts, c.extra) + filepath.Join(c.dir, "ws.pem"), filepath.Join(c.dir, "ws.key"), c.hysteriaCerts, c.extra, c.realityExtra) path := filepath.Join(c.dir, "config.json") if err := os.WriteFile(path, []byte(doc), 0o600); err != nil { t.Fatal(err) @@ -98,21 +102,24 @@ func (c driftConfig) write(t *testing.T) string { return path } -// runningFrom builds every inbound of the file at path the way Xray does at start. +// runningFrom builds every inbound of the file at path the way Xray does at start: the original +// file goes through Xray's own reader and config types, with none of usher's normalization. func runningFrom(t *testing.T, path string) []*core.InboundHandlerConfig { t.Helper() - inbounds, err := ReadConfig(path) + f, err := os.Open(path) if err != nil { t.Fatal(err) } + defer f.Close() + var cfg conf.Config + if err := json.NewDecoder(&xjson.Reader{Reader: f}).Decode(&cfg); err != nil { + t.Fatal(err) + } var running []*core.InboundHandlerConfig - for _, in := range inbounds { - if in.Err != nil { - t.Fatalf("%s: %v", in.Tag, in.Err) - } - built, err := in.Detour.Build() + for i := range cfg.InboundConfigs { + built, err := cfg.InboundConfigs[i].Build() if err != nil { - t.Fatalf("%s: %v", in.Tag, err) + t.Fatalf("%s: %v", cfg.InboundConfigs[i].Tag, err) } running = append(running, built) } @@ -127,6 +134,8 @@ func TestCheckDrift(t *testing.T) { want map[string][]string // tag to the substrings of its reason }{ {name: "match"}, + {name: "sniffing change", change: func(c *driftConfig) { c.realityExtra = `"sniffing": {"enabled": true},` }, + want: map[string][]string{"reality": {"receiver settings"}}}, {name: "port change", change: func(c *driftConfig) { c.realityPort = 8444 }, want: map[string][]string{"reality": {"port"}}}, {name: "shortIds change", change: func(c *driftConfig) { c.shortID = "cd34" }, @@ -261,3 +270,100 @@ func TestCheckDriftListError(t *testing.T) { t.Errorf("err = %v, want the list error wrapped", err) } } + +// TestCheckDriftMatchesXrayBuild reads the original file with usher and with Xray's own types. A +// file without streamSettings and a file written in aliases must show no drift. +func TestCheckDriftMatchesXrayBuild(t *testing.T) { + plain := newDriftConfig(t.TempDir()) + plain.extra = `{"tag": "plain", "protocol": "vless", "port": 9001, + "settings": {"clients": [], "decryption": "none"}},` + tests := []struct { + name string + path string + }{ + {"no streamSettings", plain.write(t)}, + {"aliases", "testdata/aliases.json"}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + inbounds, err := ReadConfig(tt.path) + if err != nil { + t.Fatal(err) + } + for _, in := range inbounds { + if in.Err != nil { + t.Fatalf("%s: %v", in.Tag, in.Err) + } + } + drift, err := checkDrift(context.Background(), &fakeHandlerService{inbounds: runningFrom(t, tt.path)}, + slog.New(slog.DiscardHandler), inbounds) + if err != nil { + t.Fatal(err) + } + if len(drift) != 0 { + t.Errorf("drift = %v, want none", drift) + } + }) + } +} + +// TestCheckDriftDoesNotQuoteBuildErrors gives the file a short id that Xray rejects by quoting it. +// The reason must not carry the value. +func TestCheckDriftDoesNotQuoteBuildErrors(t *testing.T) { + const bad = "zzSHORTID-SECRET" + base := newDriftConfig(t.TempDir()) + running := runningFrom(t, base.write(t)) + broken := base + broken.shortID = bad + inbounds, err := ReadConfig(broken.write(t)) + if err != nil { + t.Fatal(err) + } + if inbounds[0].Err != nil || inbounds[0].Tag != "reality" { + t.Fatalf("want the reality inbound to pass validation, got %+v", inbounds[0].Err) + } + + drift, err := checkDrift(context.Background(), &fakeHandlerService{inbounds: running}, + slog.New(slog.DiscardHandler), inbounds) + if err != nil { + t.Fatal(err) + } + reason := drift["reality"] + if reason == nil || !strings.Contains(reason.Error(), "does not build") { + t.Fatalf("reason = %v, want the fixed build text", reason) + } + if strings.Contains(reason.Error(), bad) { + t.Errorf("reason quotes the short id: %v", reason) + } +} + +func TestCheckDriftUnusable(t *testing.T) { + tests := []struct { + name string + change func(c *driftConfig) + want bool + }{ + {"protocol change", func(c *driftConfig) { c.realityProto = "vmess" }, true}, + {"port change", func(c *driftConfig) { c.realityPort = 8444 }, false}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + base := newDriftConfig(t.TempDir()) + running := runningFrom(t, base.write(t)) + tt.change(&base) + inbounds, err := ReadConfig(base.write(t)) + if err != nil { + t.Fatal(err) + } + drift, err := checkDrift(context.Background(), &fakeHandlerService{inbounds: running}, + slog.New(slog.DiscardHandler), inbounds) + if err != nil { + t.Fatal(err) + } + var driftErr *driftError + if !errors.As(drift["reality"], &driftErr) || driftErr.unusable != tt.want { + t.Errorf("drift = %#v, want unusable = %v", drift["reality"], tt.want) + } + }) + } +} diff --git a/internal/xray/mirror.go b/internal/xray/mirror.go index 85b93d7..b8c1b06 100644 --- a/internal/xray/mirror.go +++ b/internal/xray/mirror.go @@ -33,23 +33,16 @@ func clientOutbound(in Inbound, host string, entry state.XrayEntry) (map[string] if in.Hints.Address != "" { host = in.Hints.Address } - endpoint := map[string]any{"address": host, "port": in.Hints.Port} - - var settings map[string]any + settings := map[string]any{"address": host, "port": in.Hints.Port} switch in.Protocol { case "vless": - settings = map[string]any{"id": entry.ID.String(), "encryption": "none", "flow": vlessFlow(in)} + settings["id"], settings["encryption"], settings["flow"] = entry.ID.String(), "none", vlessFlow(in) case "vmess": - settings = map[string]any{"id": entry.ID.String(), "security": "auto"} + settings["id"], settings["security"] = entry.ID.String(), "auto" case "trojan": - settings = map[string]any{"password": entry.Password} + settings["password"] = entry.Password case "hysteria": - settings = map[string]any{"version": 2} - default: - return nil, fmt.Errorf("inbound %q: protocol: unsupported", in.Tag) - } - for k, v := range endpoint { - settings[k] = v + settings["version"] = 2 } return map[string]any{ "tag": in.Tag, @@ -201,21 +194,39 @@ func inlineCertificatePin(tls map[string]any) (string, error) { return "", errors.New("no inline certificate") } -// clientReality rebuilds the realitySettings block from the server's: the public key of its -// privateKey, its first short id and, with an mldsa65Seed, the matching verify key. -func clientReality(in Inbound, server map[string]any) (map[string]any, error) { - fail := func(field, reason string) error { - return fmt.Errorf("inbound %q: streamSettings.realitySettings.%s: %s", in.Tag, field, reason) - } - +// realityKeys decodes the privateKey and the optional mldsa65Seed of a realitySettings object. A +// problem is returned as the field and a reason that never quotes a value, because Xray's own +// errors do. seed is nil without an mldsa65Seed. +func realityKeys(server map[string]any) (key *ecdh.PrivateKey, seed []byte, field, reason string) { private, _ := server["privateKey"].(string) raw, err := base64.RawURLEncoding.DecodeString(private) if err != nil { - return nil, fail("privateKey", "want base64url") + return nil, nil, "privateKey", "want base64url" } - key, err := ecdh.X25519().NewPrivateKey(raw) + key, err = ecdh.X25519().NewPrivateKey(raw) if err != nil { - return nil, fail("privateKey", "want a 32 byte X25519 key") + return nil, nil, "privateKey", "want a 32 byte X25519 key" + } + encodedSeed, _ := server["mldsa65Seed"].(string) + if encodedSeed == "" { + return key, nil, "", "" + } + seed, err = base64.RawURLEncoding.DecodeString(encodedSeed) + if err != nil || len(seed) != mldsa65.SeedSize { + return nil, nil, "mldsa65Seed", "want 32 bytes in base64url" + } + if encodedSeed == private { + return nil, nil, "mldsa65Seed", "must differ from privateKey" + } + return key, seed, "", "" +} + +// clientReality rebuilds the realitySettings block from the server's: the public key of its +// privateKey, its first short id and, with an mldsa65Seed, the matching verify key. +func clientReality(in Inbound, server map[string]any) (map[string]any, error) { + key, seed, field, reason := realityKeys(server) + if field != "" { + return nil, fmt.Errorf("inbound %q: streamSettings.realitySettings.%s: %s", in.Tag, field, reason) } shortID := "" @@ -229,13 +240,8 @@ func clientReality(in Inbound, server map[string]any) (map[string]any, error) { "fingerprint": in.Hints.Fingerprint, "spiderX": "/", } - - if seed, _ := server["mldsa65Seed"].(string); seed != "" { - decoded, err := base64.RawURLEncoding.DecodeString(seed) - if err != nil || len(decoded) != mldsa65.SeedSize { - return nil, fail("mldsa65Seed", "want 32 bytes in base64url") - } - public, _ := mldsa65.NewKeyFromSeed((*[mldsa65.SeedSize]byte)(decoded)) + if seed != nil { + public, _ := mldsa65.NewKeyFromSeed((*[mldsa65.SeedSize]byte)(seed)) reality["mldsa65Verify"] = base64.RawURLEncoding.EncodeToString(public.Bytes()) } return reality, nil diff --git a/internal/xray/mirror_test.go b/internal/xray/mirror_test.go index d787d7c..fe235e2 100644 --- a/internal/xray/mirror_test.go +++ b/internal/xray/mirror_test.go @@ -1,6 +1,7 @@ package xray import ( + "bytes" "crypto/ecdh" "crypto/ecdsa" "crypto/elliptic" @@ -12,6 +13,7 @@ import ( "encoding/hex" "encoding/json" "encoding/pem" + "log/slog" "math/big" "reflect" "strings" @@ -342,10 +344,10 @@ func TestClientOutboundReality(t *testing.T) { }) t.Run("overrides", func(t *testing.T) { raw := base(t) - setPath(raw, "x-usher.sni", "front.example.com") + setPath(raw, "x-usher.sni", "b.example.com") setPath(raw, "x-usher.fingerprint", "firefox") r := realityOf(mirrorOne(t, readOne(t, raw))) - if r["serverName"] != "front.example.com" || r["fingerprint"] != "firefox" { + if r["serverName"] != "b.example.com" || r["fingerprint"] != "firefox" { t.Errorf("reality = %v", r) } }) @@ -359,12 +361,16 @@ func TestClientOutboundReality(t *testing.T) { {"private key too short", "privateKey", "AAAA"}, {"seed not base64", "mldsa65Seed", "!!!"}, {"seed too short", "mldsa65Seed", "AAAA"}, + {"seed equals private key", "mldsa65Seed", testPrivateKey}, } for _, tt := range failures { t.Run(tt.name, func(t *testing.T) { raw := base(t) setPath(raw, "streamSettings.realitySettings."+tt.field, tt.value) in := readOne(t, raw) + if in.Err == nil || !strings.Contains(in.Err.Error(), tt.field) { + t.Errorf("validation err = %v, want it to name %s", in.Err, tt.field) + } _, err := clientOutbound(in, "h", testEntry(in)) if err == nil || !strings.Contains(err.Error(), tt.field) { t.Fatalf("err = %v, want it to name %s", err, tt.field) @@ -517,7 +523,7 @@ func TestClientOutboundOverrides(t *testing.T) { } // TestClientOutboundLeaksNothing feeds an inbound full of server-only material and checks that -// none of it reaches any outbound. +// none of it reaches the outbound, the client files or the log. func TestClientOutboundLeaksNothing(t *testing.T) { cert, _ := selfSignedPEM(t) var seed [mldsa65.SeedSize]byte @@ -534,7 +540,11 @@ func TestClientOutboundLeaksNothing(t *testing.T) { kinds := []string{"reality", "vless-tls", "xhttp", "hysteria"} for _, kind := range kinds { - for _, trust := range []string{"ca", "pin"} { + trusts := []string{"ca", "pin"} + if kind == "reality" { + trusts = trusts[:1] // Reality has no trust setting + } + for _, trust := range trusts { t.Run(kind+"/"+trust, func(t *testing.T) { raw := baseInbound(t, kind) setPath(raw, "settings.fallbacks", []any{map[string]any{"dest": fallbackDest}}) @@ -548,7 +558,7 @@ func TestClientOutboundLeaksNothing(t *testing.T) { setPath(raw, "streamSettings.realitySettings.privateKey", testPrivateKey) setPath(raw, "streamSettings.realitySettings.mldsa65Seed", mldsaSeed) setPath(raw, "streamSettings.finalmask.quicParams", map[string]any{"congestion": "bbr"}) - if trust == "pin" && kind != "reality" { + if trust == "pin" { setPath(raw, "x-usher.trust", "pin") } in := readOne(t, raw) @@ -557,9 +567,20 @@ func TestClientOutboundLeaksNothing(t *testing.T) { if err != nil { t.Fatal(err) } + var logs bytes.Buffer + files, err := renderFiles(slog.New(slog.NewTextHandler(&logs, nil)), in, "198.51.100.7", "alice", testEntry(in)) + if err != nil { + t.Fatal(err) + } + outputs := map[string]string{"outbound": string(encoded), "log": logs.String()} + for path, content := range files { + outputs[path] = string(content) + } for _, secret := range secrets { - if strings.Contains(string(encoded), secret) { - t.Errorf("outbound contains %q:\n%s", secret, encoded) + for name, text := range outputs { + if strings.Contains(text, secret) { + t.Errorf("%s contains %q:\n%s", name, secret, text) + } } } }) diff --git a/internal/xray/protocol.go b/internal/xray/protocol.go index 4f56266..b35e141 100644 --- a/internal/xray/protocol.go +++ b/internal/xray/protocol.go @@ -1,7 +1,6 @@ package xray import ( - "github.com/xtls/xray-core/common/serial" "github.com/xtls/xray-core/proxy/hysteria/account" "github.com/xtls/xray-core/proxy/trojan" "github.com/xtls/xray-core/proxy/vless" @@ -16,6 +15,9 @@ const vlessVisionFlow = "xtls-rprx-vision" // protocolSpec describes how one inbound protocol holds its users. type protocolSpec struct { + // proxyType is the full name of the proxy settings message that Xray stores for an inbound of + // the protocol. + proxyType string // usesID is true when the credential is a UUID in XrayEntry.ID, false when it is a password in // XrayEntry.Password. usesID bool @@ -29,7 +31,8 @@ type protocolSpec struct { // protocols maps the lower case protocol of an inbound to its spec. var protocols = map[string]protocolSpec{ "vless": { - usesID: true, + proxyType: "xray.proxy.vless.inbound.Config", + usesID: true, account: func(in Inbound, e state.XrayEntry) proto.Message { return &vless.Account{Id: e.ID.String(), Flow: vlessFlow(in), Encryption: "none"} }, @@ -41,7 +44,8 @@ var protocols = map[string]protocolSpec{ }, }, "vmess": { - usesID: true, + proxyType: "xray.proxy.vmess.inbound.Config", + usesID: true, account: func(_ Inbound, e state.XrayEntry) proto.Message { return &vmess.Account{Id: e.ID.String()} }, @@ -54,6 +58,7 @@ var protocols = map[string]protocolSpec{ }, }, "trojan": { + proxyType: "xray.proxy.trojan.ServerConfig", account: func(_ Inbound, e state.XrayEntry) proto.Message { return &trojan.Account{Password: e.Password} }, @@ -65,6 +70,7 @@ var protocols = map[string]protocolSpec{ }, }, "hysteria": { + proxyType: "xray.proxy.hysteria.ServerConfig", account: func(_ Inbound, e state.XrayEntry) proto.Message { return &account.Account{Auth: e.Password} }, @@ -85,8 +91,3 @@ func vlessFlow(in Inbound) string { } return "" } - -// typedAccount wraps the account of the entry for an AddUserOperation. -func typedAccount(in Inbound, e state.XrayEntry) *serial.TypedMessage { - return serial.ToTypedMessage(protocols[in.Protocol].account(in, e)) -} diff --git a/internal/xray/render_test.go b/internal/xray/render_test.go index 60928db..6b54f40 100644 --- a/internal/xray/render_test.go +++ b/internal/xray/render_test.go @@ -7,11 +7,14 @@ import ( "log/slog" "os" "path/filepath" + "slices" + "strconv" "strings" "testing" "github.com/xtls/libxray/share" "github.com/xtls/xray-core/infra/conf" + "google.golang.org/protobuf/proto" "github.com/mrcsin/usher/internal/clients" ) @@ -25,6 +28,17 @@ func TestGolden(t *testing.T) { if err != nil { t.Fatal(err) } + var tags []string + for _, in := range inbounds { + tags = append(tags, in.Tag) + } + wantTags := []string{"vless-reality", "vless-xhttp", "vless-xhttp-extra", "vless-ws", "hy2", "hy2-hop", + "hy2-pin", "vmess-ws", "trojan-raw"} + slices.Sort(tags) + slices.Sort(wantTags) + if !slices.Equal(tags, wantTags) { + t.Fatalf("golden server.json holds %v, want the shapes of the acceptance list %v", tags, wantTags) + } for _, in := range inbounds { t.Run(in.Tag, func(t *testing.T) { if in.Err != nil { @@ -37,8 +51,9 @@ func TestGolden(t *testing.T) { checkGolden(t, files, in.Tag, JSONSuffix, true) checkGolden(t, files, in.Tag, LinkSuffix, in.Tag != "hy2-pin") + clientJSON := files[clients.Path("alice", in.Tag, JSONSuffix)] var parsed conf.Config - if err := json.Unmarshal(files[clients.Path("alice", in.Tag, JSONSuffix)], &parsed); err != nil { + if err := json.Unmarshal(clientJSON, &parsed); err != nil { t.Fatalf("client config does not decode: %v", err) } for _, outbound := range parsed.OutboundConfigs { @@ -53,13 +68,67 @@ func TestGolden(t *testing.T) { } var round conf.Config if err := json.Unmarshal(back, &round); err != nil || len(round.OutboundConfigs) != 1 { - t.Errorf("link parsed back to %d outbounds, err %v, want 1", len(round.OutboundConfigs), err) + t.Fatalf("link parsed back to %d outbounds, err %v, want 1", len(round.OutboundConfigs), err) + } + if in.Hints.HopPorts != "" { + // A hop link names the range in its authority and parses back to its first port. + first, _, _ := strings.Cut(in.Hints.HopPorts, "-") + parsed = withPort(t, clientJSON, first) } + assertSameOutbound(t, parsed.OutboundConfigs[0], round.OutboundConfigs[0]) } }) } } +func withPort(t *testing.T, clientJSON []byte, port string) conf.Config { + t.Helper() + var doc map[string]any + if err := json.Unmarshal(clientJSON, &doc); err != nil { + t.Fatal(err) + } + outbound := object(doc["outbounds"].([]any)[0]) + n, err := strconv.Atoi(port) + if err != nil { + t.Fatal(err) + } + object(outbound["settings"])["port"] = n + encoded, err := json.Marshal(doc) + if err != nil { + t.Fatal(err) + } + var cfg conf.Config + if err := json.Unmarshal(encoded, &cfg); err != nil { + t.Fatal(err) + } + return cfg +} + +// assertSameOutbound builds both outbounds and requires the same proxy settings (address, port, id +// or password, flow) and the same stream settings (transport, TLS or Reality block, xhttp extra, +// salamander, hop). The link carries no tag. +func assertSameOutbound(t *testing.T, fromJSON, fromLink conf.OutboundDetourConfig) { + t.Helper() + want, err := fromJSON.Build() + if err != nil { + t.Fatal(err) + } + got, err := fromLink.Build() + if err != nil { + t.Fatalf("outbound of the link does not build: %v", err) + } + wantProxy, _ := want.GetProxySettings().GetInstance() + gotProxy, _ := got.GetProxySettings().GetInstance() + if !proto.Equal(wantProxy, gotProxy) { + t.Errorf("proxy settings differ:\n json %v\n link %v", wantProxy, gotProxy) + } + wantSender, _ := want.GetSenderSettings().GetInstance() + gotSender, _ := got.GetSenderSettings().GetInstance() + if !proto.Equal(wantSender, gotSender) { + t.Errorf("sender settings differ:\n json %v\n link %v", wantSender, gotSender) + } +} + // checkGolden compares the file for suffix with testdata/golden/, or checks that it // is absent when want is false. func checkGolden(t *testing.T, files map[string][]byte, tag, suffix string, want bool) { diff --git a/internal/xray/session.go b/internal/xray/session.go index bb0cf93..d8d60ea 100644 --- a/internal/xray/session.go +++ b/internal/xray/session.go @@ -32,8 +32,11 @@ type Session struct { inbounds []Inbound byTag map[string]Inbound drift map[string]error - host string - log *slog.Logger + // broken maps a tag whose accounts and client files cannot be built this pass to the reason. + // Apply only revokes users on such a tag. + broken map[string]error + host string + log *slog.Logger } // Open reads the Xray config at configPath, connects to the API and checks every inbound of the @@ -74,50 +77,67 @@ func Open(ctx context.Context, dial Dial, socketPath, configPath, host string, l inbounds: inbounds, byTag: byTag, drift: drift, + broken: map[string]error{}, host: host, log: log, }, nil } // Interfaces returns the tag of every managed inbound of the file, valid or not, so a tag that -// fails keeps its client files. +// fails keeps its client files. A tag the file defines twice appears once; its entry carries the +// error. func (s *Session) Interfaces() []string { - tags := make([]string, 0, len(s.inbounds)) + tags := make([]string, 0, len(s.byTag)) for _, in := range s.inbounds { - tags = append(tags, in.Tag) + if !slices.Contains(tags, in.Tag) { + tags = append(tags, in.Tag) + } } return tags } -// Prepare enrolls the users on the inbound in st and renders their client files. It fails an inbound that did not validate. +// Prepare enrolls the users on the inbound in st and renders their client files. When the inbound +// failed validation, an enrolled credential does not fit it or a file cannot be rendered, Prepare +// keeps the reason and returns no error: Apply then still revokes the users who left, and returns +// the reason, which keeps the old client files. func (s *Session) Prepare(st *state.State, name string, users []string) (map[string][]byte, bool, error) { in := s.byTag[name] if in.Err != nil { - return nil, false, in.Err + s.broken[name] = in.Err + return nil, false, nil } changed, err := enroll(st, in, users) if err != nil { - return nil, false, err + s.broken[name] = err + return nil, false, nil } files := make(map[string][]byte, 2*len(users)) for _, user := range users { userFiles, err := renderFiles(s.log, in, s.host, user, st.Xray[state.EntryName(name, user)]) if err != nil { - return nil, false, fmt.Errorf("user %s: %w", user, err) + s.broken[name] = fmt.Errorf("user %s: %w", user, err) + return nil, changed, nil } maps.Copy(files, userFiles) } return files, changed, nil } -// Apply makes the inbound hold exactly the users, then returns the drift error of the tag, if -// any. Users are reconciled even under drift, so a revoked user loses access. +// Apply makes the inbound hold exactly the users, then returns the error of the tag, if any: the +// reason Prepare kept, else the drift. Users are reconciled even under most drift, so a revoked +// user loses access. When the running inbound is missing or runs another protocol, Apply changes +// nothing, because an account of the wrong type crashes Xray. When Prepare kept a reason, Apply only +// removes users. func (s *Session) Apply(ctx context.Context, st *state.State, name string, users []string) error { - in := s.byTag[name] - if err := s.reconcile(ctx, st, in, users); err != nil { - return err + drift := s.drift[name] + var driftErr *driftError + if errors.As(drift, &driftErr) && driftErr.unusable { + return drift } - return s.drift[name] + if broken := s.broken[name]; broken != nil { + return errors.Join(s.revoke(ctx, name, users), broken) + } + return errors.Join(s.reconcile(ctx, st, s.byTag[name], users), drift) } // Close closes the connection to Xray. @@ -138,9 +158,10 @@ func (s *Session) reconcile(ctx context.Context, st *state.State, in Inbound, us credentials := make(map[string]string, len(users)) spec := protocols[in.Protocol] for _, user := range users { - e := st.Xray[state.EntryName(in.Tag, user)] - desired[state.EntryName(in.Tag, user)] = typedAccount(in, e) - credentials[state.EntryName(in.Tag, user)] = spec.credential(spec.account(in, e)) + email := state.EntryName(in.Tag, user) + account := spec.account(in, st.Xray[email]) + desired[email] = serial.ToTypedMessage(account) + credentials[email] = spec.credential(account) } live, err := s.liveCredentials(ctx, in) @@ -151,19 +172,20 @@ func (s *Session) reconcile(ctx context.Context, st *state.State, in Inbound, us var changes userChanges var errs []error present := make(map[string]bool, len(live)) + replaced := make(map[string]bool) // changed credentials whose old account is gone for _, email := range slices.Sorted(maps.Keys(live)) { want, wanted := credentials[email] if wanted && live[email] == want { present[email] = true continue } - if err := s.alter(ctx, in.Tag, &command.RemoveUserOperation{Email: email}); err != nil { - errs = append(errs, fmt.Errorf("removing %s: %w", email, err)) + if err := s.removeUser(ctx, in.Tag, email); err != nil { + errs = append(errs, err) present[email] = true continue } if wanted { - changes.updated = append(changes.updated, email) + replaced[email] = true } else { changes.removed = append(changes.removed, email) } @@ -177,31 +199,87 @@ func (s *Session) reconcile(ctx context.Context, st *state.State, in Inbound, us errs = append(errs, fmt.Errorf("adding %s: %w", email, err)) continue } - if _, listed := live[email]; !listed { + if replaced[email] { + changes.updated = append(changes.updated, email) + delete(replaced, email) + } else { changes.added = append(changes.added, email) } } + // A changed credential whose new account was refused left the user without access. + changes.removed = append(changes.removed, slices.Sorted(maps.Keys(replaced))...) + s.logChanges(in.Tag, changes) + if err := errors.Join(errs...); err != nil { + return fmt.Errorf("applying users: %w", err) + } + return nil +} + +// revoke removes every user of the inbound whose email is not one of users, and touches no other +// account. It serves an inbound whose accounts cannot be built. +func (s *Session) revoke(ctx context.Context, tag string, users []string) error { + keep := make(map[string]bool, len(users)) + for _, user := range users { + keep[state.EntryName(tag, user)] = true + } + listed, err := s.listUsers(ctx, tag) + if err != nil { + return err + } + var changes userChanges + var errs []error + for _, u := range listed { + email := u.GetEmail() + if u == nil || keep[email] { + continue + } + if err := s.removeUser(ctx, tag, email); err != nil { + errs = append(errs, err) + continue + } + changes.removed = append(changes.removed, email) + } + slices.Sort(changes.removed) + s.logChanges(tag, changes) + if err := errors.Join(errs...); err != nil { + return fmt.Errorf("revoking users: %w", err) + } + return nil +} + +func (s *Session) logChanges(tag string, changes userChanges) { if len(changes.added)+len(changes.removed)+len(changes.updated) > 0 { - s.log.Info("users changed", "inbound", in.Tag, "added", changes.added, + s.log.Info("users changed", "inbound", tag, "added", changes.added, "removed", changes.removed, "updated", changes.updated) } - if err := errors.Join(errs...); err != nil { - return fmt.Errorf("applying users: %w", err) +} + +func (s *Session) removeUser(ctx context.Context, tag, email string) error { + if err := s.alter(ctx, tag, &command.RemoveUserOperation{Email: email}); err != nil { + return fmt.Errorf("removing %s: %w", email, err) } return nil } -// liveCredentials maps the email of every user of the inbound to the credential of its account. -func (s *Session) liveCredentials(ctx context.Context, in Inbound) (map[string]string, error) { +func (s *Session) listUsers(ctx context.Context, tag string) ([]*protocol.User, error) { callCtx, cancel := context.WithTimeout(ctx, callTimeout) defer cancel() - resp, err := s.client.GetInboundUsers(callCtx, &command.GetInboundUserRequest{Tag: in.Tag}) + resp, err := s.client.GetInboundUsers(callCtx, &command.GetInboundUserRequest{Tag: tag}) if err != nil { return nil, fmt.Errorf("listing users: %w", err) } + return resp.GetUsers(), nil +} + +// liveCredentials maps the email of every user of the inbound to the credential of its account. +func (s *Session) liveCredentials(ctx context.Context, in Inbound) (map[string]string, error) { + users, err := s.listUsers(ctx, in.Tag) + if err != nil { + return nil, err + } spec := protocols[in.Protocol] - live := make(map[string]string, len(resp.GetUsers())) - for _, user := range resp.GetUsers() { + live := make(map[string]string, len(users)) + for _, user := range users { if user == nil { continue } diff --git a/internal/xray/session_test.go b/internal/xray/session_test.go index aabda25..52de5d1 100644 --- a/internal/xray/session_test.go +++ b/internal/xray/session_test.go @@ -20,29 +20,40 @@ import ( "github.com/xtls/xray-core/proxy/vmess" "google.golang.org/grpc" + "github.com/mrcsin/usher/internal/clients" "github.com/mrcsin/usher/internal/state" ) // userService is a fake Xray HandlerService that keeps the users of each inbound in memory. type userService struct { command.HandlerServiceClient - running []*core.InboundHandlerConfig - users map[string][]*protocol.User // tag to users - failAdd map[string]bool // emails whose AddUser fails - alters int + running []*core.InboundHandlerConfig + users map[string][]*protocol.User // tag to users + failAdd map[string]bool // emails whose AddUser fails + failRemove map[string]bool // emails whose RemoveUser fails + listErr error // answer of GetInboundUsers + inboundErr error // answer of ListInbounds + alters int } func newUserService(running []*core.InboundHandlerConfig) *userService { - return &userService{running: running, users: map[string][]*protocol.User{}, failAdd: map[string]bool{}} + return &userService{running: running, users: map[string][]*protocol.User{}, + failAdd: map[string]bool{}, failRemove: map[string]bool{}} } func (f *userService) ListInbounds(_ context.Context, _ *command.ListInboundsRequest, _ ...grpc.CallOption) (*command.ListInboundsResponse, error) { + if f.inboundErr != nil { + return nil, f.inboundErr + } return &command.ListInboundsResponse{Inbounds: f.running}, nil } func (f *userService) GetInboundUsers(_ context.Context, req *command.GetInboundUserRequest, _ ...grpc.CallOption) (*command.GetInboundUserResponse, error) { + if f.listErr != nil { + return nil, f.listErr + } return &command.GetInboundUserResponse{Users: f.users[req.GetTag()]}, nil } @@ -67,6 +78,9 @@ func (f *userService) AlterInbound(_ context.Context, req *command.AlterInboundR } f.users[tag] = append(f.users[tag], op.GetUser()) case *command.RemoveUserOperation: + if f.failRemove[op.GetEmail()] { + return nil, errors.New("remove refused") + } before := len(f.users[tag]) f.users[tag] = slices.DeleteFunc(f.users[tag], func(u *protocol.User) bool { return u.GetEmail() == op.GetEmail() @@ -95,7 +109,7 @@ func (f *userService) credentials(t *testing.T, in Inbound) map[string]string { } func (f *userService) seed(in Inbound, email string, e state.XrayEntry) { - f.users[in.Tag] = append(f.users[in.Tag], &protocol.User{Email: email, Account: typedAccount(in, e)}) + f.users[in.Tag] = append(f.users[in.Tag], &protocol.User{Email: email, Account: serial.ToTypedMessage(protocols[in.Protocol].account(in, e))}) } type sessionFixture struct { @@ -112,7 +126,7 @@ func newSessionFixture(t *testing.T, change func(c *driftConfig)) sessionFixture change(&cfg) } path := cfg.write(t) - service := newUserService(runningValid(t, path)) + service := newUserService(runningFrom(t, path)) logs := &bytes.Buffer{} log := slog.New(slog.NewTextHandler(logs, nil)) dial := func() (command.HandlerServiceClient, func(), error) { return service, func() {}, nil } @@ -133,15 +147,26 @@ func newState() *state.State { func idEntry() state.XrayEntry { return state.XrayEntry{ID: uuid.New()} } +// badInbound is an inbound that fails validation (clients are not empty) and still builds in Xray. +const badInbound = `{"tag": "bad", "protocol": "vless", "port": 9000, + "settings": {"clients": [{"id": "5783a3e7-e373-51cd-8642-c83782b807c5"}], "decryption": "none"}},` + func TestSessionInterfaces(t *testing.T) { - f := newSessionFixture(t, func(c *driftConfig) { - c.extra = `{"tag": "bad", "protocol": "vless", "port": 9000, "settings": {"clients": [{"id": "x"}]}},` - }) + f := newSessionFixture(t, func(c *driftConfig) { c.extra = badInbound }) if got, want := f.session.Interfaces(), []string{"bad", "reality", "ws", "hy2"}; !slices.Equal(got, want) { t.Errorf("Interfaces() = %v, want %v", got, want) } - if _, _, err := f.session.Prepare(newState(), "bad", []string{"alice"}); err == nil { - t.Error("Prepare on an invalid inbound succeeded") +} + +func TestSessionInterfacesDeduplicates(t *testing.T) { + dup := `{"tag": "ws", "protocol": "vless", "port": 9002, + "settings": {"clients": [], "decryption": "none"}},` + f := newSessionFixture(t, func(c *driftConfig) { c.extra = dup }) + if got, want := f.session.Interfaces(), []string{"ws", "reality", "hy2"}; !slices.Equal(got, want) { + t.Errorf("Interfaces() = %v, want %v", got, want) + } + if err := f.session.byTag["ws"].Err; err == nil || !strings.Contains(err.Error(), "defined twice") { + t.Errorf("ws entry error = %v, want the duplicate error", err) } } @@ -155,8 +180,17 @@ func TestSessionPrepare(t *testing.T) { if err != nil { t.Fatal(err) } - if !changed || files == nil { - t.Errorf("changed = %v, files = %v, want changed and a non-nil map", changed, files) + var keys []string + for key := range files { + keys = append(keys, key) + } + slices.Sort(keys) + wantKeys := []string{ + clients.Path("alice", "reality", JSONSuffix), clients.Path("alice", "reality", LinkSuffix), + clients.Path("bob", "reality", JSONSuffix), clients.Path("bob", "reality", LinkSuffix), + } + if !changed || !slices.Equal(keys, wantKeys) { + t.Errorf("changed = %v, files = %v, want changed and %v", changed, keys, wantKeys) } if st.Xray["reality/alice"].ID == uuid.Nil { t.Error("alice has no id") @@ -171,6 +205,90 @@ func TestSessionPrepare(t *testing.T) { } } +func TestSessionPrepareHysteriaPinHasNoLink(t *testing.T) { + pinned := `{"tag": "hy2pin", "protocol": "hysteria", "port": 8444, + "x-usher": {"trust": "pin", "pin_sha256": "` + strings.Repeat("ab", 32) + `"}, + "settings": {"version": 2, "clients": []}, + "streamSettings": {"network": "hysteria", "security": "tls", + "tlsSettings": {"serverName": "hy.example.com", "certificates": [{"certificate": ["c"], "key": ["k"]}]}, + "hysteriaSettings": {"version": 2}}},` + f := newSessionFixture(t, func(c *driftConfig) { c.extra = pinned }) + + files, _, err := f.session.Prepare(newState(), "hy2pin", []string{"alice"}) + if err != nil { + t.Fatal(err) + } + if len(files) != 1 || files[clients.Path("alice", "hy2pin", JSONSuffix)] == nil { + t.Errorf("files = %v, want only alice/hy2pin.json", slices.Collect(maps.Keys(files))) + } +} + +// TestSessionBrokenInboundStillRevokes covers the three ways Prepare keeps a reason instead of +// failing: a validation error, a credential of the wrong kind and a file that cannot be rendered. +func TestSessionBrokenInboundStillRevokes(t *testing.T) { + notPEM := `{"tag": "pinned", "protocol": "vless", "port": 9003, + "x-usher": {"trust": "pin"}, + "settings": {"clients": [], "decryption": "none"}, + "streamSettings": {"network": "ws", "security": "tls", + "tlsSettings": {"serverName": "p.example.com", "certificates": [{"certificate": ["x"], "key": ["y"]}]}}},` + tests := []struct { + name string + extra string + tag string + setup func(st *state.State) + wantErr string + }{ + {"validation error", badInbound, "bad", func(*state.State) {}, "settings.clients"}, + {"credential of the wrong kind", "", "ws", func(st *state.State) { + st.Xray["ws/alice"] = state.XrayEntry{Password: "wrong-kind"} + }, "does not fit"}, + {"render error", notPEM, "pinned", func(*state.State) {}, "not a PEM certificate"}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + f := newSessionFixture(t, func(c *driftConfig) { c.extra = tt.extra }) + st := newState() + tt.setup(st) + in := f.inbound(tt.tag) + f.service.seed(in, tt.tag+"/alice", idEntry()) + f.service.seed(in, tt.tag+"/gone", idEntry()) + + files, _, err := f.session.Prepare(st, tt.tag, []string{"alice"}) + if err != nil || files != nil { + t.Fatalf("Prepare = %v, %v, want no files and no error", files, err) + } + f.service.alters = 0 + err = f.session.Apply(t.Context(), st, tt.tag, []string{"alice"}) + if err == nil || !strings.Contains(err.Error(), tt.wantErr) { + t.Fatalf("Apply error = %v, want one with %q", err, tt.wantErr) + } + if got := f.service.credentials(t, in); len(got) != 1 || got[tt.tag+"/alice"] == "" { + t.Errorf("users = %v, want only alice", got) + } + if f.service.alters != 1 { + t.Errorf("AlterInbound calls = %d, want 1 removal", f.service.alters) + } + if !strings.Contains(f.logs.String(), "removed=["+tt.tag+"/gone]") { + t.Errorf("log lacks the removal:\n%s", f.logs) + } + }) + } +} + +func TestSessionPrepareKeepsChangedWhenRenderFails(t *testing.T) { + notPEM := `{"tag": "pinned", "protocol": "vless", "port": 9003, + "x-usher": {"trust": "pin"}, + "settings": {"clients": [], "decryption": "none"}, + "streamSettings": {"network": "ws", "security": "tls", + "tlsSettings": {"serverName": "p.example.com", "certificates": [{"certificate": ["x"], "key": ["y"]}]}}},` + f := newSessionFixture(t, func(c *driftConfig) { c.extra = notPEM }) + st := newState() + _, changed, err := f.session.Prepare(st, "pinned", []string{"alice"}) + if err != nil || !changed || st.Xray["pinned/alice"].ID == uuid.Nil { + t.Errorf("changed = %v, err = %v, entry = %v, want the enrolled entry reported", changed, err, st.Xray["pinned/alice"]) + } +} + func TestSessionApply(t *testing.T) { tests := []struct { name string @@ -195,7 +313,7 @@ func TestSessionApply(t *testing.T) { } }, alter: 2, - logs: []string{"users changed", "added", "reality/alice"}, + logs: []string{"users changed", `added="[reality/alice reality/bob]"`, "removed=[]", "updated=[]"}, }, { name: "no change makes no call", @@ -222,7 +340,7 @@ func TestSessionApply(t *testing.T) { }, want: func(*state.State) map[string]string { return map[string]string{"hy2/alice": "alice-pass"} }, alter: 2, - logs: []string{"removed", "hy2/carol", "stranger"}, + logs: []string{`removed="[hy2/carol stranger]"`, "added=[]", "updated=[]"}, }, { name: "replaces a changed credential", @@ -234,7 +352,7 @@ func TestSessionApply(t *testing.T) { }, want: func(*state.State) map[string]string { return map[string]string{"hy2/alice": "new-pass"} }, alter: 2, - logs: []string{"updated", "hy2/alice"}, + logs: []string{"updated=[hy2/alice]", "added=[]", "removed=[]"}, }, { name: "no users empties the inbound", @@ -245,6 +363,7 @@ func TestSessionApply(t *testing.T) { }, want: func(*state.State) map[string]string { return map[string]string{} }, alter: 1, + logs: []string{"removed=[hy2/alice]"}, }, } for _, tt := range tests { @@ -307,6 +426,145 @@ func TestSessionApplyUnderDrift(t *testing.T) { } } +// openWithConfig opens a session on the service for the file the config writes. +func openWithConfig(t *testing.T, service *userService, cfg driftConfig) *Session { + t.Helper() + dial := func() (command.HandlerServiceClient, func(), error) { return service, func() {}, nil } + session, err := Open(t.Context(), dial, filepath.Join(t.TempDir(), "xray.sock"), cfg.write(t), + "198.51.100.7", slog.New(slog.DiscardHandler)) + if err != nil { + t.Fatal(err) + } + return session +} + +func TestSessionApplyUnusableDriftChangesNothing(t *testing.T) { + tests := []struct { + name string + change func(c *driftConfig) + running func(running []*core.InboundHandlerConfig) []*core.InboundHandlerConfig + want string + }{ + {"protocol change", func(c *driftConfig) { c.realityProto = "trojan" }, nil, "proxy type"}, + {"tag missing at runtime", nil, func(running []*core.InboundHandlerConfig) []*core.InboundHandlerConfig { + return slices.DeleteFunc(running, func(r *core.InboundHandlerConfig) bool { return r.Tag == "reality" }) + }, "not running"}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + f := newSessionFixture(t, nil) + cfg := f.cfg + if tt.change != nil { + tt.change(&cfg) + } + if tt.running != nil { + f.service.running = tt.running(f.service.running) + } + session := openWithConfig(t, f.service, cfg) + f.service.seed(f.inbound("reality"), "reality/gone", idEntry()) + st := newState() + st.Xray["reality/alice"] = idEntry() + f.service.alters = 0 + + err := session.Apply(t.Context(), st, "reality", []string{"alice"}) + if err == nil || !strings.Contains(err.Error(), tt.want) { + t.Fatalf("Apply error = %v, want one with %q", err, tt.want) + } + if f.service.alters != 0 { + t.Errorf("AlterInbound calls = %d, want 0: an account of the wrong type crashes Xray", f.service.alters) + } + }) + } +} + +func TestSessionApplyReportsReconcileAndDriftErrors(t *testing.T) { + f := newSessionFixture(t, nil) + f.cfg.realityPort = 8444 + session := openWithConfig(t, f.service, f.cfg) + f.service.listErr = errors.New("api down") + + err := session.Apply(t.Context(), newState(), "reality", []string{"alice"}) + if err == nil || !strings.Contains(err.Error(), "api down") || !strings.Contains(err.Error(), "port") { + t.Errorf("Apply error = %v, want the list error and the drift reason", err) + } +} + +func TestSessionApplyFailures(t *testing.T) { + t.Run("removal fails, the add is skipped", func(t *testing.T) { + f := newSessionFixture(t, nil) + st := newState() + st.Xray["hy2/alice"] = state.XrayEntry{Password: "new-pass"} + f.service.seed(f.inbound("hy2"), "hy2/alice", state.XrayEntry{Password: "old-pass"}) + f.service.failRemove["hy2/alice"] = true + + err := f.session.Apply(t.Context(), st, "hy2", []string{"alice"}) + if err == nil || !strings.Contains(err.Error(), "removing hy2/alice") { + t.Fatalf("Apply error = %v, want the removal error", err) + } + if got := f.service.credentials(t, f.inbound("hy2")); got["hy2/alice"] != "old-pass" { + t.Errorf("users = %v, want the old account untouched", got) + } + if f.service.alters != 1 { + t.Errorf("AlterInbound calls = %d, want only the failed removal", f.service.alters) + } + }) + t.Run("listing fails", func(t *testing.T) { + f := newSessionFixture(t, nil) + f.service.listErr = errors.New("api down") + err := f.session.Apply(t.Context(), newState(), "hy2", []string{"alice"}) + if err == nil || !strings.Contains(err.Error(), "listing users") { + t.Fatalf("Apply error = %v, want the listing error", err) + } + if f.service.alters != 0 { + t.Errorf("AlterInbound calls = %d, want 0", f.service.alters) + } + }) + t.Run("changed credential, the new add fails", func(t *testing.T) { + f := newSessionFixture(t, nil) + st := newState() + st.Xray["hy2/alice"] = state.XrayEntry{Password: "new-pass"} + f.service.seed(f.inbound("hy2"), "hy2/alice", state.XrayEntry{Password: "old-pass"}) + f.service.failAdd["hy2/alice"] = true + + err := f.session.Apply(t.Context(), st, "hy2", []string{"alice"}) + if err == nil || !strings.Contains(err.Error(), "adding hy2/alice") { + t.Fatalf("Apply error = %v, want the add error", err) + } + if got := f.service.credentials(t, f.inbound("hy2")); len(got) != 0 { + t.Errorf("users = %v, want none: the old account is gone", got) + } + if !strings.Contains(f.logs.String(), "removed=[hy2/alice]") || !strings.Contains(f.logs.String(), "updated=[]") { + t.Errorf("log should report the user as removed, not updated:\n%s", f.logs) + } + }) +} + +func TestOpenClosesConnectionWhenDriftCheckFails(t *testing.T) { + cfg := newDriftConfig(t.TempDir()) + service := newUserService(nil) + service.inboundErr = errors.New("api down") + closed := false + dial := func() (command.HandlerServiceClient, func(), error) { return service, func() { closed = true }, nil } + _, err := Open(t.Context(), dial, filepath.Join(t.TempDir(), "xray.sock"), cfg.write(t), "198.51.100.7", + slog.New(slog.DiscardHandler)) + if err == nil || !strings.Contains(err.Error(), "api down") { + t.Errorf("Open error = %v, want the list error", err) + } + if !closed { + t.Error("Open did not close the connection") + } +} + +func TestOpenDialFails(t *testing.T) { + cfg := newDriftConfig(t.TempDir()) + dial := func() (command.HandlerServiceClient, func(), error) { return nil, nil, errors.New("no socket") } + _, err := Open(t.Context(), dial, filepath.Join(t.TempDir(), "xray.sock"), cfg.write(t), "198.51.100.7", + slog.New(slog.DiscardHandler)) + if err == nil || !strings.Contains(err.Error(), "connecting to xray") { + t.Errorf("Open error = %v, want the connect error", err) + } +} + func TestSessionApplyPartialFailureConverges(t *testing.T) { f := newSessionFixture(t, nil) st := newState() @@ -384,27 +642,6 @@ func TestProtocolCredential(t *testing.T) { } } -// runningValid builds the valid inbounds of the file the way Xray does at start. -func runningValid(t *testing.T, path string) []*core.InboundHandlerConfig { - t.Helper() - inbounds, err := ReadConfig(path) - if err != nil { - t.Fatal(err) - } - var running []*core.InboundHandlerConfig - for _, in := range inbounds { - if in.Err != nil { - continue - } - built, err := in.Detour.Build() - if err != nil { - t.Fatal(err) - } - running = append(running, built) - } - return running -} - func TestOpenStaleSocketReportsDown(t *testing.T) { cfg := newDriftConfig(t.TempDir()) path := cfg.write(t) diff --git a/internal/xray/socket.go b/internal/xray/socket.go index eb0796c..b75fd67 100644 --- a/internal/xray/socket.go +++ b/internal/xray/socket.go @@ -4,38 +4,69 @@ import ( "errors" "fmt" "io/fs" + "net" "os" "syscall" + "time" ) -// removeStaleSocket unlinks the API socket when no Xray holds its lock. Xray holds an exclusive +// dialTimeout bounds the probe of a socket that has no lock file. +const dialTimeout = time.Second + +// removeStaleSocket unlinks the API socket when no Xray listens on it. Xray holds an exclusive // flock on .lock while it listens, so a lock that is free means a killed Xray left the -// socket behind and the next start would fail on it. It reports whether it removed the socket. +// socket behind and the next start would fail on it. A failed start removes the lock file, so a +// socket without one is probed instead: a refused connection means nobody listens. A live Xray +// accepts the probe, and its socket stays. It reports whether it removed the socket. func removeStaleSocket(socketPath string) (bool, error) { - if _, err := os.Lstat(socketPath); errors.Is(err, fs.ErrNotExist) { + before, err := os.Lstat(socketPath) + if errors.Is(err, fs.ErrNotExist) { return false, nil } else if err != nil { return false, fmt.Errorf("checking xray socket: %w", err) } + if before.Mode()&fs.ModeSocket == 0 { + return false, fmt.Errorf("xray socket path %s is not a socket", socketPath) + } + // Xray creates the lock as root with mode 0644, so usher opens it read-only; flock needs no - // write access. A socket without a lock file has no live owner either. + // write access. lock, err := os.Open(socketPath + ".lock") switch { case errors.Is(err, fs.ErrNotExist): + if !connectionRefused(socketPath) { + return false, nil + } case err != nil: return false, fmt.Errorf("opening xray socket lock: %w", err) default: - defer lock.Close() + defer lock.Close() // closing the file releases the flock if err := syscall.Flock(int(lock.Fd()), syscall.LOCK_EX|syscall.LOCK_NB); err != nil { if errors.Is(err, syscall.EWOULDBLOCK) { return false, nil } return false, fmt.Errorf("locking xray socket: %w", err) } - defer syscall.Flock(int(lock.Fd()), syscall.LOCK_UN) //nolint:errcheck // closing the file releases it + } + + // A new Xray may have replaced the socket while usher looked; leave a different file alone. + after, err := os.Lstat(socketPath) + if err != nil || !os.SameFile(before, after) { + return false, nil } if err := os.Remove(socketPath); err != nil && !errors.Is(err, fs.ErrNotExist) { return false, fmt.Errorf("removing stale xray socket: %w", err) } return true, nil } + +// connectionRefused reports whether a unix socket answers a connection with ECONNREFUSED, which is +// what a socket file without a listener does. Any other outcome counts as live. +func connectionRefused(socketPath string) bool { + conn, err := net.DialTimeout("unix", socketPath, dialTimeout) + if err == nil { + conn.Close() + return false + } + return errors.Is(err, syscall.ECONNREFUSED) +} diff --git a/internal/xray/socket_test.go b/internal/xray/socket_test.go index 1cc3ce9..125bb6c 100644 --- a/internal/xray/socket_test.go +++ b/internal/xray/socket_test.go @@ -31,6 +31,9 @@ func staleSocket(t *testing.T) string { func TestRemoveStaleSocket(t *testing.T) { t.Run("free lock removes the socket", func(t *testing.T) { path := staleSocket(t) + if err := os.WriteFile(path+".lock", nil, 0o644); err != nil { + t.Fatal(err) + } removed, err := removeStaleSocket(path) if err != nil || !removed { t.Fatalf("removed=%v err=%v", removed, err) @@ -49,7 +52,7 @@ func TestRemoveStaleSocket(t *testing.T) { t.Fatalf("removed=%v err=%v", removed, err) } }) - t.Run("missing lock file removes the socket", func(t *testing.T) { + t.Run("missing lock file and no listener removes the socket", func(t *testing.T) { path := staleSocket(t) removed, err := removeStaleSocket(path) if err != nil || !removed { @@ -59,6 +62,42 @@ func TestRemoveStaleSocket(t *testing.T) { t.Fatalf("lock file created: %v", err) } }) + t.Run("missing lock file and a live listener keeps the socket", func(t *testing.T) { + dir, err := os.MkdirTemp("", "usher") + if err != nil { + t.Fatal(err) + } + t.Cleanup(func() { os.RemoveAll(dir) }) + path := filepath.Join(dir, "x.sock") + l, err := net.Listen("unix", path) + if err != nil { + t.Fatal(err) + } + defer l.Close() + removed, err := removeStaleSocket(path) + if err != nil || removed { + t.Fatalf("removed=%v err=%v", removed, err) + } + if _, err := os.Lstat(path); err != nil { + t.Fatalf("socket gone: %v", err) + } + }) + t.Run("regular file is an error and stays", func(t *testing.T) { + path := filepath.Join(t.TempDir(), "xray.sock") + if err := os.WriteFile(path, []byte("data"), 0o600); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(path+".lock", nil, 0o600); err != nil { + t.Fatal(err) + } + removed, err := removeStaleSocket(path) + if err == nil || removed { + t.Fatalf("removed=%v err=%v, want an error", removed, err) + } + if _, err := os.Lstat(path); err != nil { + t.Fatalf("file gone: %v", err) + } + }) t.Run("held lock keeps the socket", func(t *testing.T) { path := staleSocket(t) lock, err := os.OpenFile(path+".lock", os.O_RDWR|os.O_CREATE, 0o600) From fb1c08994bd20f325ddf6a951a0f6a521b8e6e57 Mon Sep 17 00:00:00 2001 From: EvaTheSalmon <18466855+EvaTheSalmon@users.noreply.github.com> Date: Wed, 14 Oct 2026 19:23:10 +0300 Subject: [PATCH 20/32] fix(usher): reject stray env and hide state ids Co-Authored-By: Claude Sonnet 5.5 --- cmd/usher/env.go | 14 +++++++++----- cmd/usher/env_test.go | 16 ++++++++++++++-- cmd/usher/main_test.go | 36 +++++++++++++++++++++++++++++++++--- deploy/compose.example.yml | 3 +++ deploy/xray/config.json | 4 ++-- internal/state/state.go | 22 ++++++++++++++++++++++ internal/state/state_test.go | 4 +++- 7 files changed, 86 insertions(+), 13 deletions(-) diff --git a/cmd/usher/env.go b/cmd/usher/env.go index 11686a9..0a3bf82 100644 --- a/cmd/usher/env.go +++ b/cmd/usher/env.go @@ -36,20 +36,24 @@ func environmentFrom(getenv func(string) string) (environment, error) { var awgSocket, xraySocket, xrayConfig string var dns []netip.Addr if getenv("USHER_AWG_SOCKET") != "" { - if awgSocket, err = parseSocketPath("USHER_AWG_SOCKET", getenv("USHER_AWG_SOCKET")); err != nil { + if awgSocket, err = parseAbsPath("USHER_AWG_SOCKET", getenv("USHER_AWG_SOCKET")); err != nil { return environment{}, err } if dns, err = parseDNS(getenv("USHER_DNS")); err != nil { return environment{}, err } + } else if getenv("USHER_DNS") != "" { + return environment{}, errors.New("USHER_DNS is set without USHER_AWG_SOCKET") } if getenv("USHER_XRAY_SOCKET") != "" { - if xraySocket, err = parseSocketPath("USHER_XRAY_SOCKET", getenv("USHER_XRAY_SOCKET")); err != nil { + if xraySocket, err = parseAbsPath("USHER_XRAY_SOCKET", getenv("USHER_XRAY_SOCKET")); err != nil { return environment{}, err } - if xrayConfig, err = parseSocketPath("USHER_XRAY_CONFIG", getenv("USHER_XRAY_CONFIG")); err != nil { + if xrayConfig, err = parseAbsPath("USHER_XRAY_CONFIG", getenv("USHER_XRAY_CONFIG")); err != nil { return environment{}, err } + } else if getenv("USHER_XRAY_CONFIG") != "" { + return environment{}, errors.New("USHER_XRAY_CONFIG is set without USHER_XRAY_SOCKET") } if awgSocket == "" && xraySocket == "" { return environment{}, errors.New("USHER_AWG_SOCKET or USHER_XRAY_SOCKET is required") @@ -90,8 +94,8 @@ func parseIPv4(name, value string) (netip.Addr, error) { return addr, nil } -// parseSocketPath checks that value is an absolute path; it serves the socket and config paths. -func parseSocketPath(name, value string) (string, error) { +// parseAbsPath checks that value is an absolute path; it serves the socket and config paths. +func parseAbsPath(name, value string) (string, error) { if value == "" { return "", fmt.Errorf("%s is required", name) } diff --git a/cmd/usher/env_test.go b/cmd/usher/env_test.go index 0158402..18ddce3 100644 --- a/cmd/usher/env_test.go +++ b/cmd/usher/env_test.go @@ -106,9 +106,21 @@ func TestEnvironmentFrom(t *testing.T) { wantErr: "USHER_XRAY_CONFIG", }, { - name: "xray config without socket", + name: "xray config without any socket", env: map[string]string{"USHER_HOST": "203.0.113.10", "USHER_XRAY_CONFIG": "/etc/xray/config.json"}, - wantErr: "USHER_AWG_SOCKET or USHER_XRAY_SOCKET is required", + wantErr: "USHER_XRAY_CONFIG is set without USHER_XRAY_SOCKET", + }, + { + name: "xray config without xray socket", + env: map[string]string{"USHER_HOST": "203.0.113.10", "USHER_AWG_SOCKET": "/run/awg.sock", + "USHER_DNS": "1.1.1.1", "USHER_XRAY_CONFIG": "/etc/xray/config.json"}, + wantErr: "USHER_XRAY_CONFIG is set without USHER_XRAY_SOCKET", + }, + { + name: "DNS without awg socket", + env: map[string]string{"USHER_HOST": "203.0.113.10", "USHER_DNS": "1.1.1.1", + "USHER_XRAY_SOCKET": "/run/xray/api.sock", "USHER_XRAY_CONFIG": "/etc/xray/config.json"}, + wantErr: "USHER_DNS is set without USHER_AWG_SOCKET", }, { name: "empty DNS", diff --git a/cmd/usher/main_test.go b/cmd/usher/main_test.go index e5cdda3..5e45b21 100644 --- a/cmd/usher/main_test.go +++ b/cmd/usher/main_test.go @@ -3,8 +3,10 @@ package main import ( "bytes" "context" + "encoding/base64" "errors" "log/slog" + "os" "path/filepath" "slices" "strings" @@ -116,7 +118,11 @@ func TestServe(t *testing.T) { } func TestDeployExampleConfig(t *testing.T) { - inbounds, err := xray.ReadConfig(filepath.Join("..", "..", "deploy", "xray", "config.json")) + path := filepath.Join("..", "..", "deploy", "xray", "config.json") + wantTags := []string{"vless-reality", "hysteria2"} + + // As shipped, the placeholders make the Reality inbound fail on its key. + inbounds, err := xray.ReadConfig(path) if err != nil { t.Fatal(err) } @@ -124,7 +130,31 @@ func TestDeployExampleConfig(t *testing.T) { for _, in := range inbounds { tags = append(tags, in.Tag) } - if want := []string{"vless-reality", "hysteria2"}; !slices.Equal(tags, want) { - t.Errorf("tags = %v, want %v", tags, want) + if !slices.Equal(tags, wantTags) { + t.Errorf("tags = %v, want %v", tags, wantTags) + } + if inbounds[0].Err == nil || !strings.Contains(inbounds[0].Err.Error(), "privateKey") { + t.Errorf("shipped Reality inbound error = %v, want one about privateKey", inbounds[0].Err) + } + + // With the placeholder key replaced, every inbound is valid. + data, err := os.ReadFile(path) + if err != nil { + t.Fatal(err) + } + key := base64.RawURLEncoding.EncodeToString(bytes.Repeat([]byte{1}, 32)) + replaced := strings.ReplaceAll(string(data), "REPLACE_WITH_X25519_PRIVATE_KEY", key) + filled := filepath.Join(t.TempDir(), "config.json") + if err := os.WriteFile(filled, []byte(replaced), 0o600); err != nil { + t.Fatal(err) + } + inbounds, err = xray.ReadConfig(filled) + if err != nil { + t.Fatal(err) + } + for _, in := range inbounds { + if in.Err != nil { + t.Errorf("inbound %s: %v", in.Tag, in.Err) + } } } diff --git a/deploy/compose.example.yml b/deploy/compose.example.yml index 24ce986..7caf681 100644 --- a/deploy/compose.example.yml +++ b/deploy/compose.example.yml @@ -35,6 +35,9 @@ services: volumes: # The directory, not the file: usher and Xray then both see a replaced config. - ./xray:/etc/xray:ro + # The certificates and keys that the config names. Only Xray mounts them: usher needs no + # access to a key file, and Xray reloads the files itself. + - ./xray-certs:/etc/xray-certs:ro # A host directory that uid 1000 owns (create it before the first start), not a named volume, # which belongs to root: after a crash usher unlinks the stale API socket, and that needs # write access to the directory. diff --git a/deploy/xray/config.json b/deploy/xray/config.json index 07a62bd..58cb705 100644 --- a/deploy/xray/config.json +++ b/deploy/xray/config.json @@ -41,8 +41,8 @@ // A certificate that a public CA issued for the domain above. "certificates": [ { - "certificateFile": "/etc/xray/certs/REPLACE_WITH_FULLCHAIN.pem", - "keyFile": "/etc/xray/certs/REPLACE_WITH_PRIVKEY.pem" + "certificateFile": "/etc/xray-certs/REPLACE_WITH_FULLCHAIN.pem", + "keyFile": "/etc/xray-certs/REPLACE_WITH_PRIVKEY.pem" } ] }, diff --git a/internal/state/state.go b/internal/state/state.go index c96e997..b8e2aa1 100644 --- a/internal/state/state.go +++ b/internal/state/state.go @@ -82,6 +82,28 @@ type XrayEntry struct { Password string `json:"password,omitempty"` } +// UnmarshalJSON decodes an entry. A malformed id fails with a fixed text, because the UUID parser +// quotes the start of the value it rejects. +func (e *XrayEntry) UnmarshalJSON(data []byte) error { + var raw struct { + ID string `json:"id"` + Password string `json:"password"` + } + if err := json.Unmarshal(data, &raw); err != nil { + return err + } + e.Password = raw.Password + e.ID = uuid.Nil + if raw.ID != "" { + id, err := uuid.Parse(raw.ID) + if err != nil { + return errors.New("id is not a UUID") + } + e.ID = id + } + return nil +} + // EntryName returns the key of a user's entry on an interface or inbound, in State.AWG or // State.Xray. func EntryName(iface, user string) string { diff --git a/internal/state/state_test.go b/internal/state/state_test.go index 77ac0b3..3c4becd 100644 --- a/internal/state/state_test.go +++ b/internal/state/state_test.go @@ -161,6 +161,7 @@ func TestSaveMode(t *testing.T) { func TestLoadRejects(t *testing.T) { secret := "SECRETSECRETSECRET" encodedSecret := base64.StdEncoding.EncodeToString([]byte(secret)) + longSecret := strings.Repeat("LONGSECRET", 5)[:45] // the UUID parser would quote its first 9 characters tests := []struct { name string json string @@ -185,6 +186,7 @@ func TestLoadRejects(t *testing.T) { {"xray neither", `{"version":2,"xray":{"reality/a":{}}}`}, {"xray nil id", `{"version":2,"xray":{"reality/a":{"id":"00000000-0000-0000-0000-000000000000"}}}`}, {"xray bad id", `{"version":2,"xray":{"reality/a":{"id":"` + secret + `"}}}`}, + {"xray long bad id", `{"version":2,"xray":{"reality/a":{"id":"` + longSecret + `"}}}`}, {"not json", `version: 1`}, } for _, tt := range tests { @@ -197,7 +199,7 @@ func TestLoadRejects(t *testing.T) { if err == nil { t.Fatal("Load succeeded, want error") } - for _, stored := range []string{secret, encodedSecret, testKey(1), testKey(2)} { + for _, stored := range []string{secret, encodedSecret, testKey(1), testKey(2), longSecret, longSecret[:9]} { if strings.Contains(err.Error(), stored) { t.Fatalf("error quotes a stored value %q: %v", stored, err) } From 8f334c40da4d9bc048c0085b324965e5635d67cf Mon Sep 17 00:00:00 2001 From: EvaTheSalmon <18466855+EvaTheSalmon@users.noreply.github.com> Date: Wed, 14 Oct 2026 19:23:10 +0300 Subject: [PATCH 21/32] test(e2e): prove xray passes ran and clients up Co-Authored-By: Claude Sonnet 5.5 --- .github/workflows/ci.yml | 2 ++ test/e2e/xray/client_test.go | 25 ++++++++++++++++++++++++- test/e2e/xray/xray_test.go | 33 ++++++++++++++++++++++++++++----- 3 files changed, 54 insertions(+), 6 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 3348385..6cb5d6b 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -61,5 +61,7 @@ jobs: - name: e2e test run: go test -tags e2e -count=1 -v ./test/e2e/ + # Needs no kernel module; runs even when the awg e2e step failed. - name: xray e2e test + if: ${{ !cancelled() }} run: go test -tags e2e -count=1 -v ./test/e2e/xray/ diff --git a/test/e2e/xray/client_test.go b/test/e2e/xray/client_test.go index 9fa6bd2..1a57382 100644 --- a/test/e2e/xray/client_test.go +++ b/test/e2e/xray/client_test.go @@ -224,8 +224,31 @@ func allWork(user string, among []route) error { return errors.Join(errs...) } -// noneWorks returns nil when every route of the list fails to return the page. +// proxyReachable reports why the proxy port of the route on the user's client is not open. +func proxyReachable(user string, r route) error { + conn, err := net.DialTimeout("tcp", net.JoinHostPort(proxyHosts[user], strconv.Itoa(r.port)), requestTimeout) + if err != nil { + return fmt.Errorf("%s of %s: proxy is not reachable: %w", r, user, err) + } + return conn.Close() +} + +// proxiesReachable returns nil when the proxy port of every route of the list is open. +func proxiesReachable(user string, among []route) error { + var errs []error + for _, r := range among { + errs = append(errs, proxyReachable(user, r)) + } + return errors.Join(errs...) +} + +// noneWorks returns nil when the proxy of every route of the list is reachable and the request +// through it fails. A refused connection to the proxy of a client that is still starting is an +// error, not a failed route. func noneWorks(user string, among []route) error { + if err := proxiesReachable(user, among); err != nil { + return err + } results := fetchAll(user, among) var errs []error for _, r := range among { diff --git a/test/e2e/xray/xray_test.go b/test/e2e/xray/xray_test.go index d1af9a8..8a73982 100644 --- a/test/e2e/xray/xray_test.go +++ b/test/e2e/xray/xray_test.go @@ -249,6 +249,9 @@ func startClient(t *testing.T, user string) { // one transient error cannot stand for a removed user. func assertOffRepeatedly(t *testing.T, user string, among []route) { t.Helper() + waitFor(t, connectTimeout, "the proxies of the client of "+user, func() error { + return proxiesReachable(user, among) + }) for range failedRounds { if err := noneWorks(user, among); err != nil { t.Fatal(err) @@ -397,15 +400,32 @@ func testDrift(t *testing.T) { return allWork("alice", routesOf(tagReality)) }) - // With the file back in step with xray, the next tick has no failure. + // With the file back in step with xray, no tick fails. Adding bob proves that a pass ran and + // opened xray after the file was restored. if err := srv.writeConfig(project.path(xrayDir), realityShortID); err != nil { t.Fatal(err) } time.Sleep(2 * time.Second) mark = markLogs(t) time.Sleep(refillTick) - if failed := logsSince(t, mark); strings.Contains(failed, "interface failed") { - t.Fatalf("a tick failed after the drift was undone:\n%s", failed) + writeUsherConfig(t, "alice", "bob") + for _, tag := range tags { + waitForLog(t, mark, changeTimeout, addedLine(tag, "bob")) + } + assertNoFailureSince(t, mark, "the drift was undone") +} + +// assertNoFailureSince fails the test when usher logged a failed interface, a backend that is down +// or any error after the mark. The caller first makes usher log a change, which proves that a +// pass ran: a pass that succeeds logs nothing else. +func assertNoFailureSince(t *testing.T, mark logMark, what string) { + t.Helper() + time.Sleep(2 * time.Second) // the pass that logged the change finishes its other inbounds + logs := logsSince(t, mark) + for _, bad := range []string{"interface failed", "backend is down", "level=ERROR"} { + if strings.Contains(logs, bad) { + t.Fatalf("usher logged %q after %s:\n%s", bad, what, logs) + } } } @@ -431,9 +451,12 @@ func testCertificateRewrite(t *testing.T) { t.Fatal(err) } time.Sleep(refillTick) - if failed := logsSince(t, mark); strings.Contains(failed, "interface failed") { - t.Fatalf("a tick failed after the certificate was rewritten:\n%s", failed) + // Removing bob proves that a pass ran and opened xray after the rewrite. + writeUsherConfig(t, "alice") + for _, tag := range tags { + waitForLog(t, mark, changeTimeout, removedLine(tag, "bob")) } + assertNoFailureSince(t, mark, "the certificate was rewritten") waitFor(t, restartTimeout, "a page through ws after the rewrite", func() error { return allWork("alice", routesOf(tagWS)) }) From 311a8a3830d924dc866de5870c299633cd7818f1 Mon Sep 17 00:00:00 2001 From: EvaTheSalmon <18466855+EvaTheSalmon@users.noreply.github.com> Date: Wed, 14 Oct 2026 19:23:10 +0300 Subject: [PATCH 22/32] docs(xray): correct plan, README and architecture Co-Authored-By: Claude Sonnet 5.5 --- AGENTS.md | 8 +++-- README.md | 40 +++++++++++++++++----- docs/architecture/overview.md | 37 +++++++++++++------- docs/plans/20261014-xray-config-backend.md | 27 ++++++++++----- 4 files changed, 81 insertions(+), 31 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 047fcea..9ece50e 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -57,8 +57,11 @@ tar adds `._*` AppleDouble files. A run killed by a signal leaves containers up; ```sh docker compose -f deploy/compose.e2e.yml --profile handshake down -v --remove-orphans +docker compose -f deploy/compose.e2e-xray.yml --profile client down -v --remove-orphans ``` +The Xray suite also takes its project down at start. + ## Format ```sh @@ -82,7 +85,8 @@ Enable the pre-commit hook per clone: `git config core.hooksPath scripts/git-hoo `.github/workflows/ci.yml` runs on every pull request and every push to `master`. - `check`: `scripts/format.sh`, `go test -race`, shellcheck. -- `e2e`: loads the pinned kernel module with `scripts/ci/module.sh`, then runs the e2e suite. +- `e2e`: loads the pinned kernel module with `scripts/ci/module.sh`, runs `./test/e2e/` (awg), then + `./test/e2e/xray/`. The Xray step needs no module and runs even when the awg step failed. The module pin is `AWG_MODULE_REF` in `ci.yml`. @@ -100,7 +104,7 @@ the notes of the annotated tag. - usher never writes the Xray server config. It owns the users of the managed inbounds and `clients/`. - `state/users.json` is the only source of secrets. The kernel and `clients/` derive from it, and - usher saves it before `ApplyPeers`. + usher saves it before any backend's `Apply`. - A failure of one interface never stops the daemon; the pass logs it and goes on. - No interface without a second implementation, no exported surface only for tests. diff --git a/README.md b/README.md index 02e8fa2..fa41562 100644 --- a/README.md +++ b/README.md @@ -31,7 +31,8 @@ bob: [] - Removing a name, or setting it to `[]`, switches the user off. The keys stay, and putting the name back restores the same config. - A broken file changes nothing: usher keeps the last valid one and logs the error with its line. -- A service that restarts empty is refilled within 30 s without any change to the file. +- A service that restarts empty is refilled at the next refill tick (every 30 s; Xray measured + 28 to 32 s, bounded at 45 s in the e2e) without any change to the file. Client files appear in `clients//`: `.conf` for AmneziaWG, `.json` and `.txt` for Xray. Secrets live in `state/users.json`, the only copy of every key: keep the state volume in @@ -54,14 +55,25 @@ Images are `ghcr.io/mrcsin/usher:`, one per release. Drop the service you d usher the socket group of the service containers. To issue new keys for a user, stop usher, delete its `/` entry from -`users.json` and start it again. +`users.json` and start it again. The Xray entries are under `xray` and named `/`. If you +change an inbound between a protocol with a UUID (VLESS, VMess) and one with a password (Trojan, +Hysteria2), the stored credentials no longer fit: usher reports "stored credential does not fit" +and keeps the old client files until you stop usher, delete the `/` entries of that tag +and start it again. + +### Upgrading + +- `USHER_AWG_SOCKET` is required for the AmneziaWG backend; earlier releases hardcoded the path. + Set it to the awg-grpc socket (`/run/awg-grpc/awg.sock` in the example) before you upgrade. +- The first save writes `users.json` as version 2 and copies the old file to `users.json.v1`. To + roll back to an earlier image, stop usher and rename `users.json.v1` back to `users.json`. ## Xray You write the Xray server config and usher fills its inbounds with users. An inbound tag is the interface name in `usher.yml`, up to 32 characters. usher reads the config file and puts users into the running inbounds through Xray's `HandlerService`. It never writes the config. Xray keeps -users in memory only, so usher refills them after a restart within 30 s. +users in memory only, so usher refills them at the next refill tick after a restart. Per user and inbound usher writes `clients//.json`, a full client config with a local socks proxy on 127.0.0.1:10808 and an http proxy on 127.0.0.1:10809, and `clients//.txt`, @@ -76,7 +88,12 @@ a share link. `/run/xray/api.sock,0660`. The Xray process group must equal usher's group. - The socket directory is a bind mount that usher can write. After a hard kill Xray leaves a stale socket and fails to start on it. usher unlinks the socket at its next tick, and Xray's restart - policy (`restart: unless-stopped`) starts it again. + policy (`restart: unless-stopped`) starts it again. usher unlinks a socket only when no process + holds the `.lock` file that Xray keeps beside it, or, when that file is gone (a failed + start removes it), when a connection to the socket is refused. A live Xray keeps its socket. +- Keep certificates and key files outside the directory that usher mounts. usher does not read + them, and Xray reloads them itself. The example mounts `./xray-certs` into the Xray container + only. - The Xray image follows `xray-core` in `go.mod` (now `ghcr.io/xtls/xray-core:26.10.10`). Another version can change how Xray builds an inbound and put every inbound in drift. - Every managed inbound has a tag, a single integer `port`, no unix `listen` and an empty @@ -119,14 +136,16 @@ optional, and an unknown key fails the inbound. | ------------- | ------------------------------------ | ---------------------------------------------------- | | `address` | `USHER_HOST` | address clients dial | | `port` | the inbound `port` | port clients dial | -| `sni` | `tlsSettings.serverName`, or the first Reality `serverNames` entry without `*` | name clients send | +| `sni` | `tlsSettings.serverName`, or the first Reality `serverNames` entry without `*` | name clients send; on Reality one of the `serverNames` entries | | `fingerprint` | `chrome` | uTLS fingerprint; Reality allows `chrome`, `firefox`, `safari` | | `trust` | `ca` | `ca` or `pin`, see below | | `pin_sha256` | hash of the inline certificate | hex SHA-256 of the leaf; only with `trust: pin` | | `hop_ports` | none | Hysteria2 port range `from-to`, from < to | -A TLS inbound with neither `sni` nor `tlsSettings.serverName` fails. An explicit `sni` also -replaces the Reality `serverNames` rule, for a config that lists only wildcard names. +A TLS inbound with neither `sni` nor `tlsSettings.serverName` fails. Reality matches the name by +exact lookup in `serverNames`, so on Reality an `sni` that is not literally one of its entries +fails the inbound, and so does a config with only wildcard names. The privateKey and the +`mldsa65Seed` must be base64url keys of the right size, and the seed must differ from the key. ### TLS trust @@ -159,7 +178,12 @@ usher compares each inbound with the running Xray at every pass. After you edit in the file, restart Xray. Until then usher logs a drift reason that names the part (port, proxy settings, stream settings), keeps the old client files and still removes users who left `usher.yml`. Certificate files that the config names are reloaded by Xray itself and cause no -drift. +drift. When the running inbound is missing or runs another protocol, usher changes nothing on it, +because an account of the wrong type crashes Xray; restart Xray. + +An inbound that fails validation (an `x-usher` typo, a bad Reality key) also keeps its old client +files, and usher still removes the users who left `usher.yml` from it. It adds no user and +changes no credential until you fix the file. ## License diff --git a/docs/architecture/overview.md b/docs/architecture/overview.md index b190986..5940e12 100644 --- a/docs/architecture/overview.md +++ b/docs/architecture/overview.md @@ -45,7 +45,8 @@ in both. The steps below run per backend; the text names the awg calls. 5. For each interface that did not fail: call `ApplyPeers` with one peer per referencing user (`address/32`), with `allow_empty` when no user references the interface. An error fails the interface. -6. Log every user and interface pair whose interface `awg-grpc` does not report. +6. When no backend is down, log every user and interface pair that no backend reports ("interface + is not reported by any backend"). 7. Make `clients/` equal to the rendered files of applied interfaces plus the current files of failed ones. @@ -61,7 +62,7 @@ pass and every call has a 10 s deadline. - `state/users.json` is the only source of secrets. Entries are keyed `interface/user` under the `awg` group. The kernel and `clients/` derive from it. usher saves it (temp file, fsync, rename, - directory fsync, mode 0600) only when an entry changed, and always before `ApplyPeers`. + directory fsync, mode 0600) only when an entry changed, and always before any backend's `Apply`. - A new entry gets an X25519 key pair (`crypto/ecdh`), 32 random PSK bytes and an address. - The address is the lowest host address of the interface's first IPv4 prefix that is not the network or broadcast address, not an interface address and not held by another entry of that @@ -79,11 +80,13 @@ pass and every call has a 10 s deadline. - A file without any user and interface pair is invalid, so a truncated file cannot wipe a node. - A failure of one interface never stops the daemon and never blocks the others. - An interface without an IPv4 prefix fails only when a user references it. -- `clients/` files of an interface that `awg-grpc` no longer reports are removed. +- `clients/` files of an interface that no backend reports are removed. - `USHER_HOST` is an IPv4 address, not a name: a client whose DNS points into the tunnel cannot resolve the name again to reconnect. - No private key, PSK or `client_params` value is logged. A public key without an entry is logged as base64. +- No Reality `privateKey` or `mldsa65Seed`, key file, `sockopt`, `fallbacks` entry or drift value + is logged or written to `clients/`. - No subcommands besides `usher run`, no central management, no migration of existing peers. ## Operations @@ -117,16 +120,21 @@ pass and every call has a 10 s deadline. Xray keeps users in memory, so the refill tick restores them after a restart. - A managed inbound has a tag that matches the interface name pattern (32 characters), one integer port, no unix `listen`, empty `clients` and a protocol of vless, hysteria, vmess or trojan. - Other inbounds are skipped. An inbound that fails validation carries its error: `Prepare` - returns it, the pass skips its `Apply` and keeps its files. + Other inbounds are skipped. An inbound that fails validation carries its error. `Prepare` keeps + that error (as it does for an enroll or render error) and returns none, so the pass still calls + `Apply`. `Apply` then only removes the users who left, and returns the error, which keeps the old + files. A tag defined twice is reported once, and its entry carries the error. ### Reading the config `ReadConfig` reads the file with Xray's comment-aware reader into `infra/conf` types plus raw maps. It normalizes the aliases (`method`, `splithttp`, `websocket`, `tcp`, `splithttpSettings`, `tcpSettings`, Hysteria `users`) before validation and mirroring. Every error names the field and -never a value. `x-usher` decodes with unknown keys rejected. An explicit `x-usher.sni` relaxes the -Reality `serverNames` rule. `hop_ports` requires from < to. ReadConfig registers a discard handler +never a value. `x-usher` decodes with unknown keys rejected. Reality matches names by exact lookup, +so there an explicit `x-usher.sni` must be one of the `serverNames` entries, and the config needs +an entry without a wildcard when `sni` is unset. A Reality `privateKey` and `mldsa65Seed` must +decode, and the seed must differ from the key. `hop_ports` requires from < to. An inbound without +`streamSettings` stays without them in `Raw`, so it builds as Xray builds the file. ReadConfig registers a discard handler for Xray's global logger once, because `Build()` prints deprecation warnings. ### Users @@ -138,6 +146,8 @@ for Xray's global logger once, because `Build()` prints deprecation warnings. VLESS `id` and `flow`, VMess `id`, Trojan `password`, Hysteria `auth`. It removes foreign and changed emails and adds missing and changed ones. It tries every operation, joins the errors and logs `users changed`. The next pass converges from the listed state. +- When the running inbound is missing or runs another protocol, `Apply` changes nothing: an + account of the wrong type makes Xray's validator panic and kills the process. - Removal stops new connections. An open Hysteria2 QUIC connection stays until the client reconnects. @@ -145,10 +155,11 @@ for Xray's global logger once, because `Build()` prints deprecation warnings. `Open` calls `ListInbounds` once and compares each valid inbound with `Build()` of its file entry: proxy type, proxy settings, port, stream settings and receiver settings. Certificate entries that -name a file are dropped on both sides, because usher cannot read the key files and Xray reloads -them. A tag missing at runtime counts as drift. The reason names the differing parts. `Apply` -reconciles users first and then returns the drift error, so revocation works while the old client -files stay. A running managed inbound that the file does not define is logged each pass. +name a file are dropped on both sides, because Xray reloads those files itself. A tag missing at +runtime counts as drift. The reason names the differing parts; an error of `Build()` becomes a fixed +text, because Xray quotes the keys and short ids it rejects. `Apply` reconciles users first and then +returns the drift error, so revocation works while the old client files stay. The exception is a +missing inbound or another proxy type (see Users). A running managed inbound that the file does not define is logged each pass. ### Stale socket @@ -156,7 +167,9 @@ A killed Xray leaves its API socket, and a new start fails on it. Xray holds `fl `.lock` while it listens. Before dialing, usher opens the lock read-only and tries `LOCK_EX|LOCK_NB`. When it gets the lock, no Xray listens: usher unlinks the socket and reports Xray down for this pass. The restart policy starts Xray at its next attempt. The socket directory -is a bind mount that usher can write. +is a bind mount that usher can write. A failed Xray start removes the lock file, so for a socket without +one usher probes the socket instead: only a refused connection counts as stale, and a live Xray, or +a socket that a new Xray replaced meanwhile, stays. A path that is not a socket is an error. ### Client mirror diff --git a/docs/plans/20261014-xray-config-backend.md b/docs/plans/20261014-xray-config-backend.md index 93b39bb..9e622ab 100644 --- a/docs/plans/20261014-xray-config-backend.md +++ b/docs/plans/20261014-xray-config-backend.md @@ -164,13 +164,16 @@ reported by awg-grpc" and nothing reaches Xray. - after one user leaves `usher.yml`, new connections through that user fail and the other user still works; - after `docker compose restart xray`, connections work again within 45 s; - - after `docker kill -s KILL` of Xray, the restart policy restarts it, it fails on the stale - socket until usher unlinks it at its next tick, and connections work again within 90 s; + - after SIGKILL of the Xray process (sent from a helper container in its PID namespace, because + `docker kill` bypasses the restart policy), the restart policy restarts it, it fails on the + stale socket until usher unlinks it at its next tick, and connections work again within 90 s; - changing `shortIds` in the mounted config without restarting Xray fails that inbound with a drift reason at the next tick, and a user removed from `usher.yml` while that drift is present is still removed from Xray; - rewriting the `certificateFile` with a new leaf from the same CA causes no drift. 4. awg e2e: `go test -tags e2e -count=1 -v ./test/e2e/` still passes on a host with the module. + Not run: no available host has the `amneziawg` module, so this item is deferred to the CI `e2e` + job and has no evidence yet. 5. Manual smoke on a public host: import one VLESS+Reality link and one Hysteria2 link into v2rayNG and open a page through each. @@ -214,11 +217,12 @@ reported by awg-grpc" and nothing reaches Xray. usher. - **Drift guard.** `Open` calls `ListInbounds` once and compares each managed inbound with `Build()` of its file entry: proxy type, proxy settings and receiver settings. Certificate - entries with a `certificateFile` are removed from the file entry before `Build()` (usher has no - access to the key files) and entries with a `CertificatePath` from the runtime side; Xray + entries with a `certificateFile` are removed from the file entry before `Build()` (Xray reloads + those files itself, so a changed file is no drift) and entries with a `CertificatePath` from the runtime side; Xray reloads those files itself. A tag in the file but not at runtime also counts as drift. `Apply` reconciles users regardless and then returns the drift error, so revocation keeps working while - the old client files stay. The drift reason names the differing part (port, proxy type, proxy + the old client files stay; when the running inbound is missing or runs another proxy type, + `Apply` changes nothing, because an account of the wrong type panics Xray. The drift reason names the differing part (port, proxy type, proxy settings, stream settings), never a value. - **Client mirror.** - Taken from the server: `network`, `security`; from `wsSettings`, `grpcSettings` and @@ -332,8 +336,9 @@ reported by awg-grpc" and nothing reaches Xray. - [x] keep the awg behaviour of `master` unchanged - [x] write tests for the seam: two fake backends, a collision, a down backend with two suffixes, the `USHER_DNS` rule -- [x] run `go test -race ./...` and the awg e2e (CI) - must pass before Task 2 (awg e2e deferred - to CI, not run locally) +- [x] run `go test -race ./...` - passes +- [ ] run the awg e2e - deferred to the CI `e2e` job; it has not run on any host, because no + available host has the `amneziawg` module ### Task 2: users.json version 2 with Xray credentials @@ -482,7 +487,10 @@ reported by awg-grpc" and nothing reaches Xray. - [x] verify edge cases are handled - [x] run full test suite: `go test -race -count=1 ./...` - [x] run `scripts/format.sh`, shellcheck and actionlint -- [x] run e2e tests: `go test -tags e2e -count=1 -v ./test/e2e/...` (CI) (awg e2e deferred to CI, not run locally; Xray e2e passed on ger in Task 10, code unchanged since) +- [x] run the Xray e2e: `go test -tags e2e -count=1 -v ./test/e2e/xray/` (passed on ger; re-run + there after the review fixes) +- [ ] run the awg e2e: `go test -tags e2e -count=1 -v ./test/e2e/` - deferred to the CI `e2e` job, + not run on any host - [x] verify test coverage meets project standard (go test -cover: xray 90.9%, awg 96.6%, config 97.6%, pass 97.9%, state 91.0%, watch 100%, clients 78.9%, cmd/usher 73.2%, atomicfile 61.5%) ### Task 12: [Final] Update documentation @@ -494,7 +502,8 @@ reported by awg-grpc" and nothing reaches Xray. - [x] `docs/architecture/overview.md`: who owns which JSON, the seam, the mirror, drift guard, state v2, the stale socket rule - [x] `AGENTS.md`: layout, the Xray e2e, never-logged fields -- [x] move this plan to `docs/plans/completed/` (left in place: archiving is a delivery step) +- [ ] move this plan to `docs/plans/completed/` - not done, left in place: archiving is a delivery + step ## Post-Completion From 945767c970a88f52df6f74363755c9df07bd10e5 Mon Sep 17 00:00:00 2001 From: EvaTheSalmon <18466855+EvaTheSalmon@users.noreply.github.com> Date: Wed, 14 Oct 2026 19:40:54 +0300 Subject: [PATCH 23/32] fix(xray): leave untrusted inbounds untouched Prepare returns structural validation errors so the pass skips Apply. Only x-usher hint, enroll and render errors still revoke. Drift compares the proxy type before Build and marks a failed comparison unusable, so no account reaches a handler of another type. Co-Authored-By: Claude Sonnet 5.5 --- README.md | 15 ++++--- docs/architecture/overview.md | 24 +++++++----- internal/xray/config.go | 14 +++++-- internal/xray/drift.go | 51 +++++++++++++++++------- internal/xray/drift_test.go | 5 +++ internal/xray/session.go | 18 ++++++--- internal/xray/session_test.go | 74 ++++++++++++++++++++++++++++++++++- 7 files changed, 160 insertions(+), 41 deletions(-) diff --git a/README.md b/README.md index fa41562..2547a2f 100644 --- a/README.md +++ b/README.md @@ -178,12 +178,15 @@ usher compares each inbound with the running Xray at every pass. After you edit in the file, restart Xray. Until then usher logs a drift reason that names the part (port, proxy settings, stream settings), keeps the old client files and still removes users who left `usher.yml`. Certificate files that the config names are reloaded by Xray itself and cause no -drift. When the running inbound is missing or runs another protocol, usher changes nothing on it, -because an account of the wrong type crashes Xray; restart Xray. - -An inbound that fails validation (an `x-usher` typo, a bad Reality key) also keeps its old client -files, and usher still removes the users who left `usher.yml` from it. It adds no user and -changes no credential until you fix the file. +drift. When the running inbound is missing, runs another protocol or cannot be compared with the +file, usher changes nothing on it, because an account of the wrong type crashes Xray; restart Xray. + +An inbound with an `x-usher` error, or one whose credential or client file cannot be built, keeps +its old client files, and usher still removes the users who left `usher.yml` from it. It adds no +user and changes no credential until you fix the file. An inbound that fails any other check (port, +`listen`, non-empty `clients`, transport, Reality keys, a tag that is not a valid interface name +or that the file defines twice) is left alone: usher cannot trust the file entry, so it adds and +removes no user on that tag and keeps its old client files. ## License diff --git a/docs/architecture/overview.md b/docs/architecture/overview.md index 5940e12..93a2abb 100644 --- a/docs/architecture/overview.md +++ b/docs/architecture/overview.md @@ -120,10 +120,14 @@ pass and every call has a 10 s deadline. Xray keeps users in memory, so the refill tick restores them after a restart. - A managed inbound has a tag that matches the interface name pattern (32 characters), one integer port, no unix `listen`, empty `clients` and a protocol of vless, hysteria, vmess or trojan. - Other inbounds are skipped. An inbound that fails validation carries its error. `Prepare` keeps - that error (as it does for an enroll or render error) and returns none, so the pass still calls - `Apply`. `Apply` then only removes the users who left, and returns the error, which keeps the old - files. A tag defined twice is reported once, and its entry carries the error. + Other inbounds are skipped. An inbound that fails validation carries its error. A structural error + (port, listen, clients, decryption, transport, Hysteria auth, masks, xhttp, Reality keys, a + decode failure, an invalid or duplicate tag) is returned by `Prepare`, so the pass skips `Apply` + and touches no user of the tag; `Apply` also returns it if called. An `x-usher` hint error, like + an enroll or render error, is kept by `Prepare`, which returns none: the running inbound is known + good, so the pass still calls `Apply`, which only removes the users who left and returns the + error, which keeps the old files. `validate` runs the hint check last, so a hint error is the only + reason left. A tag defined twice is reported once, and its entry carries the error. ### Reading the config @@ -146,8 +150,8 @@ for Xray's global logger once, because `Build()` prints deprecation warnings. VLESS `id` and `flow`, VMess `id`, Trojan `password`, Hysteria `auth`. It removes foreign and changed emails and adds missing and changed ones. It tries every operation, joins the errors and logs `users changed`. The next pass converges from the listed state. -- When the running inbound is missing or runs another protocol, `Apply` changes nothing: an - account of the wrong type makes Xray's validator panic and kills the process. +- When the running inbound is missing, runs another protocol or could not be compared, `Apply` + changes nothing: an account of the wrong type makes Xray's validator panic and kills the process. - Removal stops new connections. An open Hysteria2 QUIC connection stays until the client reconnects. @@ -156,10 +160,12 @@ for Xray's global logger once, because `Build()` prints deprecation warnings. `Open` calls `ListInbounds` once and compares each valid inbound with `Build()` of its file entry: proxy type, proxy settings, port, stream settings and receiver settings. Certificate entries that name a file are dropped on both sides, because Xray reloads those files itself. A tag missing at -runtime counts as drift. The reason names the differing parts; an error of `Build()` becomes a fixed +runtime counts as drift. The proxy type of the running handler is compared with the file's +protocol before any `Build()`; a mismatch, or a file entry that does not decode or build, marks the +drift unusable. The reason names the differing parts; an error of `Build()` becomes a fixed text, because Xray quotes the keys and short ids it rejects. `Apply` reconciles users first and then -returns the drift error, so revocation works while the old client files stay. The exception is a -missing inbound or another proxy type (see Users). A running managed inbound that the file does not define is logged each pass. +returns the drift error, so revocation works while the old client files stay. The exception is an +unusable drift: a missing inbound, another proxy type or a failed comparison (see Users). A running managed inbound that the file does not define is logged each pass. ### Stale socket diff --git a/internal/xray/config.go b/internal/xray/config.go index bc193d4..1504b01 100644 --- a/internal/xray/config.go +++ b/internal/xray/config.go @@ -36,6 +36,9 @@ type Inbound struct { Hints Hints Raw map[string]any Err error + // hintsOnly is true when Err comes from the x-usher block alone: the rest of the inbound is + // sound, so the running inbound can be trusted to match it. + hintsOnly bool } // Hints is the x-usher block of an inbound with the defaults that do not depend on usher's @@ -191,10 +194,6 @@ func (in *Inbound) validate() error { return fail("streamSettings.realitySettings."+field, reason) } } - if field, reason := in.readHints(stream); field != "" { - return fail(field, reason) - } - // The decode catches field type errors that the checks above do not look at. Xray's decoders // can quote the value they reject, so the error names the Go field of a type mismatch at most. encoded, err := json.Marshal(in.Raw) @@ -209,6 +208,13 @@ func (in *Inbound) validate() error { } return fail("inbound", "does not decode as an Xray inbound") } + + // Last, so that a hint error is the only reason left: the inbound is sound and the running + // Xray can be trusted to match it. + if field, reason := in.readHints(stream); field != "" { + in.hintsOnly = true + return fail(field, reason) + } return nil } diff --git a/internal/xray/drift.go b/internal/xray/drift.go index 7bf3c17..362f5b1 100644 --- a/internal/xray/drift.go +++ b/internal/xray/drift.go @@ -20,7 +20,8 @@ import ( ) // driftError is the reason a running inbound differs from the file. unusable is true when the -// running inbound cannot take the file's users: it is missing or runs another protocol. +// running inbound may not take the file's users: it is missing, runs another protocol or could not +// be compared. type driftError struct { reason string unusable bool @@ -32,6 +33,28 @@ func (e *driftError) Error() string { return e.reason } // ids it rejects, so its text never reaches a reason. var errDoesNotBuild = errors.New("the file inbound does not build as an Xray inbound") +// buildInbound builds the inbound object the way Xray does, without the certificate entries that +// name a file, which Xray reads from its own filesystem. +func buildInbound(raw map[string]any) (*core.InboundHandlerConfig, error) { + raw, err := withoutFileCertificates(raw) + if err != nil { + return nil, err + } + encoded, err := json.Marshal(raw) + if err != nil { + return nil, fmt.Errorf("encoding the file inbound: %w", err) + } + var detour conf.InboundDetourConfig + if err := json.Unmarshal(encoded, &detour); err != nil { + return nil, errDoesNotBuild + } + built, err := detour.Build() + if err != nil { + return nil, errDoesNotBuild + } + return built, nil +} + // isManagedProxyType reports whether proxyType is the proxy settings type of a protocol the // backend manages. func isManagedProxyType(proxyType string) bool { @@ -77,11 +100,21 @@ func checkDrift(ctx context.Context, client command.HandlerServiceClient, log *s } continue } + if live.GetProxySettings().GetType() != protocols[in.Protocol].proxyType { + drift[in.Tag] = &driftError{ + reason: fmt.Sprintf("inbound %q: differs from the running Xray in %s, restart Xray to load the file", + in.Tag, partProxyType), + unusable: true, + } + continue + } parts, err := differingParts(in, live) switch { case err != nil: + // Nothing proves the running inbound takes the file's accounts, so Apply stays away. drift[in.Tag] = &driftError{ - reason: fmt.Sprintf("inbound %q: comparing with the running Xray: %v", in.Tag, err), + reason: fmt.Sprintf("inbound %q: comparing with the running Xray: %v", in.Tag, err), + unusable: true, } case len(parts) > 0: drift[in.Tag] = &driftError{ @@ -105,22 +138,10 @@ const partProxyType = "proxy type" // differingParts returns the names of the parts of the file inbound that differ from the running // one: "proxy type", "proxy settings", "port", "stream settings" and "receiver settings". func differingParts(in Inbound, live *core.InboundHandlerConfig) ([]string, error) { - raw, err := withoutFileCertificates(in.Raw) + built, err := buildInbound(in.Raw) if err != nil { return nil, err } - encoded, err := json.Marshal(raw) - if err != nil { - return nil, fmt.Errorf("encoding the file inbound: %w", err) - } - var detour conf.InboundDetourConfig - if err := json.Unmarshal(encoded, &detour); err != nil { - return nil, errDoesNotBuild - } - built, err := detour.Build() - if err != nil { - return nil, errDoesNotBuild - } var parts []string if built.GetProxySettings().GetType() != live.GetProxySettings().GetType() { diff --git a/internal/xray/drift_test.go b/internal/xray/drift_test.go index 453d261..e9b640a 100644 --- a/internal/xray/drift_test.go +++ b/internal/xray/drift_test.go @@ -345,6 +345,11 @@ func TestCheckDriftUnusable(t *testing.T) { }{ {"protocol change", func(c *driftConfig) { c.realityProto = "vmess" }, true}, {"port change", func(c *driftConfig) { c.realityPort = 8444 }, false}, + {"file does not build", func(c *driftConfig) { c.shortID = "zz" }, true}, + {"protocol change and file does not build", func(c *driftConfig) { + c.realityProto = "vmess" + c.shortID = "zz" + }, true}, } for _, tt := range tests { t.Run(tt.name, func(t *testing.T) { diff --git a/internal/xray/session.go b/internal/xray/session.go index d8d60ea..61f1141 100644 --- a/internal/xray/session.go +++ b/internal/xray/session.go @@ -97,12 +97,17 @@ func (s *Session) Interfaces() []string { } // Prepare enrolls the users on the inbound in st and renders their client files. When the inbound -// failed validation, an enrolled credential does not fit it or a file cannot be rendered, Prepare -// keeps the reason and returns no error: Apply then still revokes the users who left, and returns -// the reason, which keeps the old client files. +// failed structural validation, Prepare returns the error and the pass skips Apply, so usher +// touches no user of that tag. When only the x-usher block failed, an enrolled credential does not +// fit the inbound or a file cannot be rendered, the running inbound is known good: Prepare keeps +// the reason and returns no error, and Apply then still revokes the users who left and returns the +// reason, which keeps the old client files. func (s *Session) Prepare(st *state.State, name string, users []string) (map[string][]byte, bool, error) { in := s.byTag[name] if in.Err != nil { + if !in.hintsOnly { + return nil, false, in.Err + } s.broken[name] = in.Err return nil, false, nil } @@ -126,9 +131,12 @@ func (s *Session) Prepare(st *state.State, name string, users []string) (map[str // Apply makes the inbound hold exactly the users, then returns the error of the tag, if any: the // reason Prepare kept, else the drift. Users are reconciled even under most drift, so a revoked // user loses access. When the running inbound is missing or runs another protocol, Apply changes -// nothing, because an account of the wrong type crashes Xray. When Prepare kept a reason, Apply only -// removes users. +// nothing, because an account of the wrong type crashes Xray. It also changes nothing when the +// inbound failed structural validation. When Prepare kept a reason, Apply only removes users. func (s *Session) Apply(ctx context.Context, st *state.State, name string, users []string) error { + if in := s.byTag[name]; in.Err != nil && !in.hintsOnly { + return in.Err + } drift := s.drift[name] var driftErr *driftError if errors.As(drift, &driftErr) && driftErr.unusable { diff --git a/internal/xray/session_test.go b/internal/xray/session_test.go index 52de5d1..dd85f69 100644 --- a/internal/xray/session_test.go +++ b/internal/xray/session_test.go @@ -224,13 +224,17 @@ func TestSessionPrepareHysteriaPinHasNoLink(t *testing.T) { } // TestSessionBrokenInboundStillRevokes covers the three ways Prepare keeps a reason instead of -// failing: a validation error, a credential of the wrong kind and a file that cannot be rendered. +// failing: an x-usher hint error, a credential of the wrong kind and a file that cannot be +// rendered. The running inbound is known good in each, so Apply removes the users who left. func TestSessionBrokenInboundStillRevokes(t *testing.T) { notPEM := `{"tag": "pinned", "protocol": "vless", "port": 9003, "x-usher": {"trust": "pin"}, "settings": {"clients": [], "decryption": "none"}, "streamSettings": {"network": "ws", "security": "tls", "tlsSettings": {"serverName": "p.example.com", "certificates": [{"certificate": ["x"], "key": ["y"]}]}}},` + badHint := `{"tag": "hinted", "protocol": "vless", "port": 9004, + "x-usher": {"trust": "sideways"}, + "settings": {"clients": [], "decryption": "none"}},` tests := []struct { name string extra string @@ -238,7 +242,7 @@ func TestSessionBrokenInboundStillRevokes(t *testing.T) { setup func(st *state.State) wantErr string }{ - {"validation error", badInbound, "bad", func(*state.State) {}, "settings.clients"}, + {"x-usher hint error", badHint, "hinted", func(*state.State) {}, "x-usher.trust"}, {"credential of the wrong kind", "", "ws", func(st *state.State) { st.Xray["ws/alice"] = state.XrayEntry{Password: "wrong-kind"} }, "does not fit"}, @@ -275,6 +279,72 @@ func TestSessionBrokenInboundStillRevokes(t *testing.T) { } } +// TestSessionStructuralErrorTouchesNothing: an inbound that fails structural validation returns +// its error from Prepare, so the pass skips Apply, and a direct Apply still removes no user. +func TestSessionStructuralErrorTouchesNothing(t *testing.T) { + badTag := `{"tag": "bad tag", "protocol": "vless", "port": 9005, + "settings": {"clients": [], "decryption": "none"}},` + dup := `{"tag": "ws", "protocol": "vless", "port": 9002, + "settings": {"clients": [{"id": "5783a3e7-e373-51cd-8642-c83782b807c5"}], "decryption": "none"}},` + tests := []struct { + name string + extra string + tag string + }{ + {"static clients", badInbound, "bad"}, + {"invalid tag", badTag, "bad tag"}, + {"duplicate tag", dup, "ws"}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + f := newSessionFixture(t, func(c *driftConfig) { c.extra = tt.extra }) + in := f.inbound(tt.tag) + f.service.seed(in, "someone-else", idEntry()) + f.service.alters = 0 + + files, changed, err := f.session.Prepare(newState(), tt.tag, []string{"alice"}) + if err == nil || files != nil || changed { + t.Fatalf("Prepare = %v, %v, %v, want the validation error", files, changed, err) + } + if err := f.session.Apply(t.Context(), newState(), tt.tag, []string{"alice"}); err == nil { + t.Error("Apply error = nil, want the validation error") + } + if f.service.alters != 0 { + t.Errorf("AlterInbound calls = %d, want 0", f.service.alters) + } + }) + } +} + +// TestSessionApplySkipsFileThatDoesNotBuild: the file inbound changed protocol and no longer +// builds, so nothing proves the running vless handler can take an account of the new type. +func TestSessionApplySkipsFileThatDoesNotBuild(t *testing.T) { + base := newDriftConfig(t.TempDir()) + running := runningFrom(t, base.write(t)) + broken := base + broken.realityProto = "trojan" + broken.shortID = "zz" + path := broken.write(t) + + service := newUserService(running) + dial := func() (command.HandlerServiceClient, func(), error) { return service, func() {}, nil } + session, err := Open(t.Context(), dial, filepath.Join(t.TempDir(), "xray.sock"), path, + "198.51.100.7", slog.New(slog.DiscardHandler)) + if err != nil { + t.Fatal(err) + } + st := newState() + if _, _, err := session.Prepare(st, "reality", []string{"alice"}); err != nil { + t.Fatal(err) + } + if err := session.Apply(t.Context(), st, "reality", []string{"alice"}); err == nil { + t.Error("Apply error = nil, want the drift reason") + } + if service.alters != 0 { + t.Errorf("AlterInbound calls = %d, want 0", service.alters) + } +} + func TestSessionPrepareKeepsChangedWhenRenderFails(t *testing.T) { notPEM := `{"tag": "pinned", "protocol": "vless", "port": 9003, "x-usher": {"trust": "pin"}, From 45aab5c18e1b20ccdaa8bf86a8a46f8cb9623b3f Mon Sep 17 00:00:00 2001 From: EvaTheSalmon <18466855+EvaTheSalmon@users.noreply.github.com> Date: Wed, 14 Oct 2026 19:51:12 +0300 Subject: [PATCH 24/32] fix(xray): drift-check inbounds with hint errors checkDrift skipped every inbound with an error, so an x-usher typo let Apply revoke users from a running inbound it never compared with the file. Only structural errors skip the comparison now. Co-Authored-By: Claude Sonnet 5.5 --- README.md | 3 ++- docs/architecture/overview.md | 8 +++--- internal/xray/config.go | 7 +++-- internal/xray/drift.go | 2 +- internal/xray/session.go | 2 +- internal/xray/session_test.go | 49 ++++++++++++++++++++++++++++++++++- 6 files changed, 59 insertions(+), 12 deletions(-) diff --git a/README.md b/README.md index 2547a2f..84a9a48 100644 --- a/README.md +++ b/README.md @@ -182,7 +182,8 @@ drift. When the running inbound is missing, runs another protocol or cannot be c file, usher changes nothing on it, because an account of the wrong type crashes Xray; restart Xray. An inbound with an `x-usher` error, or one whose credential or client file cannot be built, keeps -its old client files, and usher still removes the users who left `usher.yml` from it. It adds no +its old client files, and usher still removes the users who left `usher.yml` from it, once the +drift check has compared the inbound with the running Xray and found it usable. It adds no user and changes no credential until you fix the file. An inbound that fails any other check (port, `listen`, non-empty `clients`, transport, Reality keys, a tag that is not a valid interface name or that the file defines twice) is left alone: usher cannot trust the file entry, so it adds and diff --git a/docs/architecture/overview.md b/docs/architecture/overview.md index 93a2abb..74fbccb 100644 --- a/docs/architecture/overview.md +++ b/docs/architecture/overview.md @@ -124,10 +124,10 @@ pass and every call has a 10 s deadline. (port, listen, clients, decryption, transport, Hysteria auth, masks, xhttp, Reality keys, a decode failure, an invalid or duplicate tag) is returned by `Prepare`, so the pass skips `Apply` and touches no user of the tag; `Apply` also returns it if called. An `x-usher` hint error, like - an enroll or render error, is kept by `Prepare`, which returns none: the running inbound is known - good, so the pass still calls `Apply`, which only removes the users who left and returns the - error, which keeps the old files. `validate` runs the hint check last, so a hint error is the only - reason left. A tag defined twice is reported once, and its entry carries the error. + an enroll or render error, is kept by `Prepare`, which returns none, so the pass still calls + `Apply`. `Apply` removes the users who left only when the drift check found the running inbound + usable, then returns the error, which keeps the old files. `validate` runs the hint check last, + so a hint error is the only reason left and `checkDrift` still compares the inbound. A tag defined twice is reported once, and its entry carries the error. ### Reading the config diff --git a/internal/xray/config.go b/internal/xray/config.go index 1504b01..c9ba179 100644 --- a/internal/xray/config.go +++ b/internal/xray/config.go @@ -36,8 +36,8 @@ type Inbound struct { Hints Hints Raw map[string]any Err error - // hintsOnly is true when Err comes from the x-usher block alone: the rest of the inbound is - // sound, so the running inbound can be trusted to match it. + // hintsOnly is true when Err comes from the x-usher block alone: the rest of the inbound + // passed validation, so checkDrift still compares it with the running inbound. hintsOnly bool } @@ -209,8 +209,7 @@ func (in *Inbound) validate() error { return fail("inbound", "does not decode as an Xray inbound") } - // Last, so that a hint error is the only reason left: the inbound is sound and the running - // Xray can be trusted to match it. + // Last, so that a hint error is the only reason left and the drift check can still run. if field, reason := in.readHints(stream); field != "" { in.hintsOnly = true return fail(field, reason) diff --git a/internal/xray/drift.go b/internal/xray/drift.go index 362f5b1..c2ccb19 100644 --- a/internal/xray/drift.go +++ b/internal/xray/drift.go @@ -89,7 +89,7 @@ func checkDrift(ctx context.Context, client command.HandlerServiceClient, log *s inFile := make(map[string]bool, len(inbounds)) for _, in := range inbounds { inFile[in.Tag] = true - if in.Err != nil { + if in.Err != nil && !in.hintsOnly { continue } live, ok := running[in.Tag] diff --git a/internal/xray/session.go b/internal/xray/session.go index 61f1141..6afd494 100644 --- a/internal/xray/session.go +++ b/internal/xray/session.go @@ -99,7 +99,7 @@ func (s *Session) Interfaces() []string { // Prepare enrolls the users on the inbound in st and renders their client files. When the inbound // failed structural validation, Prepare returns the error and the pass skips Apply, so usher // touches no user of that tag. When only the x-usher block failed, an enrolled credential does not -// fit the inbound or a file cannot be rendered, the running inbound is known good: Prepare keeps +// fit the inbound or a file cannot be rendered, Prepare keeps // the reason and returns no error, and Apply then still revokes the users who left and returns the // reason, which keeps the old client files. func (s *Session) Prepare(st *state.State, name string, users []string) (map[string][]byte, bool, error) { diff --git a/internal/xray/session_test.go b/internal/xray/session_test.go index dd85f69..311ac1c 100644 --- a/internal/xray/session_test.go +++ b/internal/xray/session_test.go @@ -225,7 +225,7 @@ func TestSessionPrepareHysteriaPinHasNoLink(t *testing.T) { // TestSessionBrokenInboundStillRevokes covers the three ways Prepare keeps a reason instead of // failing: an x-usher hint error, a credential of the wrong kind and a file that cannot be -// rendered. The running inbound is known good in each, so Apply removes the users who left. +// rendered. The drift check finds the running inbound usable in each, so Apply removes the users who left. func TestSessionBrokenInboundStillRevokes(t *testing.T) { notPEM := `{"tag": "pinned", "protocol": "vless", "port": 9003, "x-usher": {"trust": "pin"}, @@ -345,6 +345,53 @@ func TestSessionApplySkipsFileThatDoesNotBuild(t *testing.T) { } } +// TestSessionHintErrorStillGuardedByDrift: an x-usher hint error must not skip the drift check, so +// Apply revokes nothing from a running inbound it could not compare with the file. +func TestSessionHintErrorStillGuardedByDrift(t *testing.T) { + tests := []struct { + name string + change func(c *driftConfig) + wantErr string + }{ + {"proxy type differs", func(c *driftConfig) { c.realityProto = "trojan" }, "proxy type"}, + {"file does not build", func(c *driftConfig) { c.shortID = "zz" }, "comparing with"}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + base := newDriftConfig(t.TempDir()) + running := runningFrom(t, base.write(t)) + file := base + tt.change(&file) + file.realityExtra = `"x-usher": {"adress": "x"},` + path := file.write(t) + + service := newUserService(running) + dial := func() (command.HandlerServiceClient, func(), error) { return service, func() {}, nil } + session, err := Open(t.Context(), dial, filepath.Join(t.TempDir(), "xray.sock"), path, + "198.51.100.7", slog.New(slog.DiscardHandler)) + if err != nil { + t.Fatal(err) + } + if session.byTag["reality"].Err == nil { + t.Fatal("reality inbound has no hint error, the fixture is wrong") + } + service.users["reality"] = []*protocol.User{{Email: "operator-own"}} + + st := newState() + if _, _, err := session.Prepare(st, "reality", []string{"alice"}); err != nil { + t.Fatal(err) + } + err = session.Apply(t.Context(), st, "reality", []string{"alice"}) + if err == nil || !strings.Contains(err.Error(), tt.wantErr) { + t.Errorf("Apply error = %v, want one with %q", err, tt.wantErr) + } + if service.alters != 0 { + t.Errorf("AlterInbound calls = %d, want 0", service.alters) + } + }) + } +} + func TestSessionPrepareKeepsChangedWhenRenderFails(t *testing.T) { notPEM := `{"tag": "pinned", "protocol": "vless", "port": 9003, "x-usher": {"trust": "pin"}, From c4e00258ebfd48aef440ecbf37015999edbacebf Mon Sep 17 00:00:00 2001 From: EvaTheSalmon <18466855+EvaTheSalmon@users.noreply.github.com> Date: Wed, 14 Oct 2026 20:10:01 +0300 Subject: [PATCH 25/32] docs: rewrap lines past 100 columns Co-Authored-By: Claude Sonnet 5.5 --- AGENTS.md | 4 +- docs/architecture/overview.md | 71 ++++++++++++++++++----------------- 2 files changed, 39 insertions(+), 36 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 9ece50e..d3f1ba7 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -2,8 +2,8 @@ Container that reads `config/usher.yml`, a map of user name to interface names, and keeps the AmneziaWG peers, the Xray users and the client files in `clients/` in step with it through the -`awg-grpc` and Xray API unix sockets; `README.md` describes it. Go module `github.com/mrcsin/usher`; entry point -`cmd/usher`. All commands run from the repository root. +`awg-grpc` and Xray API unix sockets; `README.md` describes it. Go module `github.com/mrcsin/usher`; +entry point `cmd/usher`. All commands run from the repository root. ## Build diff --git a/docs/architecture/overview.md b/docs/architecture/overview.md index 74fbccb..9c3c01f 100644 --- a/docs/architecture/overview.md +++ b/docs/architecture/overview.md @@ -50,13 +50,13 @@ in both. The steps below run per backend; the text names the awg calls. 7. Make `clients/` equal to the rendered files of applied interfaces plus the current files of failed ones. -Two triggers start a pass: a change of `usher.yml` and a 30 s refill ticker. One goroutine runs -the watcher loop and the passes, with the poll ticker, the settle timer and the refill ticker as -cases of one `select`. A tick that fires during a pass waits in its ticker, which buffers one tick, -so each source yields at most one following pass. A refill tick and a file change during one pass -give two following passes. File polling follows `reproxy` (`app/discovery/provider/file.go`): `os.Stat` every 3 s, and a new -modification time counts once it has stayed unchanged for 500 ms. The connection is opened per -pass and every call has a 10 s deadline. +Two triggers start a pass: a change of `usher.yml` and a 30 s refill ticker. One goroutine runs the +watcher loop and the passes, with the poll ticker, the settle timer and the refill ticker as cases +of one `select`. A tick that fires during a pass waits in its ticker, which buffers one tick, so +each source yields at most one following pass. A refill tick and a file change during one pass give +two following passes. File polling follows `reproxy` (`app/discovery/provider/file.go`): `os.Stat` +every 3 s, and a new modification time counts once it has stayed unchanged for 500 ms. The +connection is opened per pass and every call has a 10 s deadline. ## Keys and addresses @@ -116,30 +116,32 @@ pass and every call has a 10 s deadline. - The operator owns one Xray server config: inbounds, outbounds, routing, `api`, certificates, Reality keys, and a per-inbound `x-usher` block with client hints. Xray reads it at start. usher reads the same file read-only at every pass. -- usher owns the users inside the inbounds, through the API, and everything under `clients/`. - Xray keeps users in memory, so the refill tick restores them after a restart. +- usher owns the users inside the inbounds, through the API, and everything under `clients/`. Xray + keeps users in memory, so the refill tick restores them after a restart. - A managed inbound has a tag that matches the interface name pattern (32 characters), one integer - port, no unix `listen`, empty `clients` and a protocol of vless, hysteria, vmess or trojan. - Other inbounds are skipped. An inbound that fails validation carries its error. A structural error - (port, listen, clients, decryption, transport, Hysteria auth, masks, xhttp, Reality keys, a - decode failure, an invalid or duplicate tag) is returned by `Prepare`, so the pass skips `Apply` - and touches no user of the tag; `Apply` also returns it if called. An `x-usher` hint error, like - an enroll or render error, is kept by `Prepare`, which returns none, so the pass still calls - `Apply`. `Apply` removes the users who left only when the drift check found the running inbound - usable, then returns the error, which keeps the old files. `validate` runs the hint check last, - so a hint error is the only reason left and `checkDrift` still compares the inbound. A tag defined twice is reported once, and its entry carries the error. + port, no unix `listen`, empty `clients` and a protocol of vless, hysteria, vmess or trojan. Other + inbounds are skipped. An inbound that fails validation carries its error. A structural error + (port, listen, clients, decryption, transport, Hysteria auth, masks, xhttp, Reality keys, a decode + failure, an invalid or duplicate tag) is returned by `Prepare`, so the pass skips `Apply` and + touches no user of the tag; `Apply` also returns it if called. An `x-usher` hint error, like an + enroll or render error, is kept by `Prepare`, which returns none, so the pass still calls `Apply`. + `Apply` removes the users who left only when the drift check found the running inbound usable, + then returns the error, which keeps the old files. `validate` runs the hint check last, so a hint + error is the only reason left and `checkDrift` still compares the inbound. A tag defined twice is + reported once, and its entry carries the error. ### Reading the config -`ReadConfig` reads the file with Xray's comment-aware reader into `infra/conf` types plus raw -maps. It normalizes the aliases (`method`, `splithttp`, `websocket`, `tcp`, `splithttpSettings`, +`readConfig` reads the file with Xray's comment-aware reader into `infra/conf` types plus raw maps. +It normalizes the aliases (`method`, `splithttp`, `websocket`, `tcp`, `splithttpSettings`, `tcpSettings`, Hysteria `users`) before validation and mirroring. Every error names the field and never a value. `x-usher` decodes with unknown keys rejected. Reality matches names by exact lookup, -so there an explicit `x-usher.sni` must be one of the `serverNames` entries, and the config needs -an entry without a wildcard when `sni` is unset. A Reality `privateKey` and `mldsa65Seed` must -decode, and the seed must differ from the key. `hop_ports` requires from < to. An inbound without -`streamSettings` stays without them in `Raw`, so it builds as Xray builds the file. ReadConfig registers a discard handler -for Xray's global logger once, because `Build()` prints deprecation warnings. +so there an explicit `x-usher.sni` must be one of the `serverNames` entries, and the config needs an +entry without a wildcard when `sni` is unset. A Reality `privateKey` and `mldsa65Seed` must decode, +and the seed must differ from the key. `hop_ports` requires from < to. An inbound without +`streamSettings` stays without them in `Raw`, so it builds as Xray builds the file. readConfig +registers a discard handler for Xray's global logger once, because `Build()` prints deprecation +warnings. ### Users @@ -160,20 +162,21 @@ for Xray's global logger once, because `Build()` prints deprecation warnings. `Open` calls `ListInbounds` once and compares each valid inbound with `Build()` of its file entry: proxy type, proxy settings, port, stream settings and receiver settings. Certificate entries that name a file are dropped on both sides, because Xray reloads those files itself. A tag missing at -runtime counts as drift. The proxy type of the running handler is compared with the file's -protocol before any `Build()`; a mismatch, or a file entry that does not decode or build, marks the -drift unusable. The reason names the differing parts; an error of `Build()` becomes a fixed -text, because Xray quotes the keys and short ids it rejects. `Apply` reconciles users first and then -returns the drift error, so revocation works while the old client files stay. The exception is an -unusable drift: a missing inbound, another proxy type or a failed comparison (see Users). A running managed inbound that the file does not define is logged each pass. +runtime counts as drift. The proxy type of the running handler is compared with the file's protocol +before any `Build()`; a mismatch, or a file entry that does not decode or build, marks the drift +unusable. The reason names the differing parts; an error of `Build()` becomes a fixed text, because +Xray quotes the keys and short ids it rejects. `Apply` reconciles users first and then returns the +drift error, so revocation works while the old client files stay. The exception is an unusable +drift: a missing inbound, another proxy type or a failed comparison (see Users). A running managed +inbound that the file does not define is logged each pass. ### Stale socket A killed Xray leaves its API socket, and a new start fails on it. Xray holds `flock(LOCK_EX)` on `.lock` while it listens. Before dialing, usher opens the lock read-only and tries -`LOCK_EX|LOCK_NB`. When it gets the lock, no Xray listens: usher unlinks the socket and reports -Xray down for this pass. The restart policy starts Xray at its next attempt. The socket directory -is a bind mount that usher can write. A failed Xray start removes the lock file, so for a socket without +`LOCK_EX|LOCK_NB`. When it gets the lock, no Xray listens: usher unlinks the socket and reports Xray +down for this pass. The restart policy starts Xray at its next attempt. The socket directory is a +bind mount that usher can write. A failed Xray start removes the lock file, so for a socket without one usher probes the socket instead: only a refused connection counts as stale, and a live Xray, or a socket that a new Xray replaced meanwhile, stays. A path that is not a socket is an error. From 11726d1db8305caffcf48f576d63c4ebc92327c2 Mon Sep 17 00:00:00 2001 From: EvaTheSalmon <18466855+EvaTheSalmon@users.noreply.github.com> Date: Wed, 14 Oct 2026 20:10:01 +0300 Subject: [PATCH 26/32] refactor(pass): split Run into three steps Co-Authored-By: Claude Sonnet 5.5 --- internal/pass/pass.go | 46 ++++++++++++++++++++++++++++++++----------- 1 file changed, 34 insertions(+), 12 deletions(-) diff --git a/internal/pass/pass.go b/internal/pass/pass.go index 0f33a67..7488154 100644 --- a/internal/pass/pass.go +++ b/internal/pass/pass.go @@ -61,6 +61,30 @@ func (p *Pass) Run(ctx context.Context) { byInterface := usersByInterface(cfg) owners := interfaceOwners(open) failed := make(map[interfaceKey]bool) + rendered, changed := p.prepareAll(st, open, byInterface, owners, failed) + + if changed { + if err := state.Save(p.settings.StatePath, st); err != nil { + p.log.Error("saving state", "error", err) + return + } + } + + p.applyAll(ctx, st, open, byInterface, failed) + + // A backend that is down hides its interface names, so every unreferenced name could be one. + if len(down) == 0 { + p.logUnknown(cfg, owners) + } + + p.mirrorClients(open, down, rendered, failed) +} + +// prepareAll prepares every interface that exactly one backend reports. It marks a failed +// interface in failed, and returns the rendered client files by interface and whether any +// backend changed the state. +func (p *Pass) prepareAll(st *state.State, open []openBackend, byInterface map[string][]string, + owners map[string][]string, failed map[interfaceKey]bool) (map[interfaceKey]map[string][]byte, bool) { rendered := make(map[interfaceKey]map[string][]byte) changed := false for i, b := range open { @@ -82,14 +106,12 @@ func (p *Pass) Run(ctx context.Context) { changed = changed || entriesChanged } } + return rendered, changed +} - if changed { - if err := state.Save(p.settings.StatePath, st); err != nil { - p.log.Error("saving state", "error", err) - return - } - } - +// applyAll applies every interface that Prepare did not fail and marks the ones that fail now. +func (p *Pass) applyAll(ctx context.Context, st *state.State, open []openBackend, + byInterface map[string][]string, failed map[interfaceKey]bool) { for i, b := range open { for _, name := range b.session.Interfaces() { key := interfaceKey{i, name} @@ -102,12 +124,12 @@ func (p *Pass) Run(ctx context.Context) { } } } +} - // A backend that is down hides its interface names, so every unreferenced name could be one. - if len(down) == 0 { - p.logUnknown(cfg, owners) - } - +// mirrorClients writes the client files of the interfaces that did not fail and keeps the current +// files of the failed interfaces and of the backends that are down. +func (p *Pass) mirrorClients(open []openBackend, down []Backend, + rendered map[interfaceKey]map[string][]byte, failed map[interfaceKey]bool) { desired := make(map[string][]byte) for key, files := range rendered { if !failed[key] { From 6acacc26a2dec13b0a7eaa63d400d20de2f21c50 Mon Sep 17 00:00:00 2001 From: EvaTheSalmon <18466855+EvaTheSalmon@users.noreply.github.com> Date: Wed, 14 Oct 2026 20:10:01 +0300 Subject: [PATCH 27/32] refactor(usher): share the unix socket dial Co-Authored-By: Claude Sonnet 5.5 --- cmd/usher/main.go | 25 ++++++++----------------- 1 file changed, 8 insertions(+), 17 deletions(-) diff --git a/cmd/usher/main.go b/cmd/usher/main.go index 8ed9570..a4c948e 100644 --- a/cmd/usher/main.go +++ b/cmd/usher/main.go @@ -63,7 +63,7 @@ func serve(ctx context.Context, settings pass.Settings, backends []pass.Backend, } func awgBackend(env environment, log *slog.Logger) pass.Backend { - dial := dialAWGSocket(env.AWGSocket) + dial := dialUnix(env.AWGSocket, awgv1.NewManagementServiceClient) return pass.Backend{ Name: "awg-grpc", Suffixes: []string{awg.Suffix}, @@ -74,7 +74,7 @@ func awgBackend(env environment, log *slog.Logger) pass.Backend { } func xrayBackend(env environment, log *slog.Logger) pass.Backend { - dial := dialXraySocket(env.XraySocket) + dial := dialUnix(env.XraySocket, command.NewHandlerServiceClient) return pass.Backend{ Name: "xray", Suffixes: []string{xray.JSONSuffix, xray.LinkSuffix}, @@ -93,24 +93,15 @@ func asSession[S pass.Session](session S, err error) (pass.Session, error) { return session, nil } -func dialAWGSocket(path string) awg.Dial { +// dialUnix returns a dial function that opens a gRPC client over the unix socket at path. +func dialUnix[C any](path string, newClient func(grpc.ClientConnInterface) C) func() (C, func(), error) { target := "unix://" + path - return func() (awgv1.ManagementServiceClient, func(), error) { + return func() (C, func(), error) { conn, err := grpc.NewClient(target, grpc.WithTransportCredentials(insecure.NewCredentials())) if err != nil { - return nil, nil, fmt.Errorf("opening %s: %w", target, err) + var zero C + return zero, nil, fmt.Errorf("opening %s: %w", target, err) } - return awgv1.NewManagementServiceClient(conn), func() { _ = conn.Close() }, nil - } -} - -func dialXraySocket(path string) xray.Dial { - target := "unix://" + path - return func() (command.HandlerServiceClient, func(), error) { - conn, err := grpc.NewClient(target, grpc.WithTransportCredentials(insecure.NewCredentials())) - if err != nil { - return nil, nil, fmt.Errorf("opening %s: %w", target, err) - } - return command.NewHandlerServiceClient(conn), func() { _ = conn.Close() }, nil + return newClient(conn), func() { _ = conn.Close() }, nil } } From 9f77178f73d6e83f108356d6a55132155fc6cabf Mon Sep 17 00:00:00 2001 From: EvaTheSalmon <18466855+EvaTheSalmon@users.noreply.github.com> Date: Wed, 14 Oct 2026 20:10:01 +0300 Subject: [PATCH 28/32] style(state): separate import groups Co-Authored-By: Claude Sonnet 5.5 --- internal/state/state.go | 1 + 1 file changed, 1 insertion(+) diff --git a/internal/state/state.go b/internal/state/state.go index b8e2aa1..999126c 100644 --- a/internal/state/state.go +++ b/internal/state/state.go @@ -14,6 +14,7 @@ import ( "strings" "github.com/google/uuid" + "github.com/mrcsin/usher/internal/atomicfile" ) From 8e9dc4987042c8d8695bff191496bc7a2e851095 Mon Sep 17 00:00:00 2001 From: EvaTheSalmon <18466855+EvaTheSalmon@users.noreply.github.com> Date: Wed, 14 Oct 2026 20:10:01 +0300 Subject: [PATCH 29/32] refactor(xray): unexport reader, share decode, tidy Co-Authored-By: Claude Sonnet 5.5 --- cmd/usher/main_test.go | 45 ------- internal/xray/config.go | 116 ++++++++++-------- internal/xray/config_test.go | 105 ++++++++++++----- internal/xray/drift.go | 50 +++----- internal/xray/drift_test.go | 43 ++----- internal/xray/enroll.go | 2 +- internal/xray/enroll_test.go | 16 +-- internal/xray/mirror.go | 29 ++--- internal/xray/mirror_test.go | 12 +- internal/xray/protocol.go | 36 ++++-- internal/xray/protocol_test.go | 42 +++++++ internal/xray/render.go | 4 +- internal/xray/render_test.go | 6 +- internal/xray/session.go | 25 ++-- internal/xray/session_test.go | 210 +++++++++++---------------------- internal/xray/socket.go | 5 +- test/e2e/xray/xray_test.go | 4 +- 17 files changed, 353 insertions(+), 397 deletions(-) create mode 100644 internal/xray/protocol_test.go diff --git a/cmd/usher/main_test.go b/cmd/usher/main_test.go index 5e45b21..13644d6 100644 --- a/cmd/usher/main_test.go +++ b/cmd/usher/main_test.go @@ -3,10 +3,8 @@ package main import ( "bytes" "context" - "encoding/base64" "errors" "log/slog" - "os" "path/filepath" "slices" "strings" @@ -14,7 +12,6 @@ import ( "time" "github.com/mrcsin/usher/internal/pass" - "github.com/mrcsin/usher/internal/xray" ) func TestRun(t *testing.T) { @@ -116,45 +113,3 @@ func TestServe(t *testing.T) { } } } - -func TestDeployExampleConfig(t *testing.T) { - path := filepath.Join("..", "..", "deploy", "xray", "config.json") - wantTags := []string{"vless-reality", "hysteria2"} - - // As shipped, the placeholders make the Reality inbound fail on its key. - inbounds, err := xray.ReadConfig(path) - if err != nil { - t.Fatal(err) - } - var tags []string - for _, in := range inbounds { - tags = append(tags, in.Tag) - } - if !slices.Equal(tags, wantTags) { - t.Errorf("tags = %v, want %v", tags, wantTags) - } - if inbounds[0].Err == nil || !strings.Contains(inbounds[0].Err.Error(), "privateKey") { - t.Errorf("shipped Reality inbound error = %v, want one about privateKey", inbounds[0].Err) - } - - // With the placeholder key replaced, every inbound is valid. - data, err := os.ReadFile(path) - if err != nil { - t.Fatal(err) - } - key := base64.RawURLEncoding.EncodeToString(bytes.Repeat([]byte{1}, 32)) - replaced := strings.ReplaceAll(string(data), "REPLACE_WITH_X25519_PRIVATE_KEY", key) - filled := filepath.Join(t.TempDir(), "config.json") - if err := os.WriteFile(filled, []byte(replaced), 0o600); err != nil { - t.Fatal(err) - } - inbounds, err = xray.ReadConfig(filled) - if err != nil { - t.Fatal(err) - } - for _, in := range inbounds { - if in.Err != nil { - t.Errorf("inbound %s: %v", in.Tag, in.Err) - } - } -} diff --git a/internal/xray/config.go b/internal/xray/config.go index c9ba179..4995c08 100644 --- a/internal/xray/config.go +++ b/internal/xray/config.go @@ -21,10 +21,11 @@ import ( "github.com/mrcsin/usher/internal/config" ) -// Inbound is one managed inbound of the Xray server config. Raw is the inbound object with the +// inbound is one managed inbound of the Xray server config. Raw is the inbound object with the // config aliases normalized to their canonical names. A non-nil Err means the inbound failed -// validation and must not be used. -type Inbound struct { +// validation. Unless hintsOnly is set, it must not be used; with hintsOnly the drift check and +// the revoke of foreign users still use it. +type inbound struct { Tag string Protocol string // lower case: vless, hysteria, vmess or trojan Port int @@ -33,7 +34,7 @@ type Inbound struct { // ServerName is the name the client sends: x-usher.sni, else the first usable Reality // serverNames entry or the tlsSettings serverName. It is empty when security is none. ServerName string - Hints Hints + Hints hints Raw map[string]any Err error // hintsOnly is true when Err comes from the x-usher block alone: the rest of the inbound @@ -41,9 +42,9 @@ type Inbound struct { hintsOnly bool } -// Hints is the x-usher block of an inbound with the defaults that do not depend on usher's +// hints is the x-usher block of an inbound with the defaults that do not depend on usher's // environment applied. An empty Address means USHER_HOST. -type Hints struct { +type hints struct { Address string Port int Fingerprint string @@ -53,12 +54,21 @@ type Hints struct { } const ( - // TrustCA makes the client verify the server certificate against its CA roots. - TrustCA = "ca" - // TrustPin makes the client also pin the leaf certificate. - TrustPin = "pin" + // trustCA makes the client verify the server certificate against its CA roots. + trustCA = "ca" + // trustPin makes the client also pin the leaf certificate. + trustPin = "pin" defaultFingerprint = "chrome" + + protocolVLESS = "vless" + protocolVMess = "vmess" + protocolTrojan = "trojan" + protocolHysteria = "hysteria" + + securityNone = "none" + securityTLS = "tls" + securityReality = "reality" ) var ( @@ -86,11 +96,11 @@ func silenceXrayLog() { silenceOnce.Do(func() { log.RegisterHandler(discardLog{}) }) } -// ReadConfig reads the Xray server config at path and returns its managed inbounds in file order. +// readConfig reads the Xray server config at path and returns its managed inbounds in file order. // Inbounds of other protocols and inbounds without a tag are skipped. An inbound that fails // validation is returned with Err set, so the caller can keep its files. An error means the file // as a whole is unusable. -func ReadConfig(path string) ([]Inbound, error) { +func readConfig(path string) ([]inbound, error) { silenceXrayLog() f, err := os.Open(path) if err != nil { @@ -105,7 +115,7 @@ func ReadConfig(path string) ([]Inbound, error) { return nil, fmt.Errorf("parsing xray config %s: %w", path, err) } - var inbounds []Inbound + var inbounds []inbound seen := map[string]bool{} for i, data := range file.Inbounds { raw, err := decodeObject(data) @@ -118,7 +128,7 @@ func ReadConfig(path string) ([]Inbound, error) { if _, managed := protocols[protocol]; tag == "" || !managed { continue } - in := Inbound{Tag: tag, Protocol: protocol, Raw: raw} + in := inbound{Tag: tag, Protocol: protocol, Raw: raw} switch { case seen[tag]: in.Err = fmt.Errorf("inbound %q: tag: defined twice", tag) @@ -133,6 +143,17 @@ func ReadConfig(path string) ([]Inbound, error) { return inbounds, nil } +// decodeInbound decodes an inbound object as Xray's config reader does. +func decodeInbound(raw map[string]any) (conf.InboundDetourConfig, error) { + var detour conf.InboundDetourConfig + encoded, err := json.Marshal(raw) + if err != nil { + return detour, fmt.Errorf("encoding the inbound: %w", err) + } + err = json.Unmarshal(encoded, &detour) + return detour, err +} + func decodeObject(data []byte) (map[string]any, error) { dec := json.NewDecoder(bytes.NewReader(data)) dec.UseNumber() @@ -148,7 +169,7 @@ func decodeObject(data []byte) (map[string]any, error) { // validate normalizes in.Raw, checks it against what the backend supports and fills the derived // fields. Every error names the field and never quotes a value. -func (in *Inbound) validate() error { +func (in *inbound) validate() error { fail := func(field, reason string) error { return fmt.Errorf("inbound %q: %s: %s", in.Tag, field, reason) } @@ -169,7 +190,7 @@ func (in *Inbound) validate() error { if err := in.checkUsersEmpty(settings); err != nil { return fail("settings.clients", err.Error()) } - if in.Protocol == "vless" && lowerString(settings["decryption"]) != "none" { + if in.Protocol == protocolVLESS && lowerString(settings["decryption"]) != "none" { return fail("settings.decryption", `want "none"`) } @@ -189,19 +210,14 @@ func (in *Inbound) validate() error { if field, reason := in.checkXHTTP(stream); field != "" { return fail(field, reason) } - if in.Security == "reality" { + if in.Security == securityReality { if _, _, field, reason := realityKeys(object(stream["realitySettings"])); field != "" { return fail("streamSettings.realitySettings."+field, reason) } } // The decode catches field type errors that the checks above do not look at. Xray's decoders // can quote the value they reject, so the error names the Go field of a type mismatch at most. - encoded, err := json.Marshal(in.Raw) - if err != nil { - return fail("inbound", "cannot be encoded") - } - var detour conf.InboundDetourConfig - if err := json.Unmarshal(encoded, &detour); err != nil { + if _, err := decodeInbound(in.Raw); err != nil { var typeErr *json.UnmarshalTypeError if errors.As(err, &typeErr) { return fail("inbound", "does not decode as an Xray inbound: wrong type at "+typeErr.Field) @@ -217,7 +233,7 @@ func (in *Inbound) validate() error { return nil } -func (in *Inbound) singlePort() (int, error) { +func (in *inbound) singlePort() (int, error) { number, ok := in.Raw["port"].(json.Number) if !ok { return 0, errors.New("want a single integer") @@ -229,7 +245,7 @@ func (in *Inbound) singlePort() (int, error) { return port, nil } -func (in *Inbound) settings() (map[string]any, error) { +func (in *inbound) settings() (map[string]any, error) { v, present := in.Raw["settings"] if !present { settings := map[string]any{} @@ -245,8 +261,8 @@ func (in *Inbound) settings() (map[string]any, error) { // checkUsersEmpty folds the Hysteria alias users into clients and requires the list to be empty, // because usher owns every user of the inbound. -func (in *Inbound) checkUsersEmpty(settings map[string]any) error { - if in.Protocol == "hysteria" { +func (in *inbound) checkUsersEmpty(settings map[string]any) error { + if in.Protocol == protocolHysteria { if users, ok := settings["users"]; ok { if clients, both := settings["clients"]; both && !isEmptyList(clients) { return errors.New("users and clients are both set") @@ -269,7 +285,7 @@ func isEmptyList(v any) bool { // normalizeStream rewrites the aliases of streamSettings to their canonical names and fills in // Network and Security. It returns the streamSettings object, a detached empty one when the inbound // has none. -func (in *Inbound) normalizeStream() (map[string]any, error) { +func (in *inbound) normalizeStream() (map[string]any, error) { v, present := in.Raw["streamSettings"] stream, ok := v.(map[string]any) if present && !ok { @@ -315,7 +331,7 @@ func (in *Inbound) normalizeStream() (map[string]any, error) { delete(stream, alias) } - security := "none" + security := securityNone if v, ok := stream["security"]; ok { s, isString := v.(string) if !isString { @@ -330,16 +346,16 @@ func (in *Inbound) normalizeStream() (map[string]any, error) { return stream, nil } -func (in *Inbound) checkTransport(stream map[string]any) (field, reason string) { +func (in *inbound) checkTransport(stream map[string]any) (field, reason string) { if !networks[in.Network] { return "streamSettings.network", "unsupported transport" } switch in.Security { - case "none", "tls", "reality": + case securityNone, securityTLS, securityReality: default: return "streamSettings.security", "want none, tls or reality" } - if (in.Protocol == "hysteria") != (in.Network == "hysteria") { + if (in.Protocol == protocolHysteria) != (in.Network == "hysteria") { return "streamSettings.network", "hysteria transport and hysteria protocol go together" } if in.Network == "raw" { @@ -351,11 +367,11 @@ func (in *Inbound) checkTransport(stream map[string]any) (field, reason string) return "", "" } -func (in *Inbound) checkHysteria(stream map[string]any) (field, reason string) { - if in.Protocol != "hysteria" { +func (in *inbound) checkHysteria(stream map[string]any) (field, reason string) { + if in.Protocol != protocolHysteria { return "", "" } - if in.Security != "tls" { + if in.Security != securityTLS { return "streamSettings.security", `hysteria needs "tls"` } if !isVersion2(object(in.Raw["settings"])["version"]) { @@ -377,7 +393,7 @@ func isVersion2(v any) bool { } // checkMasks allows only salamander without packetSize in the udp list, and quicParams. -func (in *Inbound) checkMasks(stream map[string]any) (field, reason string) { +func (in *inbound) checkMasks(stream map[string]any) (field, reason string) { mask, present := stream["finalmask"] if !present { return "", "" @@ -402,13 +418,13 @@ func (in *Inbound) checkMasks(stream map[string]any) (field, reason string) { return "streamSettings.finalmask.udp.settings.packetSize", "unsupported with salamander" } } - if in.Protocol != "hysteria" && len(udp) > 0 { + if in.Protocol != protocolHysteria && len(udp) > 0 { return "streamSettings.finalmask.udp", "salamander is supported on hysteria only" } return "", "" } -func (in *Inbound) checkXHTTP(stream map[string]any) (field, reason string) { +func (in *inbound) checkXHTTP(stream map[string]any) (field, reason string) { if in.Network != "xhttp" { return "", "" } @@ -438,7 +454,7 @@ type xUsher struct { } // readHints decodes x-usher, checks it against the inbound and fills Hints and ServerName. -func (in *Inbound) readHints(stream map[string]any) (field, reason string) { +func (in *inbound) readHints(stream map[string]any) (field, reason string) { var hint xUsher if block, ok := in.Raw["x-usher"]; ok { encoded, err := json.Marshal(block) @@ -463,23 +479,23 @@ func (in *Inbound) readHints(stream map[string]any) (field, reason string) { trust := hint.Trust switch trust { case "": - trust = TrustCA - case TrustCA, TrustPin: + trust = trustCA + case trustCA, trustPin: default: return "x-usher.trust", `want "ca" or "pin"` } - if trust == TrustPin && in.Security != "tls" { + if trust == trustPin && in.Security != securityTLS { return "x-usher.trust", `"pin" needs security "tls"` } if hint.PinSHA256 != "" { - if trust != TrustPin { + if trust != trustPin { return "x-usher.pin_sha256", `valid only with trust "pin"` } if !sha256Pattern.MatchString(hint.PinSHA256) { return "x-usher.pin_sha256", "want 64 hex characters" } } - if trust == TrustPin && hint.PinSHA256 == "" && !hasInlineCertificate(stream) { + if trust == trustPin && hint.PinSHA256 == "" && !hasInlineCertificate(stream) { return "x-usher.pin_sha256", `trust "pin" needs it or an inline certificate` } @@ -487,12 +503,12 @@ func (in *Inbound) readHints(stream map[string]any) (field, reason string) { if fingerprint == "" { fingerprint = defaultFingerprint } - if in.Security == "reality" && !slices.Contains(realityFingerprints, fingerprint) { + if in.Security == securityReality && !slices.Contains(realityFingerprints, fingerprint) { return "x-usher.fingerprint", "reality supports chrome, firefox and safari only" } if hint.HopPorts != "" { - if in.Protocol != "hysteria" { + if in.Protocol != protocolHysteria { return "x-usher.hop_ports", "valid on hysteria only" } if !validHopPorts(hint.HopPorts) { @@ -505,7 +521,7 @@ func (in *Inbound) readHints(stream map[string]any) (field, reason string) { return field, reason } in.ServerName = serverName - in.Hints = Hints{ + in.Hints = hints{ Address: hint.Address, Port: port, Fingerprint: fingerprint, @@ -519,9 +535,9 @@ func (in *Inbound) readHints(stream map[string]any) (field, reason string) { // serverName resolves the name the client sends. An explicit x-usher.sni wins over the server // config on TLS. Reality looks the name up in serverNames by exact match, so there the override // must be one of the entries; it is how an operator picks one of several names. -func (in *Inbound) serverName(stream map[string]any, override string) (name, field, reason string) { +func (in *inbound) serverName(stream map[string]any, override string) (name, field, reason string) { switch in.Security { - case "tls": + case securityTLS: name = override if name == "" { name, _ = object(stream["tlsSettings"])["serverName"].(string) @@ -529,7 +545,7 @@ func (in *Inbound) serverName(stream map[string]any, override string) (name, fie if name == "" { return "", "x-usher.sni", "tls needs it or tlsSettings.serverName" } - case "reality": + case securityReality: names, _ := object(stream["realitySettings"])["serverNames"].([]any) if override != "" { if !slices.Contains(names, any(override)) || strings.Contains(override, "*") { diff --git a/internal/xray/config_test.go b/internal/xray/config_test.go index aecec67..6f381b5 100644 --- a/internal/xray/config_test.go +++ b/internal/xray/config_test.go @@ -1,41 +1,44 @@ package xray import ( + "bytes" + "encoding/base64" "encoding/json" "fmt" "os" "path/filepath" + "slices" "strings" "testing" ) func TestReadConfigShapes(t *testing.T) { - inbounds, err := ReadConfig("testdata/server.json") + inbounds, err := readConfig("testdata/server.json") if err != nil { t.Fatal(err) } - want := []Inbound{ + want := []inbound{ {Tag: "vless-reality", Protocol: "vless", Port: 443, Network: "raw", Security: "reality", ServerName: "www.example.com", - Hints: Hints{Address: "203.0.113.10", Port: 443, Fingerprint: "chrome", Trust: "ca"}}, + Hints: hints{Address: "203.0.113.10", Port: 443, Fingerprint: "chrome", Trust: "ca"}}, {Tag: "xh", Protocol: "vless", Port: 2053, Network: "xhttp", Security: "tls", - ServerName: "xh.example.com", Hints: Hints{Port: 2053, Fingerprint: "chrome", Trust: "ca"}}, + ServerName: "xh.example.com", Hints: hints{Port: 2053, Fingerprint: "chrome", Trust: "ca"}}, {Tag: "xh-extra", Protocol: "vless", Port: 2054, Network: "xhttp", Security: "tls", - ServerName: "xh.example.com", Hints: Hints{Port: 2054, Fingerprint: "chrome", Trust: "ca"}}, + ServerName: "xh.example.com", Hints: hints{Port: 2054, Fingerprint: "chrome", Trust: "ca"}}, {Tag: "ws", Protocol: "vless", Port: 2083, Network: "ws", Security: "tls", - ServerName: "ws.example.com", Hints: Hints{Port: 2083, Fingerprint: "chrome", Trust: "ca"}}, + ServerName: "ws.example.com", Hints: hints{Port: 2083, Fingerprint: "chrome", Trust: "ca"}}, {Tag: "grpc", Protocol: "vless", Port: 2087, Network: "grpc", Security: "tls", - ServerName: "g.example.com", Hints: Hints{Port: 2087, Fingerprint: "chrome", Trust: "ca"}}, + ServerName: "g.example.com", Hints: hints{Port: 2087, Fingerprint: "chrome", Trust: "ca"}}, {Tag: "hu", Protocol: "vless", Port: 2088, Network: "httpupgrade", Security: "tls", - ServerName: "hu.example.com", Hints: Hints{Port: 2088, Fingerprint: "chrome", Trust: "ca"}}, + ServerName: "hu.example.com", Hints: hints{Port: 2088, Fingerprint: "chrome", Trust: "ca"}}, {Tag: "hy2", Protocol: "hysteria", Port: 8443, Network: "hysteria", Security: "tls", ServerName: "hy.example.com", - Hints: Hints{Port: 8443, Fingerprint: "chrome", Trust: "pin", HopPorts: "20000-30000", + Hints: hints{Port: 8443, Fingerprint: "chrome", Trust: "pin", HopPorts: "20000-30000", PinSHA256: "aabbccddeeff00112233445566778899aabbccddeeff00112233445566778899"}}, {Tag: "vmess-ws", Protocol: "vmess", Port: 2096, Network: "ws", Security: "none", - Hints: Hints{Port: 2096, Fingerprint: "chrome", Trust: "ca"}}, + Hints: hints{Port: 2096, Fingerprint: "chrome", Trust: "ca"}}, {Tag: "trojan", Protocol: "trojan", Port: 8444, Network: "raw", Security: "tls", - ServerName: "t.example.com", Hints: Hints{Port: 8444, Fingerprint: "chrome", Trust: "ca"}}, + ServerName: "t.example.com", Hints: hints{Port: 8444, Fingerprint: "chrome", Trust: "ca"}}, } if len(inbounds) != len(want) { t.Fatalf("got %d inbounds, want %d", len(inbounds), len(want)) @@ -55,26 +58,26 @@ func TestReadConfigShapes(t *testing.T) { } } -func summary(in Inbound) string { +func summary(in inbound) string { return fmt.Sprintf("%s %s %d %s %s %q %+v", in.Tag, in.Protocol, in.Port, in.Network, in.Security, in.ServerName, in.Hints) } func TestReadConfigAliases(t *testing.T) { - inbounds, err := ReadConfig("testdata/aliases.json") + inbounds, err := readConfig("testdata/aliases.json") if err != nil { t.Fatal(err) } - byTag := map[string]Inbound{} + byTag := map[string]inbound{} for _, in := range inbounds { byTag[in.Tag] = in } tests := []struct { tag string network string - check func(t *testing.T, in Inbound) + check func(t *testing.T, in inbound) }{ - {"a1", "xhttp", func(t *testing.T, in Inbound) { // method wins over network + {"a1", "xhttp", func(t *testing.T, in inbound) { // method wins over network stream := object(in.Raw["streamSettings"]) if _, ok := stream["method"]; ok { t.Error("method kept") @@ -87,12 +90,12 @@ func TestReadConfigAliases(t *testing.T) { } }}, {"a2", "ws", nil}, - {"a3", "raw", func(t *testing.T, in Inbound) { + {"a3", "raw", func(t *testing.T, in inbound) { if _, ok := object(in.Raw["streamSettings"])["rawSettings"]; !ok { t.Error("tcpSettings not moved to rawSettings") } }}, - {"a4", "hysteria", func(t *testing.T, in Inbound) { + {"a4", "hysteria", func(t *testing.T, in inbound) { settings := object(in.Raw["settings"]) if _, ok := settings["users"]; ok { t.Error("users kept") @@ -163,7 +166,7 @@ func setPath(m map[string]any, path string, value any) { m[last] = value } -func readOne(t *testing.T, inbound map[string]any) Inbound { +func readOne(t *testing.T, inbound map[string]any) inbound { t.Helper() data, err := json.Marshal(map[string]any{"inbounds": []any{inbound}}) if err != nil { @@ -173,7 +176,7 @@ func readOne(t *testing.T, inbound map[string]any) Inbound { if err := os.WriteFile(path, data, 0o600); err != nil { t.Fatal(err) } - inbounds, err := ReadConfig(path) + inbounds, err := readConfig(path) if err != nil { t.Fatal(err) } @@ -296,9 +299,9 @@ func TestReadConfigAcceptsEdgeShapes(t *testing.T) { name string kind string set map[string]any - check func(t *testing.T, in Inbound) + check func(t *testing.T, in inbound) }{ - {"no streamSettings", "vless-tls", map[string]any{"streamSettings": nil}, func(t *testing.T, in Inbound) { + {"no streamSettings", "vless-tls", map[string]any{"streamSettings": nil}, func(t *testing.T, in inbound) { if in.Network != "raw" || in.Security != "none" || in.ServerName != "" { t.Errorf("got %s/%s/%q", in.Network, in.Security, in.ServerName) } @@ -307,19 +310,19 @@ func TestReadConfigAcceptsEdgeShapes(t *testing.T) { } }}, {"sni override picks one of the serverNames", "reality", map[string]any{ - "x-usher.sni": "b.example.com"}, func(t *testing.T, in Inbound) { + "x-usher.sni": "b.example.com"}, func(t *testing.T, in inbound) { if in.ServerName != "b.example.com" { t.Errorf("ServerName = %q", in.ServerName) } }}, {"sni override beside wildcard entries", "reality", map[string]any{ "streamSettings.realitySettings.serverNames": []any{"*.example.com", "front.example.com"}, - "x-usher.sni": "front.example.com"}, func(t *testing.T, in Inbound) { + "x-usher.sni": "front.example.com"}, func(t *testing.T, in inbound) { if in.ServerName != "front.example.com" { t.Errorf("ServerName = %q", in.ServerName) } }}, - {"reality safari", "reality", map[string]any{"x-usher.fingerprint": "safari"}, func(t *testing.T, in Inbound) { + {"reality safari", "reality", map[string]any{"x-usher.fingerprint": "safari"}, func(t *testing.T, in inbound) { if in.Hints.Fingerprint != "safari" { t.Errorf("fingerprint = %q", in.Hints.Fingerprint) } @@ -327,13 +330,13 @@ func TestReadConfigAcceptsEdgeShapes(t *testing.T) { {"pin from inline certificate", "vless-tls", map[string]any{ "x-usher.trust": "pin", "streamSettings.tlsSettings.certificates": []any{map[string]any{"certificate": []any{"-----BEGIN CERTIFICATE-----"}}}}, - func(t *testing.T, in Inbound) { - if in.Hints.Trust != TrustPin || in.Hints.PinSHA256 != "" { + func(t *testing.T, in inbound) { + if in.Hints.Trust != trustPin || in.Hints.PinSHA256 != "" { t.Errorf("hints = %+v", in.Hints) } }}, {"x-usher port and sni", "vless-tls", map[string]any{"x-usher.port": 8443, "x-usher.sni": "o.example.com"}, - func(t *testing.T, in Inbound) { + func(t *testing.T, in inbound) { if in.Hints.Port != 8443 || in.ServerName != "o.example.com" || in.Port != 443 { t.Errorf("port %d hints %+v sni %q", in.Port, in.Hints, in.ServerName) } @@ -412,7 +415,7 @@ func TestReadConfigSkipsAndDuplicates(t *testing.T) { if err := os.WriteFile(path, []byte(doc), 0o600); err != nil { t.Fatal(err) } - inbounds, err := ReadConfig(path) + inbounds, err := readConfig(path) if err != nil { t.Fatal(err) } @@ -438,8 +441,50 @@ func TestReadConfigFileErrors(t *testing.T) { t.Fatal(err) } for _, path := range []string{filepath.Join(dir, "missing.json"), bad, notObject} { - if _, err := ReadConfig(path); err == nil { + if _, err := readConfig(path); err == nil { t.Errorf("%s: no error", filepath.Base(path)) } } } + +func TestDeployExampleConfig(t *testing.T) { + path := filepath.Join("..", "..", "deploy", "xray", "config.json") + wantTags := []string{"vless-reality", "hysteria2"} + + // As shipped, the placeholders make the Reality inbound fail on its key. + inbounds, err := readConfig(path) + if err != nil { + t.Fatal(err) + } + var tags []string + for _, in := range inbounds { + tags = append(tags, in.Tag) + } + if !slices.Equal(tags, wantTags) { + t.Errorf("tags = %v, want %v", tags, wantTags) + } + if inbounds[0].Err == nil || !strings.Contains(inbounds[0].Err.Error(), "privateKey") { + t.Errorf("shipped Reality inbound error = %v, want one about privateKey", inbounds[0].Err) + } + + // With the placeholder key replaced, every inbound is valid. + data, err := os.ReadFile(path) + if err != nil { + t.Fatal(err) + } + key := base64.RawURLEncoding.EncodeToString(bytes.Repeat([]byte{1}, 32)) + replaced := strings.ReplaceAll(string(data), "REPLACE_WITH_X25519_PRIVATE_KEY", key) + filled := filepath.Join(t.TempDir(), "config.json") + if err := os.WriteFile(filled, []byte(replaced), 0o600); err != nil { + t.Fatal(err) + } + inbounds, err = readConfig(filled) + if err != nil { + t.Fatal(err) + } + for _, in := range inbounds { + if in.Err != nil { + t.Errorf("inbound %s: %v", in.Tag, in.Err) + } + } +} diff --git a/internal/xray/drift.go b/internal/xray/drift.go index c2ccb19..c31312d 100644 --- a/internal/xray/drift.go +++ b/internal/xray/drift.go @@ -2,7 +2,6 @@ package xray import ( "context" - "encoding/json" "errors" "fmt" "log/slog" @@ -13,7 +12,6 @@ import ( "github.com/xtls/xray-core/app/proxyman/command" "github.com/xtls/xray-core/common/serial" "github.com/xtls/xray-core/core" - "github.com/xtls/xray-core/infra/conf" "github.com/xtls/xray-core/transport/internet" "github.com/xtls/xray-core/transport/internet/tls" "google.golang.org/protobuf/proto" @@ -29,28 +27,17 @@ type driftError struct { func (e *driftError) Error() string { return e.reason } -// errDoesNotBuild stands for every error of Xray's inbound Build. Xray quotes the keys and short -// ids it rejects, so its text never reaches a reason. -var errDoesNotBuild = errors.New("the file inbound does not build as an Xray inbound") - // buildInbound builds the inbound object the way Xray does, without the certificate entries that -// name a file, which Xray reads from its own filesystem. +// name a file, which Xray reads from its own filesystem. Xray quotes the keys and short ids it +// rejects, so the error of its decode or Build never reaches a reason. func buildInbound(raw map[string]any) (*core.InboundHandlerConfig, error) { - raw, err := withoutFileCertificates(raw) + detour, err := decodeInbound(withoutFileCertificates(raw)) if err != nil { - return nil, err - } - encoded, err := json.Marshal(raw) - if err != nil { - return nil, fmt.Errorf("encoding the file inbound: %w", err) - } - var detour conf.InboundDetourConfig - if err := json.Unmarshal(encoded, &detour); err != nil { - return nil, errDoesNotBuild + return nil, errors.New("the file inbound does not build as an Xray inbound") } built, err := detour.Build() if err != nil { - return nil, errDoesNotBuild + return nil, errors.New("the file inbound does not build as an Xray inbound") } return built, nil } @@ -75,7 +62,7 @@ func isManagedProxyType(proxyType string) bool { // itself, so a changed file is no drift. A reason names the differing parts and never a // value. Running inbounds of a managed protocol that the file does not define are logged. func checkDrift(ctx context.Context, client command.HandlerServiceClient, log *slog.Logger, - inbounds []Inbound) (map[string]error, error) { + inbounds []inbound) (map[string]error, error) { resp, err := client.ListInbounds(ctx, &command.ListInboundsRequest{}) if err != nil { return nil, fmt.Errorf("listing xray inbounds: %w", err) @@ -120,7 +107,6 @@ func checkDrift(ctx context.Context, client command.HandlerServiceClient, log *s drift[in.Tag] = &driftError{ reason: fmt.Sprintf("inbound %q: differs from the running Xray in %s, restart Xray to load the file", in.Tag, strings.Join(parts, ", ")), - unusable: slices.Contains(parts, partProxyType), } } } @@ -136,17 +122,16 @@ func checkDrift(ctx context.Context, client command.HandlerServiceClient, log *s const partProxyType = "proxy type" // differingParts returns the names of the parts of the file inbound that differ from the running -// one: "proxy type", "proxy settings", "port", "stream settings" and "receiver settings". -func differingParts(in Inbound, live *core.InboundHandlerConfig) ([]string, error) { +// one: "proxy settings", "port", "stream settings" and "receiver settings". The caller has +// checked that both run the same proxy type. +func differingParts(in inbound, live *core.InboundHandlerConfig) ([]string, error) { built, err := buildInbound(in.Raw) if err != nil { return nil, err } var parts []string - if built.GetProxySettings().GetType() != live.GetProxySettings().GetType() { - parts = append(parts, partProxyType) - } else if equal, err := equalMessages(built.GetProxySettings(), live.GetProxySettings()); err != nil { + if equal, err := equalMessages(built.GetProxySettings(), live.GetProxySettings()); err != nil { return nil, err } else if !equal { parts = append(parts, "proxy settings") @@ -230,19 +215,12 @@ func dropPathCertificates(stream *internet.StreamConfig) error { // withoutFileCertificates returns a copy of an inbound object without the tlsSettings certificate // entries that name a file. -func withoutFileCertificates(raw map[string]any) (map[string]any, error) { - encoded, err := json.Marshal(raw) - if err != nil { - return nil, fmt.Errorf("encoding the file inbound: %w", err) - } - clone, err := decodeObject(encoded) - if err != nil { - return nil, err - } +func withoutFileCertificates(raw map[string]any) map[string]any { + clone := deepCopy(raw).(map[string]any) tlsSettings := object(object(clone["streamSettings"])["tlsSettings"]) certs, ok := tlsSettings["certificates"].([]any) if !ok { - return clone, nil + return clone } tlsSettings["certificates"] = slices.DeleteFunc(certs, func(c any) bool { entry := object(c) @@ -250,5 +228,5 @@ func withoutFileCertificates(raw map[string]any) (map[string]any, error) { key, _ := entry["keyFile"].(string) return file != "" || key != "" }) - return clone, nil + return clone } diff --git a/internal/xray/drift_test.go b/internal/xray/drift_test.go index e9b640a..da90f60 100644 --- a/internal/xray/drift_test.go +++ b/internal/xray/drift_test.go @@ -13,28 +13,11 @@ import ( "strings" "testing" - "github.com/xtls/xray-core/app/proxyman/command" "github.com/xtls/xray-core/core" "github.com/xtls/xray-core/infra/conf" xjson "github.com/xtls/xray-core/infra/conf/json" - "google.golang.org/grpc" ) -// fakeHandlerService answers ListInbounds only; any other call panics on the nil embedded client. -type fakeHandlerService struct { - command.HandlerServiceClient - inbounds []*core.InboundHandlerConfig - err error -} - -func (f *fakeHandlerService) ListInbounds(_ context.Context, _ *command.ListInboundsRequest, - _ ...grpc.CallOption) (*command.ListInboundsResponse, error) { - if f.err != nil { - return nil, f.err - } - return &command.ListInboundsResponse{Inbounds: f.inbounds}, nil -} - // driftConfig holds the parts of the test server config that the cases change. type driftConfig struct { dir string @@ -177,13 +160,13 @@ func TestCheckDrift(t *testing.T) { if tt.change != nil { tt.change(&changed) } - inbounds, err := ReadConfig(changed.write(t)) + inbounds, err := readConfig(changed.write(t)) if err != nil { t.Fatal(err) } var logs bytes.Buffer - drift, err := checkDrift(context.Background(), &fakeHandlerService{inbounds: running}, + drift, err := checkDrift(context.Background(), newUserService(running), slog.New(slog.NewTextHandler(&logs, nil)), inbounds) if err != nil { t.Fatal(err) @@ -222,13 +205,13 @@ func TestCheckDriftExtraRuntimeTag(t *testing.T) { stray.extra = `{"tag": "stray", "protocol": "vless", "port": 2999, "settings": {"clients": [], "decryption": "none"}},` running := runningFrom(t, stray.write(t)) - inbounds, err := ReadConfig(base.write(t)) + inbounds, err := readConfig(base.write(t)) if err != nil { t.Fatal(err) } var logs bytes.Buffer - drift, err := checkDrift(context.Background(), &fakeHandlerService{inbounds: running}, + drift, err := checkDrift(context.Background(), newUserService(running), slog.New(slog.NewTextHandler(&logs, nil)), inbounds) if err != nil { t.Fatal(err) @@ -247,13 +230,13 @@ func TestCheckDriftExtraRuntimeTag(t *testing.T) { func TestCheckDriftSkipsInvalidInbound(t *testing.T) { base := newDriftConfig(t.TempDir()) running := runningFrom(t, base.write(t)) - inbounds, err := ReadConfig(base.write(t)) + inbounds, err := readConfig(base.write(t)) if err != nil { t.Fatal(err) } inbounds[0].Err = errors.New("invalid") - drift, err := checkDrift(context.Background(), &fakeHandlerService{inbounds: running}, + drift, err := checkDrift(context.Background(), newUserService(running), slog.New(slog.NewTextHandler(&bytes.Buffer{}, nil)), inbounds) if err != nil { t.Fatal(err) @@ -264,7 +247,7 @@ func TestCheckDriftSkipsInvalidInbound(t *testing.T) { } func TestCheckDriftListError(t *testing.T) { - _, err := checkDrift(context.Background(), &fakeHandlerService{err: errors.New("unavailable")}, + _, err := checkDrift(context.Background(), &userService{inboundErr: errors.New("unavailable")}, slog.New(slog.NewTextHandler(&bytes.Buffer{}, nil)), nil) if err == nil || !strings.Contains(err.Error(), "unavailable") { t.Errorf("err = %v, want the list error wrapped", err) @@ -286,7 +269,7 @@ func TestCheckDriftMatchesXrayBuild(t *testing.T) { } for _, tt := range tests { t.Run(tt.name, func(t *testing.T) { - inbounds, err := ReadConfig(tt.path) + inbounds, err := readConfig(tt.path) if err != nil { t.Fatal(err) } @@ -295,7 +278,7 @@ func TestCheckDriftMatchesXrayBuild(t *testing.T) { t.Fatalf("%s: %v", in.Tag, in.Err) } } - drift, err := checkDrift(context.Background(), &fakeHandlerService{inbounds: runningFrom(t, tt.path)}, + drift, err := checkDrift(context.Background(), newUserService(runningFrom(t, tt.path)), slog.New(slog.DiscardHandler), inbounds) if err != nil { t.Fatal(err) @@ -315,7 +298,7 @@ func TestCheckDriftDoesNotQuoteBuildErrors(t *testing.T) { running := runningFrom(t, base.write(t)) broken := base broken.shortID = bad - inbounds, err := ReadConfig(broken.write(t)) + inbounds, err := readConfig(broken.write(t)) if err != nil { t.Fatal(err) } @@ -323,7 +306,7 @@ func TestCheckDriftDoesNotQuoteBuildErrors(t *testing.T) { t.Fatalf("want the reality inbound to pass validation, got %+v", inbounds[0].Err) } - drift, err := checkDrift(context.Background(), &fakeHandlerService{inbounds: running}, + drift, err := checkDrift(context.Background(), newUserService(running), slog.New(slog.DiscardHandler), inbounds) if err != nil { t.Fatal(err) @@ -356,11 +339,11 @@ func TestCheckDriftUnusable(t *testing.T) { base := newDriftConfig(t.TempDir()) running := runningFrom(t, base.write(t)) tt.change(&base) - inbounds, err := ReadConfig(base.write(t)) + inbounds, err := readConfig(base.write(t)) if err != nil { t.Fatal(err) } - drift, err := checkDrift(context.Background(), &fakeHandlerService{inbounds: running}, + drift, err := checkDrift(context.Background(), newUserService(running), slog.New(slog.DiscardHandler), inbounds) if err != nil { t.Fatal(err) diff --git a/internal/xray/enroll.go b/internal/xray/enroll.go index fe90448..809d4c8 100644 --- a/internal/xray/enroll.go +++ b/internal/xray/enroll.go @@ -17,7 +17,7 @@ const passwordBytes = 32 // enroll makes st hold a credential of the kind the inbound's protocol uses for every user in // users. It creates the missing ones and never changes an existing entry; an existing entry of the // wrong kind is an error. It reports whether it created any. On error st is left as it was. -func enroll(st *state.State, in Inbound, users []string) (bool, error) { +func enroll(st *state.State, in inbound, users []string) (bool, error) { spec := protocols[in.Protocol] created := make(map[string]state.XrayEntry) for _, user := range users { diff --git a/internal/xray/enroll_test.go b/internal/xray/enroll_test.go index 7478aff..6bc59ae 100644 --- a/internal/xray/enroll_test.go +++ b/internal/xray/enroll_test.go @@ -15,20 +15,20 @@ func TestEnroll(t *testing.T) { seededID := uuid.New() tests := []struct { name string - in Inbound + in inbound seed map[string]state.XrayEntry users []string wantChanged bool wantErr string }{ - {name: "vless gets ids", in: Inbound{Tag: "t", Protocol: "vless"}, users: []string{"alice", "bob"}, wantChanged: true}, - {name: "vmess gets ids", in: Inbound{Tag: "t", Protocol: "vmess"}, users: []string{"alice"}, wantChanged: true}, - {name: "trojan gets passwords", in: Inbound{Tag: "t", Protocol: "trojan"}, users: []string{"alice"}, wantChanged: true}, - {name: "hysteria gets passwords", in: Inbound{Tag: "t", Protocol: "hysteria"}, users: []string{"alice"}, wantChanged: true}, - {name: "seeded entry kept", in: Inbound{Tag: "t", Protocol: "vless"}, + {name: "vless gets ids", in: inbound{Tag: "t", Protocol: "vless"}, users: []string{"alice", "bob"}, wantChanged: true}, + {name: "vmess gets ids", in: inbound{Tag: "t", Protocol: "vmess"}, users: []string{"alice"}, wantChanged: true}, + {name: "trojan gets passwords", in: inbound{Tag: "t", Protocol: "trojan"}, users: []string{"alice"}, wantChanged: true}, + {name: "hysteria gets passwords", in: inbound{Tag: "t", Protocol: "hysteria"}, users: []string{"alice"}, wantChanged: true}, + {name: "seeded entry kept", in: inbound{Tag: "t", Protocol: "vless"}, seed: map[string]state.XrayEntry{"t/alice": {ID: seededID}}, users: []string{"alice"}}, - {name: "no users", in: Inbound{Tag: "t", Protocol: "vless"}}, - {name: "entry of the wrong kind", in: Inbound{Tag: "t", Protocol: "hysteria"}, + {name: "no users", in: inbound{Tag: "t", Protocol: "vless"}}, + {name: "entry of the wrong kind", in: inbound{Tag: "t", Protocol: "hysteria"}, seed: map[string]state.XrayEntry{"t/alice": {ID: seededID}}, users: []string{"bob", "alice"}, wantErr: "does not fit protocol hysteria"}, } diff --git a/internal/xray/mirror.go b/internal/xray/mirror.go index b8c1b06..4772888 100644 --- a/internal/xray/mirror.go +++ b/internal/xray/mirror.go @@ -25,7 +25,7 @@ const ( // the client needs from the server config and rebuilds the TLS and Reality blocks, so nothing // secret or server-only reaches the result. host is the address clients dial unless x-usher sets // one. The xhttp tuning is packed into extra, the one form both the .json and the share link carry. -func clientOutbound(in Inbound, host string, entry state.XrayEntry) (map[string]any, error) { +func clientOutbound(in inbound, host string, entry state.XrayEntry) (map[string]any, error) { stream, err := clientStream(in, entry) if err != nil { return nil, err @@ -34,16 +34,7 @@ func clientOutbound(in Inbound, host string, entry state.XrayEntry) (map[string] host = in.Hints.Address } settings := map[string]any{"address": host, "port": in.Hints.Port} - switch in.Protocol { - case "vless": - settings["id"], settings["encryption"], settings["flow"] = entry.ID.String(), "none", vlessFlow(in) - case "vmess": - settings["id"], settings["security"] = entry.ID.String(), "auto" - case "trojan": - settings["password"] = entry.Password - case "hysteria": - settings["version"] = 2 - } + protocols[in.Protocol].clientSettings(in, entry, settings) return map[string]any{ "tag": in.Tag, "protocol": in.Protocol, @@ -52,7 +43,7 @@ func clientOutbound(in Inbound, host string, entry state.XrayEntry) (map[string] }, nil } -func clientStream(in Inbound, entry state.XrayEntry) (map[string]any, error) { +func clientStream(in inbound, entry state.XrayEntry) (map[string]any, error) { server := object(in.Raw["streamSettings"]) stream := map[string]any{"network": in.Network, "security": in.Security} @@ -73,13 +64,13 @@ func clientStream(in Inbound, entry state.XrayEntry) (map[string]any, error) { } switch in.Security { - case "tls": + case securityTLS: tls, err := clientTLS(in, object(server["tlsSettings"])) if err != nil { return nil, err } stream["tlsSettings"] = tls - case "reality": + case securityReality: reality, err := clientReality(in, object(server["realitySettings"])) if err != nil { return nil, err @@ -122,7 +113,7 @@ func xhttpSettings(server map[string]any) map[string]any { // hysteriaMask returns the client finalmask: the salamander password of the server, then a udphop // entry when x-usher.hop_ports is set. libxray reads a link only in that order. -func hysteriaMask(in Inbound, server map[string]any) map[string]any { +func hysteriaMask(in inbound, server map[string]any) map[string]any { var udp []any serverUDP, _ := object(server["finalmask"])["udp"].([]any) for _, item := range serverUDP { @@ -152,12 +143,12 @@ func hysteriaMask(in Inbound, server map[string]any) map[string]any { // clientTLS rebuilds the tlsSettings block: the name, the ALPN list, the fingerprint and, for // trust "pin", the pinned leaf hash. -func clientTLS(in Inbound, server map[string]any) (map[string]any, error) { +func clientTLS(in inbound, server map[string]any) (map[string]any, error) { tls := map[string]any{"serverName": in.ServerName, "fingerprint": in.Hints.Fingerprint} if alpn, ok := server["alpn"]; ok { tls["alpn"] = deepCopy(alpn) } - if in.Hints.Trust != TrustPin { + if in.Hints.Trust != trustPin { return tls, nil } pin := in.Hints.PinSHA256 @@ -223,7 +214,7 @@ func realityKeys(server map[string]any) (key *ecdh.PrivateKey, seed []byte, fiel // clientReality rebuilds the realitySettings block from the server's: the public key of its // privateKey, its first short id and, with an mldsa65Seed, the matching verify key. -func clientReality(in Inbound, server map[string]any) (map[string]any, error) { +func clientReality(in inbound, server map[string]any) (map[string]any, error) { key, seed, field, reason := realityKeys(server) if field != "" { return nil, fmt.Errorf("inbound %q: streamSettings.realitySettings.%s: %s", in.Tag, field, reason) @@ -247,7 +238,7 @@ func clientReality(in Inbound, server map[string]any) (map[string]any, error) { return reality, nil } -// deepCopy copies the maps and lists of a decoded JSON value, so a result never aliases Inbound.Raw. +// deepCopy copies the maps and lists of a decoded JSON value, so a result never aliases inbound.Raw. func deepCopy(v any) any { switch v := v.(type) { case map[string]any: diff --git a/internal/xray/mirror_test.go b/internal/xray/mirror_test.go index fe235e2..dfbf9d3 100644 --- a/internal/xray/mirror_test.go +++ b/internal/xray/mirror_test.go @@ -33,14 +33,14 @@ const ( var testID = uuid.MustParse("3f2504e0-4f89-41d3-9a0c-0305e82c3301") -func testEntry(in Inbound) state.XrayEntry { +func testEntry(in inbound) state.XrayEntry { if protocols[in.Protocol].usesID { return state.XrayEntry{ID: testID} } return state.XrayEntry{Password: testPassword} } -func mirrorOne(t *testing.T, in Inbound) map[string]any { +func mirrorOne(t *testing.T, in inbound) map[string]any { t.Helper() if in.Err != nil { t.Fatalf("inbound %q: %v", in.Tag, in.Err) @@ -72,9 +72,9 @@ func wantJSON(t *testing.T, got map[string]any, want string) { } } -func byTag(t *testing.T, inbounds []Inbound) map[string]Inbound { +func byTag(t *testing.T, inbounds []inbound) map[string]inbound { t.Helper() - m := map[string]Inbound{} + m := map[string]inbound{} for _, in := range inbounds { m[in.Tag] = in } @@ -82,7 +82,7 @@ func byTag(t *testing.T, inbounds []Inbound) map[string]Inbound { } func TestClientOutboundShapes(t *testing.T) { - inbounds, err := ReadConfig("testdata/server.json") + inbounds, err := readConfig("testdata/server.json") if err != nil { t.Fatal(err) } @@ -423,7 +423,7 @@ func TestClientOutboundXHTTP(t *testing.T) { object(extra["xmux"])["a"] = "changed" server := object(object(in.Raw["streamSettings"])["xhttpSettings"]) if object(object(server["extra"])["xmux"])["a"] != "1" { - t.Error("the result shares memory with Inbound.Raw") + t.Error("the result shares memory with inbound.Raw") } }) } diff --git a/internal/xray/protocol.go b/internal/xray/protocol.go index b35e141..e907681 100644 --- a/internal/xray/protocol.go +++ b/internal/xray/protocol.go @@ -21,8 +21,10 @@ type protocolSpec struct { // usesID is true when the credential is a UUID in XrayEntry.ID, false when it is a password in // XrayEntry.Password. usesID bool + // clientSettings adds the credential and the protocol fields of the client outbound settings. + clientSettings func(in inbound, e state.XrayEntry, settings map[string]any) // account builds the account Xray stores for the entry on the inbound. - account func(in Inbound, e state.XrayEntry) proto.Message + account func(in inbound, e state.XrayEntry) proto.Message // credential returns the part of an account that identifies what the client must present, or // "" for an account of another type. Two accounts with the same credential are the same user. credential func(account proto.Message) string @@ -30,10 +32,13 @@ type protocolSpec struct { // protocols maps the lower case protocol of an inbound to its spec. var protocols = map[string]protocolSpec{ - "vless": { + protocolVLESS: { proxyType: "xray.proxy.vless.inbound.Config", usesID: true, - account: func(in Inbound, e state.XrayEntry) proto.Message { + clientSettings: func(in inbound, e state.XrayEntry, settings map[string]any) { + settings["id"], settings["encryption"], settings["flow"] = e.ID.String(), "none", vlessFlow(in) + }, + account: func(in inbound, e state.XrayEntry) proto.Message { return &vless.Account{Id: e.ID.String(), Flow: vlessFlow(in), Encryption: "none"} }, credential: func(a proto.Message) string { @@ -43,10 +48,13 @@ var protocols = map[string]protocolSpec{ return "" }, }, - "vmess": { + protocolVMess: { proxyType: "xray.proxy.vmess.inbound.Config", usesID: true, - account: func(_ Inbound, e state.XrayEntry) proto.Message { + clientSettings: func(_ inbound, e state.XrayEntry, settings map[string]any) { + settings["id"], settings["security"] = e.ID.String(), "auto" + }, + account: func(_ inbound, e state.XrayEntry) proto.Message { return &vmess.Account{Id: e.ID.String()} }, // Xray adds SecuritySettings to a stored VMess account, so only the id is compared. @@ -57,9 +65,12 @@ var protocols = map[string]protocolSpec{ return "" }, }, - "trojan": { + protocolTrojan: { proxyType: "xray.proxy.trojan.ServerConfig", - account: func(_ Inbound, e state.XrayEntry) proto.Message { + clientSettings: func(_ inbound, e state.XrayEntry, settings map[string]any) { + settings["password"] = e.Password + }, + account: func(_ inbound, e state.XrayEntry) proto.Message { return &trojan.Account{Password: e.Password} }, credential: func(a proto.Message) string { @@ -69,9 +80,12 @@ var protocols = map[string]protocolSpec{ return "" }, }, - "hysteria": { + protocolHysteria: { proxyType: "xray.proxy.hysteria.ServerConfig", - account: func(_ Inbound, e state.XrayEntry) proto.Message { + clientSettings: func(_ inbound, _ state.XrayEntry, settings map[string]any) { + settings["version"] = 2 + }, + account: func(_ inbound, e state.XrayEntry) proto.Message { return &account.Account{Auth: e.Password} }, credential: func(a proto.Message) string { @@ -85,8 +99,8 @@ var protocols = map[string]protocolSpec{ // vlessFlow returns the flow of the VLESS users of the inbound: vision for raw behind TLS or // Reality, none otherwise. -func vlessFlow(in Inbound) string { - if in.Network == "raw" && (in.Security == "tls" || in.Security == "reality") { +func vlessFlow(in inbound) string { + if in.Network == "raw" && (in.Security == securityTLS || in.Security == securityReality) { return vlessVisionFlow } return "" diff --git a/internal/xray/protocol_test.go b/internal/xray/protocol_test.go new file mode 100644 index 0000000..23dd9ed --- /dev/null +++ b/internal/xray/protocol_test.go @@ -0,0 +1,42 @@ +package xray + +import ( + "testing" + + "github.com/google/uuid" + + "github.com/mrcsin/usher/internal/state" +) + +func TestProtocolCredential(t *testing.T) { + id := uuid.New() + tests := []struct { + name string + in inbound + entry state.XrayEntry + want string + wrongFor string // protocol whose credential function must return "" for this account + }{ + {"vless vision", inbound{Protocol: "vless", Network: "raw", Security: "reality"}, + state.XrayEntry{ID: id}, id.String() + " xtls-rprx-vision", "trojan"}, + {"vless ws", inbound{Protocol: "vless", Network: "ws", Security: "tls"}, + state.XrayEntry{ID: id}, id.String() + " ", "vmess"}, + {"vless raw without security", inbound{Protocol: "vless", Network: "raw", Security: "none"}, + state.XrayEntry{ID: id}, id.String() + " ", "hysteria"}, + {"vmess", inbound{Protocol: "vmess"}, state.XrayEntry{ID: id}, id.String(), "vless"}, + {"trojan", inbound{Protocol: "trojan"}, state.XrayEntry{Password: "pw"}, "pw", "hysteria"}, + {"hysteria", inbound{Protocol: "hysteria"}, state.XrayEntry{Password: "pw"}, "pw", "trojan"}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + spec := protocols[tt.in.Protocol] + account := spec.account(tt.in, tt.entry) + if got := spec.credential(account); got != tt.want { + t.Errorf("credential = %q, want %q", got, tt.want) + } + if got := protocols[tt.wrongFor].credential(account); got != "" { + t.Errorf("%s credential of a %s account = %q, want empty", tt.wrongFor, tt.in.Protocol, got) + } + }) + } +} diff --git a/internal/xray/render.go b/internal/xray/render.go index 2ea5670..248af47 100644 --- a/internal/xray/render.go +++ b/internal/xray/render.go @@ -70,7 +70,7 @@ func renderLink(clientJSON []byte) ([]byte, error) { // renderFiles returns the client files of one user on the inbound, keyed by clients.Path: always // the .json, and the .txt unless the inbound cannot have a link. A link libxray refuses is a // defect of usher, which builds only shapes libxray knows; it is logged and the .json stays. -func renderFiles(log *slog.Logger, in Inbound, host, user string, entry state.XrayEntry) (map[string][]byte, error) { +func renderFiles(log *slog.Logger, in inbound, host, user string, entry state.XrayEntry) (map[string][]byte, error) { outbound, err := clientOutbound(in, host, entry) if err != nil { return nil, err @@ -81,7 +81,7 @@ func renderFiles(log *slog.Logger, in Inbound, host, user string, entry state.Xr } files := map[string][]byte{clients.Path(user, in.Tag, JSONSuffix): clientJSON} - if in.Protocol == "hysteria" && in.Hints.Trust == TrustPin { + if in.Protocol == protocolHysteria && in.Hints.Trust == trustPin { return files, nil // v2rayNG and Hiddify do not read pcs for hysteria2:// } link, err := renderLink(clientJSON) diff --git a/internal/xray/render_test.go b/internal/xray/render_test.go index 6b54f40..92d6171 100644 --- a/internal/xray/render_test.go +++ b/internal/xray/render_test.go @@ -24,7 +24,7 @@ var update = flag.Bool("update", false, "rewrite the golden files") const goldenDir = "testdata/golden" func TestGolden(t *testing.T) { - inbounds, err := ReadConfig(filepath.Join(goldenDir, "server.json")) + inbounds, err := readConfig(filepath.Join(goldenDir, "server.json")) if err != nil { t.Fatal(err) } @@ -182,11 +182,11 @@ func TestRenderJSONRejectsBadOutbound(t *testing.T) { func TestRenderFilesLinkDefect(t *testing.T) { // libxray refuses a single-port hop while the outbound still builds: the .json stays and the // defect is logged without a secret. - inbounds, err := ReadConfig(filepath.Join(goldenDir, "server.json")) + inbounds, err := readConfig(filepath.Join(goldenDir, "server.json")) if err != nil { t.Fatal(err) } - var hop Inbound + var hop inbound for _, in := range inbounds { if in.Tag == "hy2-hop" { hop = in diff --git a/internal/xray/session.go b/internal/xray/session.go index 6afd494..80075ff 100644 --- a/internal/xray/session.go +++ b/internal/xray/session.go @@ -29,8 +29,8 @@ type Dial func() (command.HandlerServiceClient, func(), error) type Session struct { client command.HandlerServiceClient closeConn func() - inbounds []Inbound - byTag map[string]Inbound + inbounds []inbound + byTag map[string]inbound drift map[string]error // broken maps a tag whose accounts and client files cannot be built this pass to the reason. // Apply only revokes users on such a tag. @@ -44,7 +44,7 @@ type Session struct { // socketPath is a stale socket of a killed Xray, Open removes it and fails, so the pass reports Xray // down and the restart policy can start Xray again. func Open(ctx context.Context, dial Dial, socketPath, configPath, host string, log *slog.Logger) (*Session, error) { - inbounds, err := ReadConfig(configPath) + inbounds, err := readConfig(configPath) if err != nil { return nil, err } @@ -67,7 +67,7 @@ func Open(ctx context.Context, dial Dial, socketPath, configPath, host string, l closeConn() return nil, err } - byTag := make(map[string]Inbound, len(inbounds)) + byTag := make(map[string]inbound, len(inbounds)) for _, in := range inbounds { byTag[in.Tag] = in } @@ -99,9 +99,9 @@ func (s *Session) Interfaces() []string { // Prepare enrolls the users on the inbound in st and renders their client files. When the inbound // failed structural validation, Prepare returns the error and the pass skips Apply, so usher // touches no user of that tag. When only the x-usher block failed, an enrolled credential does not -// fit the inbound or a file cannot be rendered, Prepare keeps -// the reason and returns no error, and Apply then still revokes the users who left and returns the -// reason, which keeps the old client files. +// fit the inbound or a file cannot be rendered, Prepare keeps the reason and returns no error, and +// Apply then still revokes the users who left and returns the reason, which keeps the old client +// files. func (s *Session) Prepare(st *state.State, name string, users []string) (map[string][]byte, bool, error) { in := s.byTag[name] if in.Err != nil { @@ -161,7 +161,7 @@ type userChanges struct { // reconcile lists the users of the inbound, removes foreign and changed ones, adds missing and // changed ones, and logs the difference. It tries every operation and joins the errors, so one // failure does not hold back a removal; the next pass converges from the listed state. -func (s *Session) reconcile(ctx context.Context, st *state.State, in Inbound, users []string) error { +func (s *Session) reconcile(ctx context.Context, st *state.State, in inbound, users []string) error { desired := make(map[string]*serial.TypedMessage, len(users)) credentials := make(map[string]string, len(users)) spec := protocols[in.Protocol] @@ -237,8 +237,11 @@ func (s *Session) revoke(ctx context.Context, tag string, users []string) error var changes userChanges var errs []error for _, u := range listed { + if u == nil { + continue + } email := u.GetEmail() - if u == nil || keep[email] { + if keep[email] { continue } if err := s.removeUser(ctx, tag, email); err != nil { @@ -257,7 +260,7 @@ func (s *Session) revoke(ctx context.Context, tag string, users []string) error func (s *Session) logChanges(tag string, changes userChanges) { if len(changes.added)+len(changes.removed)+len(changes.updated) > 0 { - s.log.Info("users changed", "inbound", tag, "added", changes.added, + s.log.Info("users changed", "interface", tag, "added", changes.added, "removed", changes.removed, "updated", changes.updated) } } @@ -280,7 +283,7 @@ func (s *Session) listUsers(ctx context.Context, tag string) ([]*protocol.User, } // liveCredentials maps the email of every user of the inbound to the credential of its account. -func (s *Session) liveCredentials(ctx context.Context, in Inbound) (map[string]string, error) { +func (s *Session) liveCredentials(ctx context.Context, in inbound) (map[string]string, error) { users, err := s.listUsers(ctx, in.Tag) if err != nil { return nil, err diff --git a/internal/xray/session_test.go b/internal/xray/session_test.go index 311ac1c..b9a542a 100644 --- a/internal/xray/session_test.go +++ b/internal/xray/session_test.go @@ -95,7 +95,7 @@ func (f *userService) AlterInbound(_ context.Context, req *command.AlterInboundR } // credentials maps the emails of the tag to their credentials. -func (f *userService) credentials(t *testing.T, in Inbound) map[string]string { +func (f *userService) credentials(t *testing.T, in inbound) map[string]string { t.Helper() out := map[string]string{} for _, u := range f.users[in.Tag] { @@ -108,7 +108,7 @@ func (f *userService) credentials(t *testing.T, in Inbound) map[string]string { return out } -func (f *userService) seed(in Inbound, email string, e state.XrayEntry) { +func (f *userService) seed(in inbound, email string, e state.XrayEntry) { f.users[in.Tag] = append(f.users[in.Tag], &protocol.User{Email: email, Account: serial.ToTypedMessage(protocols[in.Protocol].account(in, e))}) } @@ -137,7 +137,7 @@ func newSessionFixture(t *testing.T, change func(c *driftConfig)) sessionFixture return sessionFixture{session: session, service: service, logs: logs, cfg: cfg} } -func (f sessionFixture) inbound(tag string) Inbound { +func (f sessionFixture) inbound(tag string) inbound { return f.session.byTag[tag] } @@ -279,82 +279,34 @@ func TestSessionBrokenInboundStillRevokes(t *testing.T) { } } -// TestSessionStructuralErrorTouchesNothing: an inbound that fails structural validation returns -// its error from Prepare, so the pass skips Apply, and a direct Apply still removes no user. -func TestSessionStructuralErrorTouchesNothing(t *testing.T) { +// TestSessionApplyGuards: Apply changes no account when the inbound failed structural validation +// or when nothing proves the running inbound takes the file's accounts. A hint error must not +// skip the drift check. The running Xray holds the base config; the file holds the change. +func TestSessionApplyGuards(t *testing.T) { badTag := `{"tag": "bad tag", "protocol": "vless", "port": 9005, "settings": {"clients": [], "decryption": "none"}},` dup := `{"tag": "ws", "protocol": "vless", "port": 9002, "settings": {"clients": [{"id": "5783a3e7-e373-51cd-8642-c83782b807c5"}], "decryption": "none"}},` + badHint := `"x-usher": {"adress": "x"},` tests := []struct { - name string - extra string - tag string + name string + change func(c *driftConfig) + tag string + prepareFails bool // Prepare returns the validation error + wantApplyError string // "" accepts any error }{ - {"static clients", badInbound, "bad"}, - {"invalid tag", badTag, "bad tag"}, - {"duplicate tag", dup, "ws"}, - } - for _, tt := range tests { - t.Run(tt.name, func(t *testing.T) { - f := newSessionFixture(t, func(c *driftConfig) { c.extra = tt.extra }) - in := f.inbound(tt.tag) - f.service.seed(in, "someone-else", idEntry()) - f.service.alters = 0 - - files, changed, err := f.session.Prepare(newState(), tt.tag, []string{"alice"}) - if err == nil || files != nil || changed { - t.Fatalf("Prepare = %v, %v, %v, want the validation error", files, changed, err) - } - if err := f.session.Apply(t.Context(), newState(), tt.tag, []string{"alice"}); err == nil { - t.Error("Apply error = nil, want the validation error") - } - if f.service.alters != 0 { - t.Errorf("AlterInbound calls = %d, want 0", f.service.alters) - } - }) - } -} - -// TestSessionApplySkipsFileThatDoesNotBuild: the file inbound changed protocol and no longer -// builds, so nothing proves the running vless handler can take an account of the new type. -func TestSessionApplySkipsFileThatDoesNotBuild(t *testing.T) { - base := newDriftConfig(t.TempDir()) - running := runningFrom(t, base.write(t)) - broken := base - broken.realityProto = "trojan" - broken.shortID = "zz" - path := broken.write(t) - - service := newUserService(running) - dial := func() (command.HandlerServiceClient, func(), error) { return service, func() {}, nil } - session, err := Open(t.Context(), dial, filepath.Join(t.TempDir(), "xray.sock"), path, - "198.51.100.7", slog.New(slog.DiscardHandler)) - if err != nil { - t.Fatal(err) - } - st := newState() - if _, _, err := session.Prepare(st, "reality", []string{"alice"}); err != nil { - t.Fatal(err) - } - if err := session.Apply(t.Context(), st, "reality", []string{"alice"}); err == nil { - t.Error("Apply error = nil, want the drift reason") - } - if service.alters != 0 { - t.Errorf("AlterInbound calls = %d, want 0", service.alters) - } -} - -// TestSessionHintErrorStillGuardedByDrift: an x-usher hint error must not skip the drift check, so -// Apply revokes nothing from a running inbound it could not compare with the file. -func TestSessionHintErrorStillGuardedByDrift(t *testing.T) { - tests := []struct { - name string - change func(c *driftConfig) - wantErr string - }{ - {"proxy type differs", func(c *driftConfig) { c.realityProto = "trojan" }, "proxy type"}, - {"file does not build", func(c *driftConfig) { c.shortID = "zz" }, "comparing with"}, + {"static clients", func(c *driftConfig) { c.extra = badInbound }, "bad", true, ""}, + {"invalid tag", func(c *driftConfig) { c.extra = badTag }, "bad tag", true, ""}, + {"duplicate tag", func(c *driftConfig) { c.extra = dup }, "ws", true, ""}, + {"proxy type differs and the file does not build", func(c *driftConfig) { + c.realityProto, c.shortID = "trojan", "zz" + }, "reality", false, ""}, + {"hint error, proxy type differs", func(c *driftConfig) { + c.realityProto, c.realityExtra = "trojan", badHint + }, "reality", false, "proxy type"}, + {"hint error, file does not build", func(c *driftConfig) { + c.shortID, c.realityExtra = "zz", badHint + }, "reality", false, "comparing with"}, } for _, tt := range tests { t.Run(tt.name, func(t *testing.T) { @@ -362,28 +314,29 @@ func TestSessionHintErrorStillGuardedByDrift(t *testing.T) { running := runningFrom(t, base.write(t)) file := base tt.change(&file) - file.realityExtra = `"x-usher": {"adress": "x"},` - path := file.write(t) service := newUserService(running) dial := func() (command.HandlerServiceClient, func(), error) { return service, func() {}, nil } - session, err := Open(t.Context(), dial, filepath.Join(t.TempDir(), "xray.sock"), path, + session, err := Open(t.Context(), dial, filepath.Join(t.TempDir(), "xray.sock"), file.write(t), "198.51.100.7", slog.New(slog.DiscardHandler)) if err != nil { t.Fatal(err) } - if session.byTag["reality"].Err == nil { - t.Fatal("reality inbound has no hint error, the fixture is wrong") - } - service.users["reality"] = []*protocol.User{{Email: "operator-own"}} + service.users[tt.tag] = []*protocol.User{{Email: "operator-own"}} + service.alters = 0 st := newState() - if _, _, err := session.Prepare(st, "reality", []string{"alice"}); err != nil { + files, changed, err := session.Prepare(st, tt.tag, []string{"alice"}) + if tt.prepareFails { + if err == nil || files != nil || changed { + t.Fatalf("Prepare = %v, %v, %v, want the validation error", files, changed, err) + } + } else if err != nil { t.Fatal(err) } - err = session.Apply(t.Context(), st, "reality", []string{"alice"}) - if err == nil || !strings.Contains(err.Error(), tt.wantErr) { - t.Errorf("Apply error = %v, want one with %q", err, tt.wantErr) + err = session.Apply(t.Context(), st, tt.tag, []string{"alice"}) + if err == nil || !strings.Contains(err.Error(), tt.wantApplyError) { + t.Errorf("Apply error = %v, want one with %q", err, tt.wantApplyError) } if service.alters != 0 { t.Errorf("AlterInbound calls = %d, want 0", service.alters) @@ -656,29 +609,37 @@ func TestSessionApplyFailures(t *testing.T) { }) } -func TestOpenClosesConnectionWhenDriftCheckFails(t *testing.T) { - cfg := newDriftConfig(t.TempDir()) - service := newUserService(nil) - service.inboundErr = errors.New("api down") - closed := false - dial := func() (command.HandlerServiceClient, func(), error) { return service, func() { closed = true }, nil } - _, err := Open(t.Context(), dial, filepath.Join(t.TempDir(), "xray.sock"), cfg.write(t), "198.51.100.7", - slog.New(slog.DiscardHandler)) - if err == nil || !strings.Contains(err.Error(), "api down") { - t.Errorf("Open error = %v, want the list error", err) - } - if !closed { - t.Error("Open did not close the connection") +func TestOpenFailures(t *testing.T) { + driftFails := newUserService(nil) + driftFails.inboundErr = errors.New("api down") + tests := []struct { + name string + service *userService // nil makes the dial fail + wantErr string + wantClosed bool + }{ + {"drift check fails", driftFails, "api down", true}, + {"dial fails", nil, "connecting to xray", false}, } -} - -func TestOpenDialFails(t *testing.T) { - cfg := newDriftConfig(t.TempDir()) - dial := func() (command.HandlerServiceClient, func(), error) { return nil, nil, errors.New("no socket") } - _, err := Open(t.Context(), dial, filepath.Join(t.TempDir(), "xray.sock"), cfg.write(t), "198.51.100.7", - slog.New(slog.DiscardHandler)) - if err == nil || !strings.Contains(err.Error(), "connecting to xray") { - t.Errorf("Open error = %v, want the connect error", err) + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + closed := false + dial := func() (command.HandlerServiceClient, func(), error) { + if tt.service == nil { + return nil, nil, errors.New("no socket") + } + return tt.service, func() { closed = true }, nil + } + cfg := newDriftConfig(t.TempDir()) + _, err := Open(t.Context(), dial, filepath.Join(t.TempDir(), "xray.sock"), cfg.write(t), + "198.51.100.7", slog.New(slog.DiscardHandler)) + if err == nil || !strings.Contains(err.Error(), tt.wantErr) { + t.Errorf("Open error = %v, want one with %q", err, tt.wantErr) + } + if closed != tt.wantClosed { + t.Errorf("connection closed = %v, want %v", closed, tt.wantClosed) + } + }) } } @@ -706,7 +667,7 @@ func TestSessionApplyPartialFailureConverges(t *testing.T) { } func TestVMessEchoIsNoChange(t *testing.T) { - in := Inbound{Tag: "vm", Protocol: "vmess"} + in := inbound{Tag: "vm", Protocol: "vmess"} e := idEntry() service := newUserService(nil) // Xray stores a VMess account with SecuritySettings added. @@ -714,7 +675,7 @@ func TestVMessEchoIsNoChange(t *testing.T) { Id: e.ID.String(), SecuritySettings: &protocol.SecurityConfig{Type: protocol.SecurityType_AES128_GCM}, })}} - session := &Session{client: service, byTag: map[string]Inbound{"vm": in}, log: slog.New(slog.DiscardHandler)} + session := &Session{client: service, byTag: map[string]inbound{"vm": in}, log: slog.New(slog.DiscardHandler)} st := newState() st.Xray["vm/alice"] = e @@ -726,39 +687,6 @@ func TestVMessEchoIsNoChange(t *testing.T) { } } -func TestProtocolCredential(t *testing.T) { - id := uuid.New() - tests := []struct { - name string - in Inbound - entry state.XrayEntry - want string - wrongFor string // protocol whose credential function must return "" for this account - }{ - {"vless vision", Inbound{Protocol: "vless", Network: "raw", Security: "reality"}, - state.XrayEntry{ID: id}, id.String() + " xtls-rprx-vision", "trojan"}, - {"vless ws", Inbound{Protocol: "vless", Network: "ws", Security: "tls"}, - state.XrayEntry{ID: id}, id.String() + " ", "vmess"}, - {"vless raw without security", Inbound{Protocol: "vless", Network: "raw", Security: "none"}, - state.XrayEntry{ID: id}, id.String() + " ", "hysteria"}, - {"vmess", Inbound{Protocol: "vmess"}, state.XrayEntry{ID: id}, id.String(), "vless"}, - {"trojan", Inbound{Protocol: "trojan"}, state.XrayEntry{Password: "pw"}, "pw", "hysteria"}, - {"hysteria", Inbound{Protocol: "hysteria"}, state.XrayEntry{Password: "pw"}, "pw", "trojan"}, - } - for _, tt := range tests { - t.Run(tt.name, func(t *testing.T) { - spec := protocols[tt.in.Protocol] - account := spec.account(tt.in, tt.entry) - if got := spec.credential(account); got != tt.want { - t.Errorf("credential = %q, want %q", got, tt.want) - } - if got := protocols[tt.wrongFor].credential(account); got != "" { - t.Errorf("%s credential of a %s account = %q, want empty", tt.wrongFor, tt.in.Protocol, got) - } - }) - } -} - func TestOpenStaleSocketReportsDown(t *testing.T) { cfg := newDriftConfig(t.TempDir()) path := cfg.write(t) diff --git a/internal/xray/socket.go b/internal/xray/socket.go index b75fd67..19348e2 100644 --- a/internal/xray/socket.go +++ b/internal/xray/socket.go @@ -22,7 +22,8 @@ func removeStaleSocket(socketPath string) (bool, error) { before, err := os.Lstat(socketPath) if errors.Is(err, fs.ErrNotExist) { return false, nil - } else if err != nil { + } + if err != nil { return false, fmt.Errorf("checking xray socket: %w", err) } if before.Mode()&fs.ModeSocket == 0 { @@ -65,7 +66,7 @@ func removeStaleSocket(socketPath string) (bool, error) { func connectionRefused(socketPath string) bool { conn, err := net.DialTimeout("unix", socketPath, dialTimeout) if err == nil { - conn.Close() + _ = conn.Close() return false } return errors.Is(err, syscall.ECONNREFUSED) diff --git a/test/e2e/xray/xray_test.go b/test/e2e/xray/xray_test.go index 8a73982..b10e963 100644 --- a/test/e2e/xray/xray_test.go +++ b/test/e2e/xray/xray_test.go @@ -262,12 +262,12 @@ func assertOffRepeatedly(t *testing.T, user string, among []route) { // removedLine matches the usher log line that reports the removal of a user from an inbound. func removedLine(tag, user string) string { - return fmt.Sprintf(`users changed" inbound=%s .*removed="?\[[^\]]*%s/%s[ \]]`, tag, tag, user) + return fmt.Sprintf(`users changed" interface=%s .*removed="?\[[^\]]*%s/%s[ \]]`, tag, tag, user) } // addedLine matches the usher log line that reports the addition of a user to an inbound. func addedLine(tag, user string) string { - return fmt.Sprintf(`users changed" inbound=%s added="?\[[^\]]*%s/%s[ \]]`, tag, tag, user) + return fmt.Sprintf(`users changed" interface=%s added="?\[[^\]]*%s/%s[ \]]`, tag, tag, user) } // TestXray runs its subtests in order; each one starts from the state the previous one left. From 80211c8a786cf153821ffac53e341804168b1dfe Mon Sep 17 00:00:00 2001 From: EvaTheSalmon <18466855+EvaTheSalmon@users.noreply.github.com> Date: Wed, 14 Oct 2026 21:42:13 +0300 Subject: [PATCH 30/32] docs(repo): fix comment audit findings Co-Authored-By: Claude Sonnet 5.5 --- cmd/usher/env.go | 2 +- deploy/compose.e2e-xray.yml | 3 +-- internal/pass/pass_test.go | 4 ++-- internal/xray/config.go | 2 +- internal/xray/drift.go | 9 +++++---- internal/xray/mirror.go | 3 +-- internal/xray/mirror_test.go | 1 - internal/xray/protocol.go | 3 +-- internal/xray/render.go | 2 +- internal/xray/session_test.go | 1 - internal/xray/socket.go | 2 +- test/e2e/xray/client_test.go | 1 - test/e2e/xray/server_test.go | 4 ++-- test/e2e/xray/xray_test.go | 3 --- 14 files changed, 16 insertions(+), 24 deletions(-) diff --git a/cmd/usher/env.go b/cmd/usher/env.go index 0a3bf82..4ccc2bd 100644 --- a/cmd/usher/env.go +++ b/cmd/usher/env.go @@ -94,7 +94,7 @@ func parseIPv4(name, value string) (netip.Addr, error) { return addr, nil } -// parseAbsPath checks that value is an absolute path; it serves the socket and config paths. +// parseAbsPath requires a non-empty absolute path. func parseAbsPath(name, value string) (string, error) { if value == "" { return "", fmt.Errorf("%s is required", name) diff --git a/deploy/compose.e2e-xray.yml b/deploy/compose.e2e-xray.yml index 62e1b32..e8892ea 100644 --- a/deploy/compose.e2e-xray.yml +++ b/deploy/compose.e2e-xray.yml @@ -19,8 +19,7 @@ services: image: ghcr.io/xtls/xray-core:26.10.10 restart: unless-stopped # With Docker's init as PID 1 the test can SIGKILL the Xray process alone. Killing PID 1 from - # inside is ignored, and "docker kill" marks the container as stopped by hand, which disables - # the restart policy. + # inside is ignored. init: true # Root uid for the ports and the socket; the group equals usher's group. user: "0:${E2E_GID:-0}" diff --git a/internal/pass/pass_test.go b/internal/pass/pass_test.go index 47e0a09..94a74bd 100644 --- a/internal/pass/pass_test.go +++ b/internal/pass/pass_test.go @@ -18,8 +18,8 @@ import ( "github.com/mrcsin/usher/internal/state" ) -// fakeSession is a Session that renders "new " into one file per user and enrolls a -// state entry the first time an interface has users. +// fakeSession is a Session that renders "new " into one file per user and suffix and +// enrolls a state entry the first time an interface has users. type fakeSession struct { names []string suffixes []string diff --git a/internal/xray/config.go b/internal/xray/config.go index 4995c08..dfc03b3 100644 --- a/internal/xray/config.go +++ b/internal/xray/config.go @@ -583,7 +583,7 @@ func validHopPorts(s string) bool { return from >= 1 && from < to && to <= 65535 } -// object returns v as a JSON object, or nil when it is not one. A read from nil is safe. +// object returns v as a JSON object, or nil when it is not one. func object(v any) map[string]any { m, _ := v.(map[string]any) return m diff --git a/internal/xray/drift.go b/internal/xray/drift.go index c31312d..73d2373 100644 --- a/internal/xray/drift.go +++ b/internal/xray/drift.go @@ -53,10 +53,11 @@ func isManagedProxyType(proxyType string) bool { return false } -// checkDrift compares every valid inbound of the file with the inbound of the same tag in the -// running Xray. The result maps a tag to the reason its running inbound differs from the file or -// is missing; a tag without an entry matches. A mismatch means Xray runs an older file, so the -// client files for the tag are not trustworthy until Xray restarts. +// checkDrift compares every inbound of the file that passed structural validation, hint errors +// included, with the inbound of the same tag in the running Xray. The result maps a tag to the +// reason its running inbound differs from the file or is missing; a tag without an entry matches. +// A mismatch means Xray runs an older file, so the client files for the tag are not trustworthy +// until Xray restarts. // // Certificate entries that name a file are left out on both sides: Xray reloads those files // itself, so a changed file is no drift. A reason names the differing parts and never a diff --git a/internal/xray/mirror.go b/internal/xray/mirror.go index 4772888..0aa154c 100644 --- a/internal/xray/mirror.go +++ b/internal/xray/mirror.go @@ -23,8 +23,7 @@ const ( // clientOutbound returns the Xray client outbound for one user on the inbound. It copies only what // the client needs from the server config and rebuilds the TLS and Reality blocks, so nothing -// secret or server-only reaches the result. host is the address clients dial unless x-usher sets -// one. The xhttp tuning is packed into extra, the one form both the .json and the share link carry. +// secret or server-only reaches the result. host is the address clients dial unless x-usher sets one. func clientOutbound(in inbound, host string, entry state.XrayEntry) (map[string]any, error) { stream, err := clientStream(in, entry) if err != nil { diff --git a/internal/xray/mirror_test.go b/internal/xray/mirror_test.go index dfbf9d3..bbc8829 100644 --- a/internal/xray/mirror_test.go +++ b/internal/xray/mirror_test.go @@ -52,7 +52,6 @@ func mirrorOne(t *testing.T, in inbound) map[string]any { return out } -// wantJSON compares got with the JSON document want. func wantJSON(t *testing.T, got map[string]any, want string) { t.Helper() encoded, err := json.Marshal(got) diff --git a/internal/xray/protocol.go b/internal/xray/protocol.go index e907681..2cad5b3 100644 --- a/internal/xray/protocol.go +++ b/internal/xray/protocol.go @@ -23,8 +23,7 @@ type protocolSpec struct { usesID bool // clientSettings adds the credential and the protocol fields of the client outbound settings. clientSettings func(in inbound, e state.XrayEntry, settings map[string]any) - // account builds the account Xray stores for the entry on the inbound. - account func(in inbound, e state.XrayEntry) proto.Message + account func(in inbound, e state.XrayEntry) proto.Message // credential returns the part of an account that identifies what the client must present, or // "" for an account of another type. Two accounts with the same credential are the same user. credential func(account proto.Message) string diff --git a/internal/xray/render.go b/internal/xray/render.go index 248af47..4cc4ddc 100644 --- a/internal/xray/render.go +++ b/internal/xray/render.go @@ -68,7 +68,7 @@ func renderLink(clientJSON []byte) ([]byte, error) { } // renderFiles returns the client files of one user on the inbound, keyed by clients.Path: always -// the .json, and the .txt unless the inbound cannot have a link. A link libxray refuses is a +// the .json, and the .txt unless no client reads its link. A link libxray refuses is a // defect of usher, which builds only shapes libxray knows; it is logged and the .json stays. func renderFiles(log *slog.Logger, in inbound, host, user string, entry state.XrayEntry) (map[string][]byte, error) { outbound, err := clientOutbound(in, host, entry) diff --git a/internal/xray/session_test.go b/internal/xray/session_test.go index b9a542a..22b46b7 100644 --- a/internal/xray/session_test.go +++ b/internal/xray/session_test.go @@ -94,7 +94,6 @@ func (f *userService) AlterInbound(_ context.Context, req *command.AlterInboundR return &command.AlterInboundResponse{}, nil } -// credentials maps the emails of the tag to their credentials. func (f *userService) credentials(t *testing.T, in inbound) map[string]string { t.Helper() out := map[string]string{} diff --git a/internal/xray/socket.go b/internal/xray/socket.go index 19348e2..993a306 100644 --- a/internal/xray/socket.go +++ b/internal/xray/socket.go @@ -41,7 +41,7 @@ func removeStaleSocket(socketPath string) (bool, error) { case err != nil: return false, fmt.Errorf("opening xray socket lock: %w", err) default: - defer lock.Close() // closing the file releases the flock + defer lock.Close() if err := syscall.Flock(int(lock.Fd()), syscall.LOCK_EX|syscall.LOCK_NB); err != nil { if errors.Is(err, syscall.EWOULDBLOCK) { return false, nil diff --git a/test/e2e/xray/client_test.go b/test/e2e/xray/client_test.go index 1a57382..d3daee3 100644 --- a/test/e2e/xray/client_test.go +++ b/test/e2e/xray/client_test.go @@ -57,7 +57,6 @@ func routes() []route { return all } -// routesOf returns the routes of one inbound. func routesOf(tag string) []route { var among []route for _, r := range routes() { diff --git a/test/e2e/xray/server_test.go b/test/e2e/xray/server_test.go index bdde970..0db157f 100644 --- a/test/e2e/xray/server_test.go +++ b/test/e2e/xray/server_test.go @@ -21,7 +21,7 @@ const ( tagWS = "vless-ws" ) -// tags lists the inbounds in the order the suite connects to them, Hysteria2 first. +// tags lists the inbounds. The order fixes the proxy port of each route. var tags = []string{tagHysteria, tagReality, tagXHTTP, tagWS} const ( @@ -33,7 +33,7 @@ const ( driftShortID = "cd34" ) -// Where the server config and the files it names sit, relative to the config directory that +// Where the files the server config names sit, relative to the config directory that // Xray mounts at /etc/xray. const ( wsCertFile = "certs/ws.pem" diff --git a/test/e2e/xray/xray_test.go b/test/e2e/xray/xray_test.go index b10e963..c144bf7 100644 --- a/test/e2e/xray/xray_test.go +++ b/test/e2e/xray/xray_test.go @@ -30,7 +30,6 @@ const ( changeTimeout = 10 * time.Second connectTimeout = 40 * time.Second driftTimeout = 60 * time.Second - // restartTimeout and killTimeout are the recovery bounds of the acceptance evidence. restartTimeout = 45 * time.Second killTimeout = 90 * time.Second // refillTick is a little longer than the 30 s refill interval. @@ -60,7 +59,6 @@ var ( srv *server ) -// path returns a path under the scratch root. func (composeProject) path(elem ...string) string { return filepath.Join(append([]string{repoDir, scratchRoot}, elem...)...) } @@ -198,7 +196,6 @@ func markLogs(t *testing.T) logMark { return logMark(len(usherLogs(t))) } -// logsSince returns the usher log after the mark. func logsSince(t *testing.T, mark logMark) string { t.Helper() logs := usherLogs(t) From 7561eb767405da77e5a7b5bfce465c1475ac4bfb Mon Sep 17 00:00:00 2001 From: EvaTheSalmon <18466855+EvaTheSalmon@users.noreply.github.com> Date: Wed, 14 Oct 2026 21:52:46 +0300 Subject: [PATCH 31/32] docs(plans): record the xray exec run Co-Authored-By: Claude Opus 5.5 (1M context) --- docs/plans/20261014-xray-config-backend.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/docs/plans/20261014-xray-config-backend.md b/docs/plans/20261014-xray-config-backend.md index 9e622ab..6ef3b89 100644 --- a/docs/plans/20261014-xray-config-backend.md +++ b/docs/plans/20261014-xray-config-backend.md @@ -520,3 +520,6 @@ reported by awg-grpc" and nothing reaches Xray. **Release** - bump the image tag in `deploy/compose.example.yml`; the notes state the new required variables, `users.json` version 2, the `users.json.v1` backup and the Xray image 26.10.10 + +**Executed by exec:** +- branch: xray-config From e0f24fb219036ef9208224185db073fdf2fd2fee Mon Sep 17 00:00:00 2001 From: EvaTheSalmon <18466855+EvaTheSalmon@users.noreply.github.com> Date: Wed, 14 Oct 2026 22:08:11 +0300 Subject: [PATCH 32/32] docs(plans): complete the xray config backend plan Co-Authored-By: Claude Opus 5.5 (1M context) --- docs/plans/{ => completed}/20261014-xray-config-backend.md | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename docs/plans/{ => completed}/20261014-xray-config-backend.md (100%) diff --git a/docs/plans/20261014-xray-config-backend.md b/docs/plans/completed/20261014-xray-config-backend.md similarity index 100% rename from docs/plans/20261014-xray-config-backend.md rename to docs/plans/completed/20261014-xray-config-backend.md