feat(xray)!: add xray backend driver

.github/workflows/ci.yml

@@ -30,7 +30,9 @@ jobs:
       - uses: actions/setup-go@40f1582b2485089dde7abd97c1529aa768e1baff # v5.6.0
         with:
           go-version-file: go.mod
-          cache-dependency-path: go.sum
+          cache-dependency-path: |
+            go.sum
+            tools/go.sum
 
       - name: format chain
         run: scripts/format.sh
@@ -53,10 +55,15 @@ jobs:
       - uses: actions/setup-go@40f1582b2485089dde7abd97c1529aa768e1baff # v5.6.0
         with:
           go-version-file: go.mod
-          cache-dependency-path: go.sum
+          cache-dependency-path: |
+            go.sum
+            tools/go.sum
 
       - name: load module
         run: scripts/ci/module.sh
 
       - name: e2e test
         run: go test -tags e2e -count=1 -v ./test/e2e/
+
+      - name: xray e2e test
+        run: go test -tags e2e -count=1 -v ./test/e2e/xray/

.gitignore

@@ -5,3 +5,4 @@ Thumbs.db
 .idea/
 .vs/
 .e2e/
+.e2e-xray/

AGENTS.md

@@ -1,9 +1,10 @@
 # usher
 
-Container that reads `config/usher.yml`, a map of user name to AmneziaWG interface names, and keeps
-the peers of those interfaces and the client configs in `clients/` in step with it through the
-`awg-grpc` unix socket; `README.md` describes it. Go module `github.com/mrcsin/usher`; entry point
-`cmd/usher`. All commands run from the repository root.
+Container that reads `config/usher.yml`, a map of user name to interface names, and keeps the
+peers of AmneziaWG interfaces, the users of Xray VLESS inbounds and the client files in `clients/`
+in step with it through the `awg-grpc` and Xray API unix sockets; `README.md` describes it. Go
+module `github.com/mrcsin/usher`; entry point `cmd/usher`. All commands run from the repository
+root.
 
 ## Build
 
@@ -24,7 +25,9 @@ go test -race ./...
 
 Table-driven tests sit beside the source as `*_test.go`; filesystem tests use `t.TempDir()`. The
 `awg-grpc` API is faked by a struct with func fields that implements
-`awgv1.ManagementServiceClient`.
+`awgv1.ManagementServiceClient`. The Xray API is faked by `fakeXray`, an in-memory
+`command.HandlerServiceClient` that applies the user operations it gets and fails on flags such
+as `failAdd`; it embeds the nil interface, so a call it does not implement panics.
 
 The e2e suite in `test/e2e` carries `//go:build e2e`. It needs Docker and a host with the
 `amneziawg` module loaded, so it runs on a Linux host, not on macOS. It uses the published
@@ -36,25 +39,51 @@ GOOS=linux GOARCH=amd64 go test -c -tags e2e -o e2e.test ./test/e2e
 ./e2e.test -test.count=1 -test.v
 ```
 
-The suite needs `docker compose` without `sudo`. It wipes and recreates the bind-mount directories
-under `.e2e/` (gitignored) at start, and passes `E2E_UID` and `E2E_GID` to compose so usher
-writes them as the test user.
+The Xray suite in `test/e2e/xray` carries the same tag and needs Docker only, no kernel module. It
+has its own `TestMain`, compose project
+(`deploy/compose.e2e-xray.yml`, network `172.30.98.0/24`) and scratch root `.e2e-xray/`
+(gitignored). Both suites take the compose lifecycle and the polling helpers from
+`test/e2e/e2eutil`. It starts the official Xray image with a self-signed nginx as the Reality target,
+builds usher from the tree and drives two users through client containers built from the rendered
+links. The server config allows the target address in the freedom outbound `finalRules`, because
+Xray 26.x blocks private destinations for VLESS by default. Run it with `-count=1`:
+
+```sh
+go test -tags e2e -count=1 -v ./test/e2e/xray/
+```
+
+Both suites need `docker compose` without `sudo`. Each wipes and recreates its own bind-mount root
+at start (`.e2e/`, `.e2e-xray/`, both gitignored) and passes `E2E_UID` and `E2E_GID` to compose so
+usher writes them as the test user.
 
 To copy the tree from macOS to a Linux host, use `COPYFILE_DISABLE=1 tar --no-xattrs`: plain macOS
 tar adds `._*` AppleDouble files. A run killed by a signal leaves containers up; remove them with:
 
 ```sh
 docker compose -f deploy/compose.e2e.yml --profile handshake down -v --remove-orphans
+docker compose -f deploy/compose.e2e-xray.yml --profile client down -v --remove-orphans
+```
+
+## Generate
+
+`gen/xray` comes from `proto/xray`; edit the protos only to refresh them (see
+`proto/xray/README.md`). Regenerate with:
+
+```sh
+go tool -modfile=tools/go.mod buf generate
 ```
 
+`scripts/format.sh` fails when `gen/xray` differs from the output of that command.
+
 ## Format
 
 ```sh
 scripts/format.sh
 ```
 
-Run before presenting changes. It checks gofmt and runs `go vet` without tags and with the `e2e`
-tag. `gofmt -w .` fixes formatting. The pre-commit hook and the CI `check` job run the same script.
+Run before presenting changes. It checks gofmt, runs `go vet` without tags and with the `e2e`
+tag, and runs the `gen/xray` check (see Generate). `gofmt -w .` fixes formatting. The pre-commit
+hook and the CI `check` job run the same script.
 
 After a change to a shell script or a workflow, also run:
 
@@ -70,7 +99,8 @@ Enable the pre-commit hook per clone: `git config core.hooksPath scripts/git-hoo
 `.github/workflows/ci.yml` runs on every pull request and every push to `master`.
 
 - `check`: `scripts/format.sh`, `go test -race`, shellcheck.
-- `e2e`: loads the pinned kernel module with `scripts/ci/module.sh`, then runs the e2e suite.
+- `e2e`: loads the pinned kernel module with `scripts/ci/module.sh`, then runs the e2e suite and
+  the Xray e2e suite.
 
 The module pin is `AWG_MODULE_REF` in `ci.yml`.
 
@@ -80,10 +110,12 @@ the notes of the annotated tag.
 
 ## Conventions
 
-- A private key, a preshared key or a `client_params` value is never logged, and an error message
-  never quotes one. A public key without an entry is logged as base64.
-- `state/users.json` is the only source of secrets. The kernel and `clients/` derive from it, and
-  usher saves it before `ApplyPeers`.
+- A private key, a preshared key, a `client_params` value, an Xray UUID or a Reality private key is
+  never logged, and an error message never quotes one. A public key without an entry is logged as
+  base64.
+- `state/users.json` is the only source of secrets. The kernel, Xray and `clients/` derive from it,
+  and usher saves it before the first `Apply` of any backend (`ApplyPeers` for awg,
+  `AlterInbound` for Xray).
 - A failure of one interface never stops the daemon; the pass logs it and goes on.
 - No interface without a second implementation, no exported surface only for tests.
 
@@ -94,18 +126,32 @@ cmd/usher/             entry point: environment, signals, run loop wiring
 internal/config/       usher.yml loader and validation
 internal/state/        users.json load and save
 internal/atomicfile/   atomic file replace shared by state and the clients mirror
-internal/pass/         settings, enrollment, address allocation, the reconcile pass
-internal/clients/      client config rendering and the clients/ mirror
+internal/pass/         settings, the Session and Backend seam, the reconcile pass
+internal/awg/          AmneziaWG backend: enrollment, address allocation, config rendering, apply
+internal/xray/         Xray backend: decoder tables, enrollment, user reconcile, vless:// links
+internal/clients/      the clients/ mirror
 internal/watch/        file poll, refill ticker and the trigger loop
-deploy/                example compose file, e2e compose file and interface config
-test/e2e/              end-to-end test through docker compose (tag e2e)
-scripts/format.sh      the Format chain (see Format)
+deploy/                example compose file and Xray config (deploy/xray), e2e compose files
+                       of both suites, e2e fixtures (deploy/e2e)
+proto/xray/            vendored Xray protos (see proto/xray/README.md)
+gen/xray/              Go code generated from proto/xray, committed
+tools/                 separate Go module that pins buf and the protobuf plugins
+test/e2e/              end-to-end test of AmneziaWG through docker compose (tag e2e)
+test/e2e/xray/         end-to-end test of Xray through docker compose (tag e2e)
+scripts/format.sh      the Format chain (see Format), including the gen/xray check
 scripts/git-hooks/     pre-commit hook: branch guard, then scripts/format.sh
 scripts/ci/module.sh   builds and loads the pinned kernel module in the CI e2e job
 .github/workflows/     ci.yml and release.yml
 Dockerfile             Go build stage and the runtime image
 ```
 
+## Adding an Xray inbound shape
+
+Add one entry to `protocols`, `transports` or `securities` in `internal/xray`, vendor the proto
+that describes its settings into `proto/xray`, run the generate command and write the entry's
+tests. A protocol whose credential is not a UUID also needs a field in `state.XrayEntry` and its
+own enrollment. `docs/architecture/overview.md`, "The Xray backend", has the details.
+
 ## Docs
 
 Stable design is in `docs/architecture/`, starting at `overview.md`. Dated plans are in

Dockerfile

@@ -6,6 +6,7 @@ WORKDIR /src
 COPY go.mod go.sum ./
 RUN go mod download
 COPY cmd ./cmd
+COPY gen ./gen
 COPY internal ./internal
 ARG VERSION=dev
 RUN CGO_ENABLED=0 go build -trimpath -ldflags "-s -w -X main.version=${VERSION}" \

README.md

@@ -8,49 +8,109 @@
 </div>
 
 User management for the VPN services of one node. You list users and the interfaces each may use
-in one file; within seconds usher creates their keys, puts them into the running services and
-writes a ready client config per user and interface. Addresses and keys stay internal: you type
-names only.
+in one file; within seconds usher creates their credentials, puts them into the running services
+and writes a ready client config or link per user and interface. Addresses, keys and UUIDs stay
+internal: you type names only.
 
 usher holds no capabilities and is never in the packet path. Every VPN service runs in its own
 container with a small management API on a unix socket; usher calls that API and nothing else, so
-a crash or an upgrade of usher leaves every session running. AmneziaWG is supported through
-[awg-grpc](https://github.com/mrcsin/awg-grpc); Xray is next.
+a crash or an upgrade of usher leaves every session running. usher supports two services:
+AmneziaWG through [awg-grpc](https://github.com/mrcsin/awg-grpc), and Xray through the API of Xray
+itself.
 
 ## Users
 
 `config/usher.yml`:
 
 ```yaml
-alice-phone: [awg0]
+alice-phone: [awg0, vless-reality]
 alice-laptop: [awg0, awg1]
 bob: []
 ```
 
-- One name per device: a WireGuard key cannot serve two devices at once.
-- Removing a name, or setting it to `[]`, switches the user off. The keys stay, and putting the
-  name back restores the same config.
+- An interface name is an AmneziaWG interface or the tag of an Xray inbound. A name that both
+  services report fails in both, so keep the names apart.
+- One name per device on AmneziaWG: a WireGuard key cannot serve two devices at once.
+- Removing a name, or setting it to `[]`, switches the user off. The credentials stay, and putting
+  the name back restores the same config or link.
 - A broken file changes nothing: usher keeps the last valid one and logs the error with its line.
 - A service that restarts empty is refilled within 30 s without any change to the file.
 
-Client configs appear in `clients/<user>/<interface>.conf`. Secrets live in `state/users.json`,
-the only copy of every key: keep the state volume in the backup set.
+Client files appear in `clients/<user>/<interface>.conf` for AmneziaWG and in
+`clients/<user>/<tag>.txt` for Xray. An Xray file holds one `vless://` link. Secrets live in
+`state/users.json`, the only copy of every key and UUID: keep the state volume in the backup set.
+`users.json` has format version 2. Once this version has saved it, the previous usher image
+cannot read it, so a rollback to that image needs a copy of `users.json` from before the upgrade.
 
-## Running
+### Switching off
 
-[`deploy/compose.example.yml`](deploy/compose.example.yml) runs usher next to awg-grpc. Images are
-`ghcr.io/mrcsin/usher:<tag>`, one per release.
+On AmneziaWG, removing a user removes the peer from the interface. On Xray, removing a user
+removes it from the inbound: new connections fail, but connections that are already open stay
+until the client closes them or Xray ends them after its `connIdle` timeout.
 
-| Variable     | Example           | Meaning                                         |
-| ------------ | ----------------- | ----------------------------------------------- |
-| `USHER_HOST` | `203.0.113.10`    | IPv4 address clients connect to; no host names  |
-| `USHER_DNS`  | `1.1.1.1,1.0.0.1` | DNS servers written into client configs         |
+## Running
 
-Both are required. Publish each interface port 1:1, and give usher the socket group of the
-service containers.
+[`deploy/compose.example.yml`](deploy/compose.example.yml) runs usher next to awg-grpc and Xray.
+Images are `ghcr.io/mrcsin/usher:<tag>`, one per release.
+
+| Variable            | Example                  | Meaning                                        |
+| ------------------- | ------------------------ | ---------------------------------------------- |
+| `USHER_HOST`        | `203.0.113.10`           | IPv4 address clients connect to; no host names |
+| `USHER_DNS`         | `1.1.1.1,1.0.0.1`        | DNS servers written into AmneziaWG configs     |
+| `USHER_AWG_SOCKET`  | `/run/awg-grpc/awg.sock` | absolute path of the awg-grpc socket           |
+| `USHER_XRAY_SOCKET` | `/run/xray/api.sock`     | absolute path of the Xray API socket           |
+
+`USHER_HOST` is required. Set `USHER_AWG_SOCKET`, `USHER_XRAY_SOCKET` or both; a variable that is
+set turns its service on, and usher exits at start when neither is set. `USHER_DNS` is required
+with `USHER_AWG_SOCKET` and is not read without it. A deployment from before the Xray driver must
+add `USHER_AWG_SOCKET` to its compose file.
+
+Unsetting a variable turns its service off: usher removes that service's client files and logs its
+interface names as unknown. The entries in `users.json` stay.
+
+Publish each interface port 1:1, and give usher the socket group of the service containers.
+
+If one service is down during a pass, usher logs it, applies the other service and keeps the
+client files of the service that is down.
+
+### Xray
+
+usher creates no inbound and starts no Xray process. It reads the VLESS inbounds of a running
+Xray and manages their users. Prepare the Xray container as follows:
+
+- Use Xray v24.11.5 or later, which has `GetInboundUsers`.
+- Set `api.listen` to a unix socket path with the mode suffix, for example
+  `/run/xray/api.sock,0660`. Xray gives the socket its own uid and gid.
+- Run Xray with the group of usher (`user: "0:1000"` in the example), so usher can open the
+  socket.
+- List only `HandlerService` in `api.services`.
+- Start every VLESS inbound with `"clients": []`; an inbound with startup clients fails. usher adds
+  the users.
+- Name each inbound tag so that it fits the interface name pattern: up to 15 characters from
+  letters, digits and `_=+.-`, and the first one a letter, a digit or `_`.
+- After a hard kill, remove the stale socket file before you start Xray again. The official image
+  has no shell, so remove it through the socket volume.
+
+usher supports one inbound shape: VLESS with `decryption: none` over raw TCP without a header,
+with Reality security. usher gives every user the flow `xtls-rprx-vision`. An inbound with another
+security, transport, header, finalmask or decryption, with startup clients, or with a port range,
+fails with an error that names the field; the other inbounds keep working. An inbound with
+another protocol, such as the `api` tunnel, or without a tag is left alone. An unreferenced VLESS
+inbound gets an empty user list.
+
+[`deploy/xray/config.json`](deploy/xray/config.json) is an example inbound. Replace its Reality
+private key and short ID before use; `xray x25519` prints a key pair.
+
+The link holds the public key derived from the inbound's private key, the first non-empty server
+name, the first short ID with trailing zero bytes removed, and the fingerprint `chrome`. usher
+sends no ML-DSA-65 verification key, so the link has no `pqv` parameter.
+
+Xray 26.x blocks private destinations for VLESS by default. If your clients must reach a private
+address through the node, allow it in the `finalRules` of the freedom outbound, as the e2e server
+config in `deploy/e2e/xray/server.json` does for its target.
 
 To issue new keys for a user, stop usher, delete its `<interface>/<user>` entry from
-`users.json` and start it again.
+the `awg` or `xray` group of `users.json` and start it again.
 
 ## License
 

buf.gen.yaml

@@ -0,0 +1,13 @@
+version: v2
+managed:
+  enabled: true
+  override:
+    - file_option: go_package_prefix
+      value: github.com/mrcsin/usher/gen/xray
+plugins:
+  - local: ["go", "tool", "-modfile=tools/go.mod", "protoc-gen-go"]
+    out: gen/xray
+    opt: paths=source_relative
+  - local: ["go", "tool", "-modfile=tools/go.mod", "protoc-gen-go-grpc"]
+    out: gen/xray
+    opt: paths=source_relative

buf.yaml

@@ -0,0 +1,3 @@
+version: v2
+modules:
+  - path: proto/xray