From 919647c8f0b8fbc406f9f7324afce5fb651f291c Mon Sep 17 00:00:00 2001 From: Konstantins Feofantovs Date: Mon, 10 Aug 2026 19:31:12 +0300 Subject: [PATCH] =?UTF-8?q?docs:=20ADR=200010=20=E2=80=94=20node=20?= =?UTF-8?q?=E2=86=92=20controller=20inventory=20channel=20(projected-token?= =?UTF-8?q?=20auth,=20local=20JWKS)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Records the decision for delivering the node role's Go-binary findings to the controller: node-initiated HTTP push of already-filtered facts, authenticated with a projected controller-audience ServiceAccount token and validated locally against the cluster JWKS (no TokenReview, so no new RBAC verb). The node keeps zero RBAC and cannot reach the API server (wrong audience and no grant); nothing is persisted on the node. Plain HTTP is chosen with the confidentiality tradeoff stated and the blast radius bounded (ADR 0008). The controller joins facts against PodWatcher into a per-(namespace, workload, container) inventory in memory, preserving the loss-harmless property (ADR 0003). Realizes the promise in ADR 0008 §4 and extends ADR 0009's log-only scope. Co-Authored-By: Claude Opus 4.8 (1M context) --- ...10-node-to-controller-inventory-channel.md | 160 ++++++++++++++++++ docs/adr/README.md | 1 + 2 files changed, 161 insertions(+) create mode 100644 docs/adr/0010-node-to-controller-inventory-channel.md diff --git a/docs/adr/0010-node-to-controller-inventory-channel.md b/docs/adr/0010-node-to-controller-inventory-channel.md new file mode 100644 index 0000000..1e6fcaf --- /dev/null +++ b/docs/adr/0010-node-to-controller-inventory-channel.md @@ -0,0 +1,160 @@ +# 0010. Node → controller inventory channel: node-initiated HTTP, projected-token auth, local JWKS validation + +Date: 2026-08-10 + +Status: Accepted (realizes the promise in 0008 §4 point 4; extends 0009, whose +node role was log-only. The one-way boundary of 0001 and the identity model of +0005/0008 are unchanged.) + +## Context + +ADR 0009 gave the node role a Go-binary scanner and deliberately stopped at the +node's own log: "Delivery to the controller for aggregation is a later slice." +This is that slice. The scanner already produces exactly what the controller +cannot learn from the Kubernetes API — Go version, module path, PGO flag, tied +to a pod UID and container ID — and the controller already holds the workload +inventory (PodWatcher: namespace, owner chain, image digest) those facts need +to be joined against. Nothing is missing except a wire between them. + +Three constraints from prior decisions bound the design, and none of them may +bend: + +- **One-way flow (0001).** The agent initiates every connection and nothing a + peer returns changes agent behavior. This governs the agent↔backend edge; + the same discipline must hold one level down, on the node↔controller edge. +- **The node holds no Kubernetes identity (0009 §4).** Its ServiceAccount is + bound to nothing and `automountServiceAccountToken: false` keeps the default + API token out of the container. A node component that could reach the API + server would widen the audited surface across every node. +- **Nothing durable lives on the node (0008 §4).** ADR 0008 already named the + mechanism this slice must use: "Node pods authenticate to the controller with + projected ServiceAccount tokens (audience-bound, kubelet-rotated, nothing + stored on the node)." The controller stays the only identity holder and the + only egress point. + +Two forces pull against a naive "just call the API" answer. Authenticating the +node to the controller with a Kubernetes-issued token is the obvious fit — the +kubelet already mints and rotates ServiceAccount tokens — but the usual way to +*verify* such a token, `TokenReview`, is a `create` verb on +`authentication.k8s.io/tokenreviews`. Granting it to the controller would add a +write verb to a product whose single write grant (0008) was justified at +length. The token can instead be verified locally: a projected token is a +signed JWT, and the cluster publishes the public keys to verify it at its OIDC +JWKS endpoint. Local validation needs no `TokenReview` and therefore no new +RBAC verb. + +## Decision + +1. **The node initiates; the controller receives.** The node role POSTs its + scan result — the kept (non-infrastructure) binaries and the aggregate + counters, already filtered on the node (0009 §5) — to the controller over + plain HTTP on the cluster network, addressed by the controller's in-cluster + Service DNS. The controller runs an HTTP receiver for this and only this. + The receiver's response carries an acknowledgment or an error and nothing + else: it never returns configuration, tasks, or anything the node acts on. + The node's behavior is fixed by its own flags and ConfigMap, exactly as the + agent's is fixed against the backend (0001, backend-requirements §1). This + is the one-way rule applied one level down — data flows node → controller → + backend, and no peer's reply is a control channel. + +2. **Authentication is a projected ServiceAccount token, audience-bound to the + controller.** The DaemonSet mounts a `serviceAccountToken` projected volume + with `audience` set to the controller's configured identifier and a short + expiration; the kubelet issues and rotates it on a `tmpfs`-backed path. The + node reads the token from that file on each send, so rotation needs no + restart. This is the *only* credential the node holds, it is not persisted, + and it is not the default API token. + +3. **The token is not an API credential, so the node still cannot reach the API + server.** An audience-bound token authenticates only to that audience: the + API server rejects a token whose audience is the controller, and the node's + ServiceAccount remains bound to zero RBAC regardless. `automountServiceAccountToken` + stays `false`. The node gaining a controller-audience token does not give it + a Kubernetes identity — 0009's "no API access" property is intact, now by two + independent barriers (wrong audience *and* no grant), not one. + +4. **The controller validates the token locally against the cluster JWKS — no + `TokenReview`.** On startup the controller resolves the cluster's OIDC + discovery document (`/.well-known/openid-configuration`) to its `jwks_uri` + and fetches the signing keys, using its existing in-cluster REST client. It + then verifies each incoming token itself: signature against the JWKS + (RS256/ES256), `exp`/`nbf`, that the audience contains the controller's + configured identifier, and that the subject is the node role's expected + ServiceAccount. This adds **no resource RBAC verb** — in particular no + `create` on `tokenreviews`. Reading the discovery/JWKS endpoints is a + read-only `nonResourceURL` GET, granted cluster-wide to ServiceAccounts by + the built-in `system:service-account-issuer-discovery` role on typical + clusters; where a chart must grant it explicitly it is a read, consistent + with the read-only principle, never a write. + +5. **The controller joins node facts against its own inventory.** Each fact + carries a pod UID and container ID (parsed from the process cgroup on the + node, 0009 §2). The controller matches them against PodWatcher — pod UID → + namespace and workload, container ID → container name and the image digest + PodWatcher already collects — and forms one Go-inventory record per + (namespace, workload, container): Go version, module path, image digest, + PGO flag. Records dedup across the many replicas and nodes that run the same + build. A fact whose pod UID or container ID is not (yet) in the inventory — + informer lag, or a pod outside the controller's filters — is counted as + unjoined and dropped, never guessed. The join state is in memory only: it is + reconstructed from the next node scan and the live cluster, so it adds no + persistent state and 0003's loss-harmless property is unaffected. + +## Transport confidentiality — the honest tradeoff + +The channel is plain HTTP with a bearer token, not TLS. The consequences, +stated rather than hidden: + +- The token and the facts travel in cleartext on the cluster network. A party + that can already sniff pod-to-pod traffic can read them and can replay the + token to the controller until it expires. +- What that buys an attacker is bounded to the same blast radius as a stolen + identity Secret (0008): they can POST fabricated Go-inventory facts for this + one cluster — data poisoning of a medium-sensitivity metadata stream. The + channel is one-way (the controller returns nothing to act on), carries no + monetary fields (0004), and grants no read or control capability. The facts + themselves were already filtered on the node, so nothing sensitive that + wasn't destined to leave the cluster is exposed. +- Compensating controls: the audience-bound token is short-lived and rotated; + a NetworkPolicy shipped with the chart restricts who may reach the receiver + port to the node DaemonSet; the subject check rejects tokens that are not the + node role's ServiceAccount. +- If a deployment needs wire confidentiality, the controller can later serve + this endpoint over TLS with the node pinning or skip-verifying (the node + holds no CA, 0008) — an additive change to this ADR, documented there, not a + silent one. Plain HTTP is the starting point because the data is + already-filtered metadata and the token is the authentication, not the + transport. + +## Consequences + +Easier: + +- The Go-version and module inventory reaches the controller and becomes a + shippable payload, closing the loop 0009 opened, at no new privilege: the + node keeps zero RBAC and no external egress, the controller adds no write + verb and no `TokenReview`. +- Verifying identity locally means the receiver has no API-server round-trip on + the hot path and no dependency on `TokenReview` availability. +- The mechanism is exactly the one 0008 promised, so no earlier decision has to + be reopened; this ADR realizes a commitment rather than changing course. + +Harder, or given up: + +- The controller now listens on a port. It is an ingress *from inside the + cluster only* — never from the backend or the internet — and its contract is + "accept a data push, answer ack/error." Keeping it that and only that is a + standing obligation; a receiver that ever influenced node behavior would + break 0001. +- Plain HTTP exposes the token and facts on the cluster network (see the + tradeoff above). This is accepted deliberately, with the blast radius bounded + and the upgrade path named. +- The controller depends on the cluster's OIDC discovery/JWKS endpoints being + reachable and on projected-token issuance being enabled (the + `ServiceAccountTokenProjection` path, on by default for years). A cluster + that disabled either cannot run this channel; the controller reports that + condition and degrades to not accepting node facts, rather than failing + closed on all collection. +- One dependency enters go.mod for JWT verification (`golang-jwt/jwt/v5`, + pure-Go, no transitive dependencies). JWKS fetch and JWK→key conversion stay + first-party and audited here. diff --git a/docs/adr/README.md b/docs/adr/README.md index 3fd12b6..895e5b6 100644 --- a/docs/adr/README.md +++ b/docs/adr/README.md @@ -46,3 +46,4 @@ stays in place. - [0007. Durability is optional: snapshot delivery bounds loss, disk is a knob](0007-optional-durability.md) - [0008. Agent identity lives in a namespaced Secret — the product's one write grant](0008-identity-in-secret.md) - [0009. The node role: an on-node Go-binary scanner with zero API access](0009-node-role-binary-scanner.md) +- [0010. Node → controller inventory channel: node-initiated HTTP, projected-token auth, local JWKS validation](0010-node-to-controller-inventory-channel.md)