Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -94,7 +94,9 @@ jobs:
go-version-file: go.mod
- uses: golangci/golangci-lint-action@v9
with:
version: v2.11.4
# v2.13.2 rather than v2.11.4: a linter built with Go 1.26 refuses a
# module targeting 1.27 outright, config unread (ADR 0086).
version: v2.13.2
# No analysis cache. On 2026-09-16 a restored one reported six
# SA5011 findings in files the branch did not touch, twice, on a
# tree byte-identical to a passing main; a cold run found none.
Expand Down
2 changes: 1 addition & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Agent image. One binary, two roles (ADR 0009): the same image runs the
# controller (a Deployment — ADR 0026) and the node scanner (a DaemonSet); the
# role is the first argument.
FROM golang:1.26.7@sha256:e30143be198ab04cf7ba25fba83ab3a692ca584c994aad0bf131fa0eb32dd8c1 AS build
FROM golang:1.27.1@sha256:f44f6e88636cfb311f9ebace870ded69d943f227bb3cb27d32ffd84ea18c43ea AS build
WORKDIR /src

# Cache module downloads separately from the source.
Expand Down
2 changes: 2 additions & 0 deletions docs/adr/0081-a-weakened-default-is-a-build-fact.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@ Date: 2026-09-16

Status: Accepted

Amended by: 0086

Amends: 0050 §1

Adds twelve allow-listed GODEBUG names to `go_build`, on a second axis from the
Expand Down
119 changes: 119 additions & 0 deletions docs/adr/0086-a-toolchain-move-reviews-what-it-reports.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,119 @@
# 0086. A toolchain move is a review of what the toolchain reports

Date: 2026-09-17

Status: Accepted

Amends: 0081 §1

Moves the toolchain floor from Go 1.26.7 to 1.27.1 in `go.mod` and in the build
image together, adds `fips140ems` to the GODEBUG allow-list, and keeps four
names Go 1.27 removed. No payload shape changes, no RBAC, no chart change.

## Context

ADR 0085 gave Dependabot the Docker ecosystem, and its first pull request
proposed Go 1.26.7 → 1.27.1 — a minor release, not a patch. The guard held: one
check failed, `go.mod` and the build image disagreed, and ADR 0038 §3 had already
said why that must not merge. CI's `setup-go` reads `go-version-file: go.mod`, so
the vulnerability gate would have been proving something about a standard library
the released image does not contain.

Moving the floor by hand is one line in two files. What makes it a slice is what
sits under those lines: this agent reports GODEBUG defaults out of other
people's binaries (ADR 0050, ADR 0081), and a minor release is exactly when that
vocabulary changes. Measured on go1.27.1 against go1.26.7, it changed in both
directions.

**Four names are gone.** Go 1.27 removed `tls10server`, `tls3des`, `tlsrsakex`
and `tlsunsafeekm` — the toolchain's `internal/godebugs` now carries a `Removed`
list, and using an old value is a build error rather than a weakened build. Four
of the fourteen names ADR 0081 chose are among them.

**One name is new, and it is the kind this list is for.** `fips140ems`, added in
1.27 and backported to 1.26.6, has no version-dependent default: it reaches a
binary only because someone wrote a `//go:debug` line. What that line turns off
is stated in the toolchain's own `doc/godebug.md` — the enforcement of extended
master secret in FIPS 140-3 mode.

## Decision

**1. `fips140ems` goes on the allow-list.** It passes both of ADR 0081's tests.
Its one-line justification is written from the standard library's own
documentation, which is §4's bar. And it is a name only a directive can set,
which §1 called the stronger signal: a build that lags says someone did not
upgrade, a build carrying this says someone decided.

Measured rather than assumed, as ADR 0081's names were: a `go 1.27` module
carrying `//go:debug fips140ems=0` emits `DefaultGODEBUG=execerrdot=1,fips140ems=0`
under go1.27.1.

Two other names arrived with `Changed: 27` and neither goes on the list.
`tracebacklabels` governs what a traceback prints. `x509sslcertoverrideplatform`
decides whether `SSL_CERT_FILE` and `SSL_CERT_DIR` override the platform root
store — but only on Windows and Darwin, and its *old* value is the stricter one,
so its presence in a Linux container's build would mean nothing this agent could
report honestly.

**2. The four removed names stay.** They are removed from the toolchain, not
from the world. This list describes binaries in a customer's cluster, and those
were built by every toolchain rather than by ours: an image built last year with
Go 1.25 still carries `tls10server=1`, and that is still the fact worth
reporting. Deleting them would blind the agent to exactly the old builds the
list exists to find.

What changes is only what a *new* build can say: a binary built by Go 1.27 is
silent about all four, and silence there now means the knob does not exist
rather than that the default was kept. Absence has always meant "the toolchain's
own default" (ADR 0050 §2), and this does not change that reading — a default
that can no longer be overridden is still the default.

**3. The measurements are fixtures, and the old one stays.** The `go 1.21`
default set measured under go1.26.7 remains a test input: it is what a binary in
a cluster looks like, and most of them were not built this week. The set measured
under go1.27.1 is added beside it, which is what pins the two claims above — that
the four removed names cannot appear in a new build, and that every name that
does appear is still one the allow-list knows.

**4. Two tools move with the floor, and one of them had to.** `golangci-lint`
v2.11.4 is built with Go 1.26 and refuses a module targeting 1.27 before reading
its configuration — not a finding, a hard stop. The pin moves to v2.13.2, whose
published binary is built with go1.27.0.

That version has opinions the old one did not, and one of them is worth the
slice on its own: `ecdsa.PublicKey`'s `X` and `Y` have been deprecated since Go
1.26, and `internal/nodeauth` built every JWKS key by assigning them. It now
assembles the uncompressed point and calls `ecdsa.ParseUncompressedPublicKey`,
which rejects a point that is not on the curve. The comment that stood there —
that an off-curve point cannot verify a signature anyway, so no check was needed
— was true and is no longer the argument: a JWKS entry that cannot be a key is
refused where it is read, not where it is used.

The second finding the new version reports is a false one, and only off Linux:
on a platform where `startCapture` is a stub that always fails, `err != nil` is
always true. CI lints the Linux build, where it is not.

## Consequences

**Easier.** The floor is current, so the stdlib in the image is the stdlib
`govulncheck` inspects. The agent reports one more deliberate weakening, in the
class where a finding names a decision rather than a lag.

**Harder / given up.** The allow-list now outlives the toolchain that defined
it, and it will keep growing that way: names are added when a release adds them
and never removed when a release removes them. That is the right asymmetry for a
list about other people's binaries, and it means the list's length is not a
measure of anything.

A minor Go release changes more than this ADR reviews — language, runtime,
library behaviour. What is claimed here is only that the facts this agent
*reports* were re-measured against the new toolchain, and that the test suite,
the vulnerability gate, the chart and the end-to-end run pass on it.

**Not changed.** The payload's shape, the reading the backend is asked to make
of an absent name, and every other decision in ADR 0050 and ADR 0081. The
`ecdsa` change is a rewrite of how one key is constructed, not of which keys are
accepted — except the off-curve ones, which never verified anything.

This ADR records decisions implemented in the same pull request, per the process
in [`README.md`](README.md).
1 change: 1 addition & 0 deletions docs/adr/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -143,3 +143,4 @@ line to paste.
- [0083. The agent names itself in the request it makes](0083-the-agent-names-itself-in-the-request.md)
- [0084. What was excluded is named, inside the cluster and only there](0084-what-was-excluded-is-named-inside-the-cluster.md)
- [0085. The image is built and scanned where the code is, and pinned to what it was built from](0085-the-image-is-built-and-scanned-by-ci.md)
- [0086. A toolchain move is a review of what the toolchain reports](0086-a-toolchain-move-reviews-what-it-reports.md)
18 changes: 9 additions & 9 deletions docs/security.md
Original file line number Diff line number Diff line change
Expand Up @@ -550,7 +550,7 @@ nothing to what leaves your cluster.
| `node_metadata` | Per node: name, size, instance and capacity type, zone, region, age, kernel version, CPU architecture, kubelet version, OS image, container runtime. Size means CPU and memory, ephemeral storage, the kubelet's pod ceiling, and allow-listed accelerators — by **hardware vendor prefix**, so an extended resource your operators named is counted, not carried. Beside them the node's state: the five standard conditions with their reason and last transition, and its taints, which are the mirror of the tolerations already collected from pods ([ADR 0064](adr/0064-the-node-is-described-and-its-comings-recorded.md)). A condition's `message` is **never read** — it is free text into which the kubelet and node-problem-detector write paths and command output. The **name** is your cluster's, and on EKS and on GKE's legacy naming it encodes the node's private address — `ip-10-42-13-201.eu-west-1.compute.internal`. The agent reads no address field; the name it does read may be one (ADR 0039) | no |
| `go_inventory` | Per (namespace, workload, container): Go version, main module, image digest, PGO flag, plus a fleet-coverage block naming each reporting node and when it last reported | no |
| `process_peaks` | Per collected workload container and build: the largest `VmHWM` among its Go processes, how many processes that was taken over, and the range of permitted CPU counts. A **floor** under the container's peak, never the figure the OOM killer compares against — the cgroup also holds page cache and every other process in it ([ADR 0052](adr/0052-the-peak-the-kernel-remembers.md)). Beside it, the largest sample taken of each process's footprint: resident memory split into the program's own pages and file-backed ones, the memory it shares with its replicas and the memory nothing else maps, its OS threads, and how many file descriptors it held against the ceiling in force ([ADR 0061](adr/0061-what-the-process-holds-beside-its-peak.md)). **What each descriptor points at is never read** — they are counted, not followed | no |
| `go_build` | Per image digest, written once: Go version, main module, each dependency module's **path and version**, allow-listed build settings, two allow-listed GODEBUG defaults, and whether `net/http/pprof` is linked into the build. This is a bill of materials for the build — see [§10.4](#104-the-build-inventory-is-a-bill-of-materials) | no |
| `go_build` | Per image digest, written once: Go version, main module, each dependency module's **path and version**, allow-listed build settings, the allow-listed GODEBUG defaults, and whether `net/http/pprof` is linked into the build. This is a bill of materials for the build — see [§10.4](#104-the-build-inventory-is-a-bill-of-materials) | no |
| `process_counters` | Per collected workload container and build: what its processes did since each node's previous scan pass — time on CPU and time waiting for one, that CPU time split between your code and the kernel acting for it, page faults that reached storage, context switches, and bytes that moved to and from block storage. Each record carries the process-time the numbers cover, because a difference without its interval is not a rate ([ADR 0062](adr/0062-the-counters-and-the-window-they-cover.md)). Sums only — the agent never ships the division | no |
| `listening_ports` | Per collected workload container and build: the TCP ports its Go processes accept connections on, each with whether it is bound to loopback, and — for a port the agent asked — whether the Go pprof index answered there ([ADR 0082](adr/0082-a-confirmed-endpoint-is-named-on-its-port.md)). Read from the processes' own sockets, so a port nothing declared is here and a declared port nothing binds is not ([ADR 0056](adr/0056-a-pprof-endpoint-is-proved-not-probed.md)). No address, and no connection the workload has open | no |
| `ebpf_profile` | One capture: allow-list-filtered symbolized pprof bytes, keyed by workload, image digest and capture window | no |
Expand Down Expand Up @@ -792,18 +792,18 @@ the filter-early rule there, before any record is formed.
| `vcs.time` | that commit's timestamp (not the build time — Go does not record it) |
| `vcs.modified` | whether the working tree was dirty |

Fourteen GODEBUG defaults are read out of the toolchain's compound
`DefaultGODEBUG` setting and shipped as `godebug`
Fifteen GODEBUG defaults are read out of the toolchain's compound
`DefaultGODEBUG` and shipped as `godebug`, each `0` or `1`
([ADR 0050](adr/0050-godebug-defaults-are-a-build-fact.md),
[ADR 0081](adr/0081-a-weakened-default-is-a-build-fact.md)), each `0` or `1`, and
each a name the toolchain chose rather than a value anyone typed.
[ADR 0081](adr/0081-a-weakened-default-is-a-build-fact.md),
[ADR 0086](adr/0086-a-toolchain-move-reviews-what-it-reports.md)).
`containermaxprocs` and `updatemaxprocs` say whether the runtime sizes
`GOMAXPROCS` from the container's CPU quota. The other twelve each say whether
the build holds one security default at its pre-tightening value: `tls10server`,
`GOMAXPROCS` from the container's CPU quota. The other thirteen say whether the
build holds one security default at its pre-tightening value: `tls10server`,
`tls3des`, `tlsrsakex`, `tlssha1`, `tlsunsafeekm`, `x509negativeserial`,
`rsa1024min`, `cryptocustomrand`, `httplaxcontentlength`, `tarinsecurepath`,
`zipinsecurepath` and `execerrdot`. The compound value is parsed, never shipped
whole, and the names outside this list are not collected.
`zipinsecurepath`, `execerrdot` and `fips140ems`. The compound is parsed, never
shipped whole; names outside this list are not collected.

Everything outside it is discarded on the node, including **`-ldflags`,
`-gcflags`, `-asmflags` and `-tags`** — free-form flags that routinely carry
Expand Down
2 changes: 1 addition & 1 deletion go.mod
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
module github.com/RebuildStackCo/runtime-agent

go 1.26.7
go 1.27.1

require (
github.com/golang-jwt/jwt/v5 v5.3.1
Expand Down
40 changes: 32 additions & 8 deletions internal/nodeauth/jwks.go
Original file line number Diff line number Diff line change
Expand Up @@ -110,23 +110,47 @@ func (k jwk) publicKey() (crypto.PublicKey, error) {
if err != nil {
return nil, err
}
x, err := decodeBigInt(k.X)
// Assembled as an uncompressed point and parsed, rather than assigned
// coordinate by coordinate: the field is deprecated since Go 1.26, and
// what replaces it also rejects a point that is not on the curve
// (ADR 0086).
point, err := uncompressedPoint(curve, k.X, k.Y)
if err != nil {
return nil, fmt.Errorf("x coordinate: %w", err)
return nil, err
}
y, err := decodeBigInt(k.Y)
key, err := ecdsa.ParseUncompressedPublicKey(curve, point)
if err != nil {
return nil, fmt.Errorf("y coordinate: %w", err)
return nil, fmt.Errorf("public key: %w", err)
}
// An off-curve point cannot verify any signature (ECDSA verification
// rejects it), so no explicit on-curve check is needed here — that
// check matters for ECDH key agreement, which this package never does.
return &ecdsa.PublicKey{Curve: curve, X: x, Y: y}, nil
return key, nil
default:
return nil, nil
}
}

// uncompressedPoint builds the SEC 1 uncompressed encoding of a JWK's two
// coordinates: the 0x04 marker and both coordinates left-padded to the curve's
// field size, which is the length the parser requires.
func uncompressedPoint(curve elliptic.Curve, xB64, yB64 string) ([]byte, error) {
size := (curve.Params().BitSize + 7) / 8
point := make([]byte, 1+2*size)
point[0] = 4
for i, part := range []struct {
name string
value string
}{{"x coordinate", xB64}, {"y coordinate", yB64}} {
raw, err := base64.RawURLEncoding.DecodeString(part.value)
if err != nil {
return nil, fmt.Errorf("%s: %w", part.name, err)
}
if len(raw) == 0 || len(raw) > size {
return nil, fmt.Errorf("%s is %d bytes, want 1 to %d", part.name, len(raw), size)
}
copy(point[1+i*size+size-len(raw):], raw)
}
return point, nil
}

func curveFor(crv string) (elliptic.Curve, error) {
switch crv {
case "P-256":
Expand Down
52 changes: 52 additions & 0 deletions internal/nodeauth/jwks_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,15 @@ package nodeauth

import (
"context"
"crypto/ecdsa"
"crypto/elliptic"
"crypto/rand"
"crypto/rsa"
"encoding/base64"
"encoding/json"
"errors"
"fmt"
"math/big"
"net/http"
"net/http/httptest"
"sync"
Expand Down Expand Up @@ -319,3 +323,51 @@ func TestAFailedRefreshIsRateLimitedToo(t *testing.T) {
type keySourceFunc func(context.Context) (*KeySet, error)

func (f keySourceFunc) Fetch(ctx context.Context) (*KeySet, error) { return f(ctx) }

// An EC key now goes through the parser that replaced the deprecated coordinate
// fields, and that parser checks the point is on the curve (ADR 0086). The old
// construction accepted any pair of integers and left the rejection to signature
// verification; this rejects the key itself, which is where a JWKS entry that
// cannot be a key should stop.
func TestAnECKeyIsParsedAndAnOffCurvePointIsRefused(t *testing.T) {
key, err := ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
if err != nil {
t.Fatal(err)
}
coordinate := func(i *big.Int) string {
padded := make([]byte, 32)
i.FillBytes(padded)
return base64.RawURLEncoding.EncodeToString(padded)
}
//nolint:staticcheck // reading the coordinates is how a JWK is written; what ADR 0086 replaced is constructing a key from them
x, y := coordinate(key.X), coordinate(key.Y)

set, err := ParseJWKS([]byte(fmt.Sprintf(
`{"keys":[{"kty":"EC","kid":"ec-1","crv":"P-256","x":%q,"y":%q}]}`, x, y)))
if err != nil {
t.Fatalf("a valid EC key was rejected: %v", err)
}
got, err := set.Key("ec-1")
if err != nil {
t.Fatalf("EC key missing from the set: %v", err)
}
if parsed, ok := got.(*ecdsa.PublicKey); !ok || !parsed.Equal(&key.PublicKey) {
t.Errorf("parsed key = %#v, want the generated public key", got)
}

// The same x with a y that is not its pair: on the curve's field, off the
// curve itself.
offCurve := coordinate(new(big.Int).Add(key.Y, big.NewInt(1))) //nolint:staticcheck // see above
if _, err := ParseJWKS([]byte(fmt.Sprintf(
`{"keys":[{"kty":"EC","kid":"ec-bad","crv":"P-256","x":%q,"y":%q}]}`, x, offCurve))); err == nil {
t.Error("an off-curve point was accepted as a key")
}

// A coordinate shorter than the field is padded, not refused: a JWK may
// carry a leading zero byte trimmed.
short := base64.RawURLEncoding.EncodeToString([]byte{1})
if _, err := ParseJWKS([]byte(fmt.Sprintf(
`{"keys":[{"kty":"EC","kid":"ec-short","crv":"P-256","x":%q,"y":%q}]}`, short, short))); err == nil {
t.Error("a short pair that is not on the curve was accepted")
}
}
Loading
Loading