Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion cmd/agent/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -376,7 +376,7 @@ func run(ctx context.Context, logger *slog.Logger, clientset kubernetes.Interfac
// a consumer pair a revision with a shape from a different moment
// (ADR 0030).
revisionRecords := revisions.Aggregate(podWatcher.ReplicaSets())
if err := spool.WriteDeploymentRevisions(capturedAt, revisionRecords); err != nil {
if err := spool.WriteWorkloadRevisions(capturedAt, revisionRecords); err != nil {
logger.Error("spooling deployment revisions", "error", err)
}
// Policy rides the same instant for the same reason: what bounds a
Expand Down
2 changes: 2 additions & 0 deletions docs/adr/0030-deployment-revisions.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@
Date: 2026-08-23
Status: Accepted

Amended by: 0049

Adds the payload kind `deployment_revisions`. Nothing existing changes shape.

## Context
Expand Down
101 changes: 101 additions & 0 deletions docs/adr/0049-revisions-are-not-only-deployments.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
# 0049. Revisions belong to whatever controls the ReplicaSet, and the payload is renamed for it

Date: 2026-08-29

Status: Accepted

Amends: 0030 §1–2

Widens `deployment_revisions` to every workload that manages ReplicaSets and
renames it `workload_revisions`. No new RBAC, no new field, no new payload kind.

## Context

ADR 0030 scoped revisions to Deployments, and at the time that was the whole of
what manages ReplicaSets in a cluster this agent could see. It is not. Argo
Rollouts is widely deployed, and a `Rollout` creates and scales ordinary
ReplicaSets exactly as a Deployment does — the agent already lists them, already
resolves their pods, and already reports `workload_kind: Rollout` (ADR 0048 §2).
The only thing standing between those sets and the payload was a kind check.

Measured on kind (Kubernetes 1.36.1) against Argo Rollouts v1.10.0, mid-canary
with `setWeight: 25` on four replicas:

payments-7bd58f4db4 rev 2 desired 1 current 1 ready 1 pause:3.10
payments-f47dc5486 rev 1 desired 3 current 3 ready 3 pause:3.9

That is the shape ADR 0030 §2 already describes — two revisions with non-zero
counts is a rollout in flight, and the split is the weight it is running at. The
record needs no new field to say it; it was simply never emitted.

`revisionHistoryLimit` defaults to 10 on a Rollout as it does on a Deployment,
so the bound ADR 0030 leaned on to keep the snapshot small holds unchanged. With
the limit set to 3 the cluster retained five sets: three historical, the stable
one, and the one in flight.

Two things this does not reach, and both are worth stating so the widening is
not read as more than it is. A `Rollout` paused at a step and a `Rollout`
degraded look identical from the ReplicaSets — `status.phase` and
`pauseConditions` are on the custom resource, which the agent does not read. And
StatefulSets and DaemonSets are still absent: their revisions live in
`controllerrevisions`, which ADR 0030 declined and this does not revisit.

## Decision

**1. Any ReplicaSet whose controller is a collected workload is reported**, and
the kind check goes. What made a Deployment special was that the agent knew the
name of its revision annotation, not anything about the object graph. Scope is
still inherited from the admitted pod index, so an excluded workload is absent
here exactly as before, and the namespace and pod opt-outs reach these workloads
even where the workload-level one cannot (`docs/security.md` §11).

**2. A ReplicaSet with no controller is not a revision.** A bare set is the
workload rather than one version of one, and it has no history to report; its
pods already resolve to the set itself. It is skipped rather than reported as a
revision of nothing.

**3. The revision number is read for Deployments and no one else.** Every
controller numbers its revisions under a key of its own —
`rollout.argoproj.io/revision` for Argo — and the agent reports `revision:
null` for all of them rather than learning vendor keys.

No finding needs the number. Deploy frequency needs the ordering, which
`created_at` gives; which revision is live needs the replica counts, which ship;
"this revision is that build" needs the image and the instant, which ship. A
table of vendor annotation keys would be a second list to keep in step with
other people's releases, bought for a field nothing reads. Records with no
number sort by age, so the payload stays deterministic.

**4. The kind is renamed `deployment_revisions` → `workload_revisions`**, and
the spool file with it. The old name would be false the moment the first Argo
record appeared, and a payload kind whose name contradicts its contents is the
defect ADR 0022 exists to prevent. The rename costs a registry row, a golden and
three documents today; after a backend exists it is a protocol migration. ADR
0019 renamed `go_dependencies` → `go_build` on the same reasoning.

## Consequences

**Easier.** The regression watch — CPU per replica across an image change — now
covers Argo-managed workloads, which was one of the three things such a workload
lost. Deploy frequency and rollout-in-flight follow for every ReplicaSet-based
controller at once, including operators nobody here has heard of, because
nothing in the reduction names a vendor.

**Harder / given up.** A consumer can no longer assume a revision number exists.
It was already optional — an unannotated ReplicaSet yielded null before this —
but it was rare, and it now happens for a whole class of workload. The backend
contract states it.

The payload can now carry a workload kind that is not a Kubernetes built-in, so
`workload.kind` is a string from a CRD rather than from a closed set. It is a
kind name, not free text, and it is already shipped in `workload_metadata` for
the same workloads (ADR 0048 §2), so this exposes nothing new.

**Not changed.** No new RBAC — ReplicaSets were already granted and read. No new
field, no new payload kind, no change to the natural key or the delivery
discipline. StatefulSet and DaemonSet revisions remain uncollected, and the
custom resource itself remains unread: what this decision uses is the object the
agent was already watching.

This ADR records decisions implemented in the same pull request, per the process
in [`README.md`](README.md).
1 change: 1 addition & 0 deletions docs/adr/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -106,3 +106,4 @@ line to paste.
- [0046. The informer cache is the source, so the fields the agent promises not to collect are removed there](0046-the-cache-is-the-source.md)
- [0047. Five environment variables are named, kept and shipped; every other one is still dropped](0047-runtime-knobs-are-named-and-kept.md)
- [0048. Four first-hour findings name fields the agent did not collect](0048-findings-name-the-fields-they-need.md)
- [0049. Revisions belong to whatever controls the ReplicaSet, and the payload is renamed for it](0049-revisions-are-not-only-deployments.md)
34 changes: 17 additions & 17 deletions docs/backend-requirements.md
Original file line number Diff line number Diff line change
Expand Up @@ -128,18 +128,14 @@ metadata are keyed per container, because requests, limits and CFS quota are
declared and enforced per container. A pod with an injected sidecar therefore
appears as several records sharing a workload and differing in `container`.
Summing their totals gives the pod or workload total — that is the intended use.
Facts that belong to the pod rather than the container are not repeated as
top-level fields but nested in a `pod` object, so a reader can tell from the
payload alone what a field is a fact about.

**`pod.phases` is placement and lifecycle, never health.** The counts are
Kubernetes pod phases verbatim, and the backend MUST NOT read `Running` as
"healthy": a pod in CrashLoopBackOff is `Running`, and so is one whose readiness
probe has never passed. Readiness and restart counts are not in this payload.
`Succeeded` and `Failed` pods are reported too — a CronJob's workload
legitimately shows dozens of them consuming nothing — so `pod.replicas` should
be read together with its breakdown rather than alone; the agent reports the
phases and does not decide which of them count.
legitimately shows dozens consuming nothing — so `pod.replicas` should be read
with its breakdown, not alone; which of them count is not the agent's call.

**Measured payloads state what was observed, and the backend MUST use it**
([ADR 0013](adr/0013-observation-completeness.md)). Each usage payload carries an
Expand Down Expand Up @@ -302,15 +298,14 @@ that does not exist.

**Structural snapshots say when they were taken, and the Go inventory says how
complete it is** ([ADR 0017](adr/0017-build-facts-keyed-by-digest.md)).
`go_inventory`, `workload_metadata` and `node_metadata` each carry `captured_at`
— the instant the snapshot was assembled, which is not a window and not the age
of the facts inside it. `go_inventory` additionally carries a `coverage` block
whose counters are cumulative from the `since` instant carried with them:
Every structural payload carries `captured_at` — the instant the snapshot was
assembled, which is not a window and not the age of the facts in it.
`go_inventory` adds a `coverage` block whose counters are cumulative from the
`since` instant carried with them:

- `nodes_reported` — how many nodes *currently in the cluster* have delivered at
least one scan; a node that has left is not counted. Read against the node
count in `node_metadata`, a shortfall means part of the fleet is not
reporting, and the inventory is incomplete for reasons no record can express.
least one scan. Read against the node count in `node_metadata`, a shortfall
means the inventory is incomplete for reasons no record can express.
- `facts_received` / `facts_joined` / `facts_unjoined` / `facts_undigested` —
what arrived and what could not be attributed. **The backend MUST NOT read a
missing workload as "not a Go workload"** when these say otherwise.
Expand All @@ -321,10 +316,15 @@ The agent drops records whose workload container has left its filtered pod index
— deleted, or opted out by annotation — so a record's disappearance between two
snapshots means "no longer collected", not "not observed this time". The backend
MUST treat a superseding snapshot as the complete current truth and MUST NOT
carry forward records absent from it. A short-lived workload (a CronJob's pods)
may legitimately appear in one snapshot and not the next; accumulating "has ever
been a Go workload" across snapshots is the backend's job, and `captured_at` is
what dates them.
carry forward records absent from it. A short-lived workload may appear in one
snapshot and not the next; accumulating history across them is the backend's
job, and `captured_at` is what dates them.

**`workload_revisions.revision` is a Deployment's number and nobody else's**
([ADR 0049](adr/0049-revisions-are-not-only-deployments.md)). A record of any
other `workload.kind` carries none — that controller numbers revisions under a
key the agent does not read — so the backend MUST NOT read the absence as
missing data, and MUST order those revisions by `created_at`.

**`go_build` is write-once and keyed by image digest.** What a build is made of
and how it was built — the Go version, the main module, the dependency module
Expand Down
13 changes: 7 additions & 6 deletions docs/security.md
Original file line number Diff line number Diff line change
Expand Up @@ -489,7 +489,7 @@ test fails if a kind ships without appearing here (ADR 0022).
| `restart_counters` | Per flush: for each collected container that has ever restarted, the kubelet's restart counter as the agent last observed it, how much of it the agent has watched and since when, when the pod was created, how long the current incarnation has been running, and the reason, exit code and instant of the most recent termination. The counter and the watched-since figure are read from one observation, so their difference is never negative (ADR 0043) | **yes** |
| `pod_disruptions` | Per hour: the pods the *cluster* removed — preempted, evicted under node pressure, drained, or removed via the eviction API — with the node and the instant | **yes** |
| `job_runs` | Per hour: each Job that finished — when it started and finished, whether it succeeded, the failure reason, its pod success/failure counts, and its declared `parallelism`/`completions`/`backoffLimit` | **yes** |
| `deployment_revisions` | Per flush: each ReplicaSet of a collected Deployment — its revision number, when it was created, its desired/current/ready replica counts, and each container's image reference | **yes** |
| `workload_revisions` | Per flush: each ReplicaSet of a collected workload that manages ReplicaSets — a Deployment, or a custom resource such as an Argo Rollout ([ADR 0049](adr/0049-revisions-are-not-only-deployments.md)) — with when it was created, its desired/current/ready replica counts, each container's image reference, and a revision number for a Deployment only | **yes** |
| `workload_metadata` | Declared shape per (namespace, workload, container, image digest): image, requests, limits, ports, probe schedules without what any probe checks, the five runtime knobs of the field-minimization list below, QoS, replica counts by phase, by node, and by unscheduled reason, the pod's reduced placement constraints (ADR 0031), and — for a Deployment, StatefulSet or DaemonSet — the update strategy by which the workload replaces its own replicas ([ADR 0048](adr/0048-findings-name-the-fields-they-need.md)). A workload of any other kind carries no strategy and nothing standing in for one | no |
| `workload_policy` | Per collected workload, what bounds it from outside its own spec: disruption budgets covering it, autoscalers driving it, the volume claims its pods mount (ADR 0032), and the Services routing to it (ADR 0048); plus `unavailable_sources`, naming any of those the agent could not read — never granted, or no longer being fed (ADR 0033, ADR 0035) | not directly — but a claim is named, and a StatefulSet's claim is `<template>-<set>-<ordinal>`, so `data-db-0` identifies pod `db-0` (ADR 0039) |
| `cluster_policy` | Per collected namespace, its LimitRanges and ResourceQuotas; plus the cluster's PriorityClasses and StorageClasses, which workloads reference by name (ADR 0032), and `unavailable_sources` for any of those the agent could not read — never granted, or no longer being fed (ADR 0033, ADR 0035). StorageClass `parameters` are never read | no |
Expand Down Expand Up @@ -589,11 +589,12 @@ recognized rather than assumed current.

### Object history, and the one place pods are named (ADR 0020)

`deployment_revisions` names ReplicaSets, which your cluster created and named.
Deployments only — StatefulSet and DaemonSet revisions live in
`controllerrevisions`, which the agent has no RBAC for — and a Deployment appears
only while it has collected pods, read from the same admitted index everything
else uses ([ADR 0030](adr/0030-deployment-revisions.md)). `job_runs` names the Job
`workload_revisions` names ReplicaSets, which your cluster created and named.
Any workload that manages ReplicaSets — a Deployment, or a custom resource such
as an Argo Rollout — and only while it has collected pods, read from the same
admitted index everything else uses ([ADR 0030](adr/0030-deployment-revisions.md),
[ADR 0049](adr/0049-revisions-are-not-only-deployments.md)). StatefulSet and
DaemonSet revisions live in `controllerrevisions`, which the agent does not read. `job_runs` names the Job
object, a generated per-run name such as `rollup-29123456`, and the CronJob that
scheduled it; a run whose Job, CronJob, namespace or pod template carries the
opt-out annotation produces no record ([ADR 0029](adr/0029-job-runs.md)).
Expand Down
Loading