Skip to content

feat: the node is described, and its comings and goings recorded (ADR 0064) - #100

Merged
kfeofantov merged 1 commit into
mainfrom
feat/the-node-is-described-and-its-comings-recorded
Sep 3, 2026
Merged

kfeofantov merged 1 commit into
mainfrom
feat/the-node-is-described-and-its-comings-recorded

Conversation

@kfeofantov

Copy link
Copy Markdown
Contributor

Nine gaps in what the agent knows about a node, closed together. Eight are
fields on an informer the agent already runs; the ninth is a fact no snapshot
can hold.

node_metadata describes the machine

age metadata.creationTimestamp
version kubeletVersion
OS osImage, operatingSystem
runtime containerRuntimeVersion
ephemeral storage capacity and allocatable
pods capacity and allocatable — the ceiling a node hits while idle in CPU and memory
GPU extended resources, allow-listed by hardware-vendor prefix
conditions the five standard types, with reason and last transition
taints key, value, effect

pods was not on the list asked for. It is one field from the same map as
ephemeral storage, and leaving it out makes "why won't this schedule" half
answerable, so it is here — flagged rather than slipped in.

node_lifecycle — the ninth

A windowed journal of arrivals and departures, keyed and delivered exactly like
pod_disruptions. Each record repeats the node's instance type, capacity and
devices, and that repetition is the whole point: a node that left is gone from
node_metadata by definition, so without the size on the event, "twelve nodes
left this hour" can never become "and they were 96 cores of spot capacity".

Two honesty constraints, both with precedent in this repository:

  • An arrival is reported only when it can be proved. The test is the
    creation timestamp against when this process started watching — at cache sync
    the add handler fires for every node in the cluster, and counting those would
    report an agent restart as the entire fleet arriving at once. ADR 0034's rule,
    applied to nodes.
  • A departure carries the agent's observation instant, and says so. A deleted
    object has no deletion time to read, so at means two different things and
    at_observed marks which — rather than one field silently mixing provenances.

What is filtered, and why there

Three reductions, each following the discipline placement.go set for
operator-written strings (ADR 0019, ADR 0031):

  • Conditions are allow-listed by type. message is never read: it is
    free text into which the kubelet and node-problem-detector write paths and
    command output. The type allow-list is what makes reason safe to carry.
    The last transition is kept, the heartbeat is not — it moves every few
    seconds on every node and would make every node differ from its previous view
    on every flush.
  • Extended resources are allow-listed by vendor prefix — prefix because a
    GPU and a MIG slice of it are the same fact, allow-list because the rest of
    that namespace belongs to the customer, where a resource named for an internal
    team or licence must not travel.
  • Taints are bounded at 128 bytes and dropped whole. These are the first
    customer-written strings read from a node; the justification is symmetry, not
    harmlessness — tolerations are already collected from every pod spec, and a
    toleration without the taint it answers is half a fact.

What each refuses is counted in collection_coverage.nodes. Aggregate only.

Consequence worth naming

NodeInfo is no longer comparable, so the informer's "report only when the view
changed" test moved from == to a deep equality. Affordable because node objects
change on the order of minutes — the kubelet heartbeats to a Lease, not to the
node — and the one sub-second field it does write is the condition heartbeat,
which is deliberately not collected.

One existing test asserted the old contract: it added a Ready condition and
called that "an update that changes nothing in the collected view". Under this
change a condition is the collected view, so the test now moves the heartbeat
instead — which makes it a regression test for the decision above.

Evidence

go test ./...            all packages pass
golangci-lint run ./...  0 issues

Goldens regenerated with -update; node-metadata.golden.json now carries an
accelerator node under memory pressure so the three reduced lists appear in the
bytes rather than only in unit tests.

Docs updated in the same PR: ADR 0064, security.md §4 and the payload table,
backend-requirements.md ingest rules for both kinds. Two prose ceilings moved,
for the new payload kind and the wider node read.

🤖 Generated with Claude Code

https://claude.ai/code/session_011YRKuL97KRndaopfpixrhp

… 0064)

`node_metadata` carried ten fields drawn for one job — turning a node name into
a zone. It now describes the machine: age, kubelet version, OS image, container
runtime, ephemeral storage and the pod ceiling, allow-listed accelerators, the
five standard conditions, and taints.

Adds `node_lifecycle`, a windowed journal of nodes joining and leaving. It is
the first payload that says what the fleet *was*: a departed node is absent from
every snapshot by definition, so the record carries its own size and devices.

Three bounded reductions, each following the discipline placement.go set for
operator-written strings: conditions allow-listed by type and never carrying
`message`, extended resources allow-listed by hardware-vendor prefix, taints
bounded at 128 bytes. What each refuses is counted in `collection_coverage`,
never named.

An arrival is reported only when the creation timestamp proves it — at cache
sync the add handler fires for every node, and counting those would report an
agent restart as the whole fleet arriving (ADR 0034's rule, applied to nodes).
A departure carries the agent's observation instant and says so.

    go test ./... — all packages pass; golangci-lint run ./... — 0 issues

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011YRKuL97KRndaopfpixrhp
@kfeofantov
kfeofantov merged commit fbf9b7d into main Sep 3, 2026
3 checks passed
@kfeofantov
kfeofantov deleted the feat/the-node-is-described-and-its-comings-recorded branch September 3, 2026 18:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant