Skip to content

feat: collect the placement constraints that stop a workload from moving (ADR 0031) - #60

Merged
kfeofantov merged 1 commit into
mainfrom
feat/workload-placement
Aug 24, 2026
Merged

kfeofantov merged 1 commit into
mainfrom
feat/workload-placement

Conversation

@kfeofantov

Copy link
Copy Markdown
Contributor

workload_metadata gains a placement block in its pod scope: the nine pod-spec fields that say where a pod may run and what it costs to move it. No new payload kind, no new informer, no new RBAC — the pod spec was already read in full.

The gap this closes

The agent could say what a workload consumes, what it asked for, and which machine it got. It could not say why it is on that machine and not a cheaper one.

Replicas each sitting on their own large node may need the size — or may be held one-per-node by a required anti-affinity on kubernetes.io/hostname. The usage numbers are identical; the conclusions are opposite. Same for DoNotSchedule across zones, which keeps a workload paying for every zone it spans whether or not the capacity is needed there.

The three payloads now compose: workload_metadata says what was asked for, node_metadata says what machine answered, and this says what forbids a different answer.

Where it lives

In the pod scope, not in a kind of its own. Placement is a pod fact, so it nests where the other pod facts nest (ADR 0014) and repeats across a pod's containers exactly as the replica counts do.

It is taken from the first pod under the key, the way image, resources and ports already are. That is safe here because the key carries the image digest: a rollout that tightens placement produces one record per build, each with its own constraints, rather than whichever pod was seen first speaking for both. The golden shows exactly this — the old build with its selector and spread, the new one with a tightened node affinity and an unschedulable replica.

Reduced, not copied — and the loss is stated

Node affinity is flattened to (key, operator, values, required, weight); pod affinity and anti-affinity keep the topology key and whether the term is required; spread keeps skew, whenUnsatisfiable and minDomains. Label selectors inside pod-affinity and spread terms are dropped.

Flattening loses the OR/AND structure of node affinity, so the scheduler's decision cannot be replayed from this payload — and that is written into the ADR rather than left for a consumer to discover. What survives answers what a workload is pinned on and how hard, which is the question the payload exists for. A faithful copy would carry an arbitrarily nested selector into every record of every container to support an inference the product does not make.

Required and preferred stay apart: only a required term can leave a pod unschedulable.

Bounded strings, and drops that are counted

This is the second place after build settings where free-form strings written by whoever wrote the manifest enter the agent, so it takes ADR 0019's discipline unchanged — bounded at 128 bytes, and anything past a bound dropped whole rather than truncated, because a prefix of an unexpected string is still an unexpected string. A value list past its bound is dropped while its key and operator survive: a partial list would read as a complete one.

placement_values_dropped and placement_terms_dropped reach the coverage report, aggregate and never named. Both sit at zero on an ordinary cluster.

Cluster-wide defaults are not collected

DefaultTolerationSeconds puts not-ready and unreachable at 300s on every pod in the cluster, so carrying them would repeat two identical entries on every record and state nothing. The match is exact on the seconds — a pod that tuned one keeps it, because that deviation is how fast it leaves a failing node. Same for default-scheduler and a 30-second grace period: only the deviation is a fact. An unconstrained pod therefore contributes zero bytes, verified by a test that encodes it and asserts {}.

What is not read

env, args and command sit in the same PodSpec, and this is checked on the encoded bytes rather than on the shape of the struct. Verified negatively: making the reduction leak a container's command fails the test with "scheduler_name":"/bin/app" printed.

volumes are not read at all. A zonal volume does pin a pod, but resolving it runs through the StorageClass — RBAC the agent does not have — and the volume list references Secrets and ConfigMaps by name.

And one fact turned out not to be observable: a pod pinned by hand through spec.nodeName is indistinguishable after scheduling from one the scheduler placed, since every scheduled pod carries the field. It is not reported rather than guessed at.

Verification

make test (17 packages) and make lint (0 issues) green. The golden bytes changed, which is a protocol change and is what makes it one.

🤖 Generated with Claude Code

…ing (ADR 0031)

The agent could say what a workload consumes, what it asked for, and which
machine it got. It could not say why it is on that machine and not a cheaper
one — and that is where resource units become actionable. Replicas each
sitting on a large node may need the size, or may be held one-per-node by a
required anti-affinity on kubernetes.io/hostname. Identical usage numbers,
opposite conclusions.

Nine pod-spec fields answer it: nodeSelector, the three affinity blocks,
topologySpreadConstraints, tolerations, priorityClassName,
terminationGracePeriodSeconds, hostNetwork and schedulerName. All of them
were already being read — the pod spec is fetched in full for requests,
limits and ports — and none was kept.

The block nests in the pod scope of workload_metadata rather than becoming a
kind of its own: placement is a pod fact, so it goes where the other pod
facts go (ADR 0014) and repeats across a pod's containers exactly as the
replica counts do. It is taken from the first pod under the key, the way
image, resources and ports already are, which is safe because the key
carries the image digest — a rollout that tightens placement produces one
record per build instead of one build speaking for both.

The affinity structures are reduced, not copied, and the loss is stated
rather than hidden: flattening loses the OR/AND structure of node affinity,
so the scheduler's decision cannot be replayed from this payload. What
survives answers what a workload is pinned on and how hard, which is the
question. Copying faithfully would carry an arbitrarily nested selector into
every record of every container to support an inference the product does not
make.

Operator-written strings enter the agent here for the second time after
build settings, so they take ADR 0019's discipline unchanged: bounded at 128
bytes, and anything past a bound dropped whole rather than truncated,
because a prefix of an unexpected string is still an unexpected string. Two
counters reach the coverage report, aggregate and unnamed.

Cluster-wide defaults are excluded because they are not facts about the
workload: DefaultTolerationSeconds puts not-ready and unreachable at 300s on
every pod in the cluster, and carrying them would repeat two identical
entries on every record. The match is exact, so a pod that tuned one keeps
it — that deviation is how fast it leaves a failing node.

env, args and command sit in the same PodSpec and are checked on the encoded
bytes, not on the shape of the struct; verified by making the reduction leak
a command and watching the test fail. volumes are not read at all, even
though a zonal volume does pin a pod: the list references Secrets and
ConfigMaps by name.

One thing turned out not to be observable: a pod pinned by hand through
spec.nodeName is indistinguishable after scheduling from one the scheduler
placed. It is not reported rather than guessed at.

No RBAC, no new informer, no new payload kind. workload_metadata changes
shape, which is a protocol change and shows as changed golden bytes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@kfeofantov
kfeofantov merged commit ec08641 into main Aug 24, 2026
3 checks passed
@kfeofantov
kfeofantov deleted the feat/workload-placement branch August 24, 2026 13:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant