Skip to content

feat: topology-aware routing is a request the cluster can silently decline (ADR 0051) - #86

Merged
kfeofantov merged 1 commit into
mainfrom
feat/topology-routing-is-asked-and-not-granted
Aug 29, 2026
Merged

kfeofantov merged 1 commit into
mainfrom
feat/topology-routing-is-asked-and-not-granted

Conversation

@kfeofantov

Copy link
Copy Markdown
Contributor

Setting spec.trafficDistribution asks for zone-local traffic. The
EndpointSlice controller decides whether to arrange it, and declines whenever the
endpoints are distributed too unevenly against the zones' CPU capacity. There is
no event, no condition and no status field: a Service whose routing works and one
whose routing was refused read identically in kubectl get svc, and the second
keeps paying a cross-zone transfer bill its owner believes they eliminated.

What ships. workload_policy now carries, per Service: traffic_distribution,
topology_mode (the two annotations that asked for this before the field
existed), and ready-endpoint counts by zone with how many the cluster gave
hints.forZones. The finding is one comparison on one payload — a mode set
against hinted: 0 — and needs no history, so it belongs to the first-hour
report.

No endpoint identity is read. Addresses (pod IPs), targetRef, nodeName,
hostname and the deprecated topology map are removed as the object enters the
cache (ADR 0046), so they are never held rather than held and dropped. The join
to a pod is never made, which is what keeps the counts clear of invariant 6. It
also bounds memory: a Service with thousands of endpoints would otherwise carry
thousands of addresses for the life of the process.

Counts are per address family. A dual-stack Service lists each pod once in an
IPv4 slice and once in an IPv6 slice. Merged, every zone doubles — uniformly, so
the skew still looks right while every absolute number is wrong. MUST NOT sum across families is stated in backend-requirements.md.

RBAC. Adds discovery.k8s.io/endpointslices (get/list/watch) — the first
resource in that group. Without it the cache never syncs, endpoint_slices names
itself in unavailable_sources, and the Services still ship (ADR 0033).

Two things the real cluster taught this PR: the API server warns that
PreferClose is deprecated in favour of PreferSameZone, which is a second
reason the value is passed through verbatim rather than normalized to a closed
set — the vocabulary moves and the agent cannot be updated to follow it. The e2e
now sets the current spelling and the unit tests keep the deprecated one, because
a fleet mid-migration has both.

backend-requirements.md ceiling 475 → 495; security.md fits under its
existing one.

Evidence:

go test -race ./... && make lint && make chart-lint   # green
make policy-e2e
# service web asked for PreferSameZone; 2 ready endpoints, 2 hinted,
# zones map[e2e-zone-a:2]

🤖 Generated with Claude Code

https://claude.ai/code/session_011YRKuL97KRndaopfpixrhp

…cline (ADR 0051)

Setting `spec.trafficDistribution` asks for zone-local traffic; the EndpointSlice
controller decides whether to arrange it, and declines when the endpoints are
distributed too unevenly against zone capacity. There is no event, no condition
and no status field — a Service whose routing works and one whose routing was
refused read identically, while the second keeps paying for cross-zone transfer.

`workload_policy` now carries, per Service, the field, the topology-mode
annotation, and ready-endpoint counts by zone with how many the cluster gave a
hint. The finding is one comparison: a mode set against `hinted: 0`.

Endpoint addresses, targetRef, node and hostname are removed as the object
enters the cache, so the agent never holds a pod IP — the join to a pod is never
made, which is what keeps the counts clear of invariant 6. Counts are per
address family: a dual-stack Service lists each pod twice, and summing doubles
every zone uniformly, leaving the skew plausible and every number wrong.

Adds `discovery.k8s.io/endpointslices` to the ClusterRole. Without it the cache
never syncs, `endpoint_slices` names itself in `unavailable_sources`, and the
Services still ship (ADR 0033).

    go test -race ./... && make lint && make chart-lint   # green
    make policy-e2e   # "service web asked for PreferSameZone; 2 ready
                      #  endpoints, 2 hinted, zones map[e2e-zone-a:2]"

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011YRKuL97KRndaopfpixrhp
@kfeofantov
kfeofantov merged commit 6e81e2b into main Aug 29, 2026
3 checks passed
@kfeofantov
kfeofantov deleted the feat/topology-routing-is-asked-and-not-granted branch August 29, 2026 08:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant