Repository navigation
feat: liveness is the process, readiness is the caches (ADR 0069) - #107
Merged
Merged
Conversation
Neither role had a probe of any kind. A wedged controller was never restarted, and `kubectl rollout status` returned success while the 18 informers were still listing. Both roles now serve an always-on HTTP listener on port 9090 answering /livez and /readyz — read-only, GET and HEAD only, and no reply names an object in the cluster. Liveness is each loop's own heartbeat and depends on nothing outside the process: an API server outage must not restart the DaemonSet on every node at once. Readiness is the caches that gate collection plus an open spool for the controller, and the first completed scan pass for the node. A startup probe covers the initial sync, with a budget above the five minutes a gating cache may fail before the agent stops itself (ADR 0035). The NetworkPolicy gains a second ingress rule for the health port with no source: the caller is the kubelet, arriving from the node's address. The security.md ceiling moves by the one row a newly opened port earns. make test && make lint && make chart-lint && make vulncheck → all pass; 0 lint issues; 3 charts linted, 0 failed; 0 vulnerabilities Mutation-checked: deleting a probe, the beat, the readiness latch, the cache gate, or the policy's port restriction each fails a named test. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KeZECJXTDHL6Nts95eKMJA
The compressed flush interval was 20ms, so the liveness deadline under test was 60ms — shorter than one flush pass takes under -race on CI. The agent reported itself stale correctly; the test's ratio was wrong. Both compressed periods now keep the ratio the real ones have. go test -race -count 5 -run StaysAlivePastItsDeadline ./cmd/agent → ok deleting either beat still fails its test Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KeZECJXTDHL6Nts95eKMJA
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Neither role had a probe of any kind — no liveness, no readiness, no startup, on neither workload. A wedged controller was never restarted, and
kubectl rollout statusreturned success while the 18 informers were still listing.What ships
/livezand/readyz. Read-only: GET and HEAD only, no body read, no query parsed, and no reply names an object in the cluster. Separate from 8080, which exists only where a DaemonSet does and carries the node reports. On no Service.health.listenAddressin the controller's config file,-health-addresson the node — its file is rendered only by theebpfprofile, and probes are needed ininventorytoo (ADR 0025).Why liveness ignores the apiserver, why readiness is not all 18 caches, and why the node's readiness is not the scope query: ADR 0069.
Evidence
Mutation-checked — each of these fails a named test: deleting a probe from either workload, deleting the heartbeat stamp from either loop, deleting the node's readiness latch, deleting the readiness cache gate, removing the policy rule's port restriction, omitting the listen address from either role.
docs/security.mdgains one §5 row and its ceiling moves by exactly that row: a port the agent opens is an obligation the document did not have before (ADR 0044).Not run here
e2e (kind) has not been run for this branch — it needs
make cluster-up e2e cluster-downlocally, and this PR touches cluster-facing templates.🤖 Generated with Claude Code
https://claude.ai/code/session_01KeZECJXTDHL6Nts95eKMJA