Repository navigation
feat: the controller confirms an endpoint once per build (ADR 0057) - #92
Merged
Merged
Conversation
The first outbound connection the agent makes to anything but the API server, and it is bounded by the three funnel stages that cost no network at all (ADR 0056): the workload passed the filters, its build links net/http/pprof, and the port came from the process's own socket. - One GET of /debug/pprof/ per (image digest, port). Fifty replicas cost one request; the answer is remembered, negatives included. - Confirmation matches the title Go's own index page carries, not the status: a 200 alone is any router with a permissive fallback. - One path, and the allow-list is the point — the same mux serves /debug/pprof/cmdline, which is the process argv the agent drops everywhere else. TestOnlyTheIndexPathIsRequested holds it. - Confirmed and absent are terminal under a digest; unreachable is a fact about the network and expires. - Counters land in collection_coverage, so "no profiles here" separates into three causes rather than being silence (ADR 0054). - The pod IP is read at the moment of connecting and enters no payload; PodWatcher.PodAddress is the only place it is read. On by default, off with profiling.pprof.enabled, inert without a node role. The `pprof` installation profile is deleted: discovery needs the node's two reads, so it is part of `inventory`, not an alternative to it. No egress NetworkPolicy — the rule would have to permit the API server, DNS, and arbitrary pod addresses, which is everything (ADR 0039). go test ./... and golangci-lint run ./... (host and GOOS=linux) — pass, 0 issues. e2e not run locally; the sample now serves pprof on :6060 and checkEndpointConfirmed asserts the confirmation in a real cluster. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011YRKuL97KRndaopfpixrhp
kfeofantov
added a commit
that referenced
this pull request
Aug 30, 2026
…s (ADR 0058) (#93) Adds the payload kind pprof_profile: a ten-second CPU profile fetched from an endpoint #92 confirmed, reduced to allowed frames in the controller. The defect found while building it, and the reason slices 4 and 5 are one PR: the symbol filter classifies a frame by module path, so a customer's own service under github.com/acme/web is "third-party" against an empty allow-list — the chart's default. The shipped behaviour would have been to pull a profile, redact the entire service layer, and throw the result away. Now the filter also admits the modules the build states it was compiled from: its main module, and every dependency a `replace` redirects, both read from the binary where no configured list can be wrong about them. A default install produces a usable profile. - Bounded by construction: 10s per capture, one at a time, ≤10 per 5-minute round. Round-robin over least-recently-profiled, not ranked by consumption — a ranking never sees the tail. - A non-200 is the process declining to start a second profiler, which means the customer runs their own. Held for 6h, never fought. No proactive list of profiler modules: it guesses at what one harmless attempt establishes. - Mappings (executable path, build ID), sample labels, comments and full source paths are removed with the redacted frames; the output is rebuilt from what survived, so no redacted name is left in an unreferenced table. - Filtering moves into the controller for this path. That is a real weakening of ADR 0011 §4's physical guarantee and security.md §10.3 says so. go test ./... and golangci-lint run ./... (host and GOOS=linux) — pass, 0 issues. e2e not run locally; checkProfilePulled asserts a filtered profile reaches the spool with the chart's allow-list left empty. Claude-Session: https://claude.ai/code/session_011YRKuL97KRndaopfpixrhp Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Slice 3 of the pprof plan. This is where the agent first connects to a customer workload, so most of the design is about making that connection rare, bounded, and about the right subject.
What one probe is
A single
GET /debug/pprof/— the index page, rendered from memory. No profiler starts, nothing is held open. Confirmation matches the title Go's own index carries, not the status code: a 200 alone is what any router with a permissive fallback returns.internal/pprofprobetests this against the standard library's realpprof.Indexhandler rather than a fixture written from memory.What makes it rare
The subject is a build's port, not a pod — every replica serves the same page, so a fifty-replica Deployment costs one request. Three funnel stages run first and cost no network at all (#91): passed the filters → build links the package → port the process was observed to bind. Loopback ports are dropped rather than asked about.
Confirmed and absent are terminal under a digest. Unreachable is a statement about the network, not the endpoint, so it expires and is retried.
The path allow-list is the point, not caution
The same mux serves
/debug/pprof/cmdline, which returns the process's own argv — the field invariant 4 drops at the source in the cache transform, on the node, and everywhere else. An endpoint that hands it over on request is not a reason to collect it.TestOnlyTheIndexPathIsRequestedasserts the only path requested is the index.Two things I decided and want visible
No egress NetworkPolicy — and that is now a decision, not a gap. The existing policy's comment deferred egress restriction to "the change that gives it something to egress to". This is that change. A rule would have to permit the API server (a ClusterIP not knowable at render time), DNS, arbitrary pod addresses and ports, and a backend later. That set is everything, and a rule permitting everything is a boundary in name only — the exact mistake ADR 0039 exists about. What bounds the egress is the code, and that bound is countable: one package, one path, addresses of admitted pods, ports their processes bind. The policy file now says this instead of promising otherwise.
The
pprofinstallation profile is deleted. Discovery needs the node's binary read and socket read, so it is part ofinventory, not an alternative to it. Thefailin_helpers.tplgoes with it; the generic unknown-profile refusal still catches anyone who tries--set profile=pprof, now with the right reason.Defaults
On. This is the one collection control shipped that way, and §10.3 stops describing a plan and describes what the agent does, before anyone installs it — a customer who learns this from the document has agreed to it. Pulling an actual profile is a separate switch and a separate ADR: that request starts a profiler inside the customer's process, which this one does not.
Evidence
go test ./...andgolangci-lint run ./...pass on host andGOOS=linux, 0 issues.internal/pprofprobe, including the real-handler confirmation, the path allow-list, redirect refusal, the per-round bound, and the retry window.0.0.0.0:6060and declares that port nowhere, socheckEndpointConfirmedproves the whole funnel in a real cluster.Ceilings
security.md980 → 996 andbackend-requirements.md560 → 575, for an access the documents did not previously describe and a coverage block the backend must not misread.🤖 Generated with Claude Code
https://claude.ai/code/session_011YRKuL97KRndaopfpixrhp