Repository navigation
feat: a profile the collector dominates is still a profile (ADR 0063) - #98
Merged
Merged
Conversation
The share of a workload's CPU spent collecting is a finding the agent already has the data for: the filter keeps runtime.* frames, so every shipped profile carries the collector's beside the workload's. Except for the profiles where it matters most. Validate ranked functions by cumulative value and required a service function among the top five. Background marking runs on its own goroutines, so those samples carry no service frame at all — the stacks are gcBgMarkWorker → gcDrain → scanobject. A service spending most of its CPU collecting fills all five top slots with runtime frames and ranks its own code below them. So the profile that proves a collector problem was refused for being about the collector, and the stronger the evidence the more certain the refusal. The rule becomes presence, not rank: one frame of the workload's own code, anywhere in any sample. What the old rule guarded is guarded better now — ADR 0059 removed the cause of a redacted service layer, and ADR 0060 made the symptom visible as third_party_dropped rising beside profiles_invalid. Both capture paths gain it at once; they share this validator. The agent still does not compute the share. It ships the samples; the division belongs to whoever asks the question (ADR 0062 §3). security.md's ceiling moves by two: which profiles may leave is a promise to the customer, and it changed. go test ./... -race, golangci-lint run ./... (host and GOOS=linux): 0 issues.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
I said this finding cost the agent nothing, because the data already ships. That
was wrong in one specific way, and this PR is that way.
The defect
The symbol filter keeps
runtime.*frames — they carry no domain, so the rulethat keeps the standard library keeps them — and every shipped
ebpf_profileand
pprof_profiletherefore carries the collector's frames beside theworkload's. Computing "what share of this workload's CPU is garbage collection"
is arithmetic on bytes that already leave.
Except
Validateranked functions by cumulative sample value and required aservice function among the top five.
Background marking runs on its own goroutines. Those samples carry no service
frame at all — the stacks are
runtime.gcBgMarkWorker→runtime.gcDrain→runtime.scanobject→ … So a service spending most of its CPU collecting fillsall five top slots with runtime frames and ranks its own code below them.
The profile that proves a collector problem was refused for being about the
collector, and the stronger the evidence the more certain the refusal. After
ADR 0060 that refusal is counted, which made the loss visible without making it
smaller.
The failing test is in the diff before the fix:
The fix
Presence, not rank: one frame of the workload's own code, anywhere in any
sample. The check walks samples rather than the function table, so it does not
depend on the table holding only what the samples reference.
Both capture paths gain it at once — the eBPF pipeline and the pprof puller
share this validator.
Why weakening the gate is safe now
ADR 0059 leaned on this rank test as a mitigation: a service redacted by an
allow-list that stopped matching would produce a profile of nothing but runtime
and placeholders, and refusing it turned a wrong answer into silence.
That mitigation is no longer load-bearing. #94 removed the cause — the node reads
the modules off the binary, so a default install no longer redacts the service
layer. #95 made the symptom visible — a rising
third_party_droppedbeside arising
profiles_invalidis what an allow-list that stopped matching looks likefrom outside the cluster. And the remaining rule still catches the original case:
a profile whose every non-runtime frame was redacted has no service frame
anywhere. That case has its own test.
What did not change
The agent still does not compute the share. It ships the samples; the division
belongs to whoever asks the question (ADR 0062 §3). Nothing new is read, nothing
new leaves, and no frame that was redacted before is kept now — only the decision
to ship what the filter produced.
security.md's ceiling moves by two lines: which profiles may leave is a promiseto the customer, and it changed.
Evidence
go test ./... -raceandgolangci-lint run ./...on host andGOOS=linux:0 issues. The e2e suites run against a kind cluster and are not part of CI, so
this has not been exercised on one.
🤖 Generated with Claude Code
https://claude.ai/code/session_011YRKuL97KRndaopfpixrhp