Skip to content

feat: a profile the collector dominates is still a profile (ADR 0063) - #98

Merged
kfeofantov merged 1 commit into
mainfrom
feat/a-profile-of-gc-is-still-a-profile
Sep 1, 2026
Merged

kfeofantov merged 1 commit into
mainfrom
feat/a-profile-of-gc-is-still-a-profile

Conversation

@kfeofantov

Copy link
Copy Markdown
Contributor

I said this finding cost the agent nothing, because the data already ships. That
was wrong in one specific way, and this PR is that way.

The defect

The symbol filter keeps runtime.* frames — they carry no domain, so the rule
that keeps the standard library keeps them — and every shipped ebpf_profile
and pprof_profile therefore carries the collector's frames beside the
workload's. Computing "what share of this workload's CPU is garbage collection"
is arithmetic on bytes that already leave.

Except Validate ranked functions by cumulative sample value and required a
service function among the top five.

Background marking runs on its own goroutines. Those samples carry no service
frame at all — the stacks are runtime.gcBgMarkWorker → runtime.gcDrain →
runtime.scanobject → … So a service spending most of its CPU collecting fills
all five top slots with runtime frames and ranks its own code below them.

The profile that proves a collector problem was refused for being about the
collector
, and the stronger the evidence the more certain the refusal. After
ADR 0060 that refusal is counted, which made the loss visible without making it
smaller.

The failing test is in the diff before the fix:

pprof_test.go:170: a GC-dominated profile was refused:
  nodeprofile: invalid profile: no service function among the top (only runtime/filtered)

The fix

Presence, not rank: one frame of the workload's own code, anywhere in any
sample. The check walks samples rather than the function table, so it does not
depend on the table holding only what the samples reference.

Both capture paths gain it at once — the eBPF pipeline and the pprof puller
share this validator.

Why weakening the gate is safe now

ADR 0059 leaned on this rank test as a mitigation: a service redacted by an
allow-list that stopped matching would produce a profile of nothing but runtime
and placeholders, and refusing it turned a wrong answer into silence.

That mitigation is no longer load-bearing. #94 removed the cause — the node reads
the modules off the binary, so a default install no longer redacts the service
layer. #95 made the symptom visible — a rising third_party_dropped beside a
rising profiles_invalid is what an allow-list that stopped matching looks like
from outside the cluster. And the remaining rule still catches the original case:
a profile whose every non-runtime frame was redacted has no service frame
anywhere. That case has its own test.

What did not change

The agent still does not compute the share. It ships the samples; the division
belongs to whoever asks the question (ADR 0062 §3). Nothing new is read, nothing
new leaves, and no frame that was redacted before is kept now — only the decision
to ship what the filter produced.

security.md's ceiling moves by two lines: which profiles may leave is a promise
to the customer, and it changed.

Evidence

go test ./... -race and golangci-lint run ./... on host and GOOS=linux:
0 issues. The e2e suites run against a kind cluster and are not part of CI, so
this has not been exercised on one.

🤖 Generated with Claude Code

https://claude.ai/code/session_011YRKuL97KRndaopfpixrhp

The share of a workload's CPU spent collecting is a finding the agent already
has the data for: the filter keeps runtime.* frames, so every shipped profile
carries the collector's beside the workload's. Except for the profiles where it
matters most.

Validate ranked functions by cumulative value and required a service function
among the top five. Background marking runs on its own goroutines, so those
samples carry no service frame at all — the stacks are gcBgMarkWorker →
gcDrain → scanobject. A service spending most of its CPU collecting fills all
five top slots with runtime frames and ranks its own code below them.

So the profile that proves a collector problem was refused for being about the
collector, and the stronger the evidence the more certain the refusal.

The rule becomes presence, not rank: one frame of the workload's own code,
anywhere in any sample. What the old rule guarded is guarded better now — ADR
0059 removed the cause of a redacted service layer, and ADR 0060 made the
symptom visible as third_party_dropped rising beside profiles_invalid.

Both capture paths gain it at once; they share this validator.

The agent still does not compute the share. It ships the samples; the division
belongs to whoever asks the question (ADR 0062 §3).

security.md's ceiling moves by two: which profiles may leave is a promise to the
customer, and it changed.

go test ./... -race, golangci-lint run ./... (host and GOOS=linux): 0 issues.
@kfeofantov
kfeofantov merged commit d1aa26a into main Sep 1, 2026
3 checks passed
@kfeofantov
kfeofantov deleted the feat/a-profile-of-gc-is-still-a-profile branch September 1, 2026 18:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant