Skip to content

fix: source paths do not leave the node; the allow-list bounds dependencies (ADR 0041) - #73

Merged
kfeofantov merged 3 commits into
mainfrom
fix/profile-source-paths
Aug 26, 2026
Merged

kfeofantov merged 3 commits into
mainfrom
fix/profile-source-paths

Conversation

@kfeofantov

Copy link
Copy Markdown
Contributor

ADR 0039 recorded two ways a frame gets past the symbol allow-list and deferred closing both to a slice of its own. Building that slice showed they are not the same kind of thing, and only one is a defect.

The exemption for the customer's own code is not a defect

SymbolFilter.keep returns early for any package whose path has no dot in its first segment — the standard library, the workload's main, and any module declared without a domain (module acmecorp). ADR 0011 §4 described the policy as "kept only if the module path is on the allow-list", so this looked like drift.

Closing it means requiring a customer to enumerate their own module paths before their own hot functions appear. Two things are wrong with that.

ADR 0025 already removed this shape. It abolished the profiling eligible set on the grounds that profiling scope is collection scope, and that a second list to keep in step with the first is a trap rather than a control. Workloads are excluded with namespace filters and opt-out annotations. Gating main behind a second list reintroduces the trap in the most visible place possible: install the agent, allow the namespace, and half the stack reads [filtered].

And the heuristic is sounder than it looked. A public Go module path must begin with a domain or it cannot be fetched. So "no domain in the first segment" means, in practice, the standard library or code the customer did not publish — and both are what they installed the profiler to see.

Checked that this does not widen the hostile-controller case: profiling targets come from the controller's already-filtered admitted-pod index, so it can only aim at workloads the customer's filters admitted. Keeping main exposes those workloads' function names and nothing else — which is the product.

So the code was right and ADR 0011 §4's wording was never true of it. This PR corrects the wording.

The source path is a defect

traceReporter copied f.SourceFile verbatim, keep() never looked at the field, and Serialize wrote it into the shipped pprof as Function.Filename. A Go binary built without -trimpath — the default — records the absolute path of the machine that compiled it.

Run through the real reporter, serializer and pprof parser:

symbolizer gave : /home/jenkins/workspace/acme-payments/src/git.acme-internal.example/payments/api/server.go
buffer holds    : File="server.go" Line=412

BEFORE (full path) -> Function{Name:"main.(*Server).handleCharge" Filename:"/home/jenkins/workspace/acme-payments/src/git.acme-internal.example/payments/api/server.go"}
AFTER  (base name) -> Function{Name:"main.(*Server).handleCharge" Filename:"server.go"}

What was leaving: the CI workspace, the internal VCS hostname, the project name, the layout of the customer's source tree. None of it is their code structure — it is our sampling carrying their build machine out of the cluster. Same class of string ADR 0019 excludes -ldflags/-gcflags/-tags for, one package away.

The cut happens at the seam, in ReportTraceEvent, so the full path never enters the buffer — filter-early (CLAUDE.md invariant 4) applied to the one field that had escaped it. Both separators are cut, not the host's: the path was recorded by whatever machine compiled the binary. Line numbers stay — a bare integer says nothing about that machine, and with the base name it is what a flame graph shows.

Evidence

Asserted in two places, because the reporter cuts the path and the serializer is a package away — an assertion only at the seam would miss a second producer of frames:

  • TestTheBuildMachinesPathNeverEntersTheBuffer — six path shapes: a CI absolute path, a -trimpath module path, a Windows build path, an already-bare name, an empty path, and a path that is nothing but a separator.
  • TestNoShippedFunctionCarriesADirectory — parses a serialized profile and requires no Function.Filename to contain a separator, with a guard against the assertion going vacuous.

go test -race ./..., make lint (0 issues), make chart-lint, go vet -tags e2e ./... all clean.

Honest gap. make profile-capture-e2e, the only suite that exercises live symbolization, SKIPPED: this host cannot grant CAP_BPF/CAP_PERFMON, so the node pod never starts (runc create failed: unable to apply caps). The change is proven against synthesized frames through the real code path, not against frames a kernel produced. That limitation is the known one recorded for the eBPF path generally, not something new here.

Also in this PR

  • docs/security.md §7.2 and §8: both passages currently warn about these as open gaps. One is closed; the other was never a gap, and the document should not describe a design decision as an outstanding defect.
  • The chart's empty-allow-list refusal keeps refusing — an empty list would give a customer with github.com/acme/… modules their main and the standard library and drop their entire service layer — but its wording is corrected; it claimed the profiler would "ship nothing".
  • docs/adr/0041-…, amending 0011 §4 and 0039 §5, with the mirror Amended by: lines.

🤖 Generated with Claude Code

kfeofantov and others added 3 commits August 26, 2026 21:48
…encies (ADR 0041)

ADR 0039 recorded two ways a frame gets past the symbol allow-list and deferred
closing both. Building that slice showed they are not the same kind of thing,
and only one is a defect.

The exemption for the customer's own code is not one. SymbolFilter.keep returns
early for any package with no dot in its first path segment, which covers the
standard library, the workload's main, and modules declared without a domain.
Closing that would make a customer enumerate their own module paths before their
own hot functions appeared — the second list ADR 0025 abolished, and the one
place it would be most visible: allow the namespace, and half the stack reads
[filtered]. The heuristic is also sounder than it looked: a public Go module
path must begin with a domain or it cannot be fetched, so "no domain" means the
standard library or code the customer did not publish. Both are what they
installed the profiler to see.

Nor does it widen anything. Profiling targets come from the controller's
admitted-pod index, so a controller can only aim at workloads the customer's
filters already admitted; keeping main exposes their function names and nothing
else, which is the product.

The source path is a defect. traceReporter copied f.SourceFile verbatim, keep()
never looked at the field, and Serialize wrote it into the shipped pprof as
Function.Filename. A binary built without -trimpath — the Go default — records
the absolute path of the machine that compiled it, so a profile carried the CI
workspace, an internal VCS hostname and the layout of the customer's source
tree. That is our sampling carrying their build machine out of the cluster: the
same class of string ADR 0019 excludes -ldflags/-gcflags/-tags for, one package
away.

Through the real reporter and serializer:

  symbolizer gave : /home/jenkins/workspace/acme-payments/src/
                    git.acme-internal.example/payments/api/server.go
  BEFORE          : Function{Name:"main.(*Server).handleCharge"
                    Filename:"/home/jenkins/workspace/…/api/server.go"}
  AFTER           : Function{Name:"main.(*Server).handleCharge"
                    Filename:"server.go"}

Cut at the seam, in ReportTraceEvent, so the full path never enters the buffer —
filter-early applied to the one field that had escaped it. Both separators are
cut: the path was recorded by whatever machine compiled the binary. Line numbers
stay; a bare integer says nothing about that machine.

Asserted twice: at the seam over six path shapes, and on the bytes of a
serialized profile, because the reporter cuts the path and the serializer is a
package away.

Honest gap in the evidence: profile-capture-e2e, the only suite that exercises
live symbolization, SKIPPED — this host cannot grant CAP_BPF/CAP_PERFMON, so the
node pod never starts. The change is proven against synthesized frames through
the real code path, not against frames a kernel produced.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Raised in review: what if an application has three files called server.go?

Measured — they stay three distinct Function entries in the shipped profile. A
symbolized Go function name carries its full package path, and a Go package is
a directory, so two files of the same name cannot share one. Package path plus
base name identifies the file exactly; what the cut removed is only the prefix
before the package path, which is the build machine's part of it.

ADR 0041's Consequences claimed this was "not a loss worth naming", which was
weaker than the truth and gave no reason. It now states the reason, and names
the case where something is genuinely lost: frames whose name carries no package
path — bare native symbols from libc or the kernel — where the base name is the
only file hint and can collide. Those are not the customer's code.

The property is now a test rather than an argument: if the serializer's dedup
key ever narrows to the function name alone, base names start losing
information, and this catches it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The comment stated it backwards: narrowing the serializer's dedup key to the
function name alone would not collapse anything, since the three names differ.
What would collapse them is keying on the file alone — three frames all reading
server.go becoming one entry. That is the case the test catches, and now the
case the comment describes.

Also records the point plainly: a consumer that rejoins the package path from
the function name with the base name gets exactly what `go build -trimpath`
would have put in the binary.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@kfeofantov
kfeofantov merged commit 0b90411 into main Aug 26, 2026
3 checks passed
@kfeofantov
kfeofantov deleted the fix/profile-source-paths branch August 26, 2026 18:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant