Repository navigation
fix: profile key names the build, provenance names the claim (ADR 0023) - #49
Merged
Merged
Conversation
Two defects in the profile payload, found by the ADR 0022 audit. They look unrelated and are the same question asked twice: what identifies a profile, and what kind of fact is it. The spool filename dropped half the natural key. WriteProfile named files profile-<ns>-<workload>-<container>-<start>-<end>.json, leaving the image digest in the payload where it separated nothing. The node cuts every window on one pair of boundaries and ships one report per targeted container, so two replicas of a workload's container on the same node arrive with identical namespace, workload, container and capture interval — and the second file replaced the first, with no counter and no log line. The rollout case, two replicas of two builds, is the exact situation the digest was in the key for, and it was the one being lost. The code claimed the opposite: WriteProfile's comment said concurrent captures "never overwrite one another", and TestProfilesDoNotSupersede varied only the window, asserting the claim in the one case where it holds trivially. The new test fails without the fix — one file where two are expected. The filename now carries the digest as twelve hex characters after the algorithm separator, with every non-hex character discarded so nothing a registry could put in a digest string can become a path. A capture the controller could not tie to a build is named `nodigest` rather than given an empty segment. That state means the informer had not caught up, never that the container had not started — sampling a container's CPU means it was running. And the provenance field carried a capture method. ADR 0011 §6 wrote `source=ebpf` into the key before ADR 0012 §2 made `source` the provenance discriminator over a closed vocabulary; two concepts landed on one field name and the older meaning stayed in the bytes. The registry, the backend contract and ADR 0012 all said `sampled` already — the payload was the last holdout, and ADR 0013 §4's "fully realized in the bytes" was true of every kind but this one. It now declares `sampled`. No `capture` field is added. There is one profile source today, and a second arrives as its own kind — a /debug/pprof profile has no container-ID join and different window semantics, so it cannot share this natural key. A field duplicating what `kind` states is a field that will eventually disagree with it. One golden changed, by one field; the other nine are byte-identical, which is the evidence the change is confined. The capture e2e matches the spool listing on the prefix through the container name, so the extra segment did not break it — luck rather than design, and recorded as such. Closes two of ADR 0022 §5's five open items. The eligible-set enforcement, the label selector, and security.md's present tense stand. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two defects in the profile payload, both found by the ADR 0022 audit. They look unrelated and are the same question asked twice: what identifies a profile, and what kind of fact is it.
The silent overwrite
WriteProfilenamed filesprofile-<ns>-<workload>-<container>-<start>-<end>.json. The image digest sat in the payload where it separated nothing.That is not theoretical. The node cuts every window on one pair of boundaries and ships one report per targeted container (
cmd/agent/pipeline.go), so two replicas of a workload's container on the same node arrive with identical namespace, workload, container and capture interval. The controller joins each to a build and writes both under the same name — the second replaced the first, with no counter and no log line. The rollout case, two replicas of two builds, is the exact situation the digest was put in the key for, and it was the one being lost.The code claimed the opposite.
WriteProfile's comment read "concurrent or rotated captures of the same workload never overwrite one another", andTestProfilesDoNotSupersedevaried only the capture window — asserting the claim in the one case where it holds trivially. This is the shape of defect ADR 0018 exists to end: complete-looking output, quietly truncated.Reproduced before fixing, per
development.md's rule that a bug fix needs a test that fails without it:The filename now carries the whole key:
profile-<ns>-<workload>-<container>-<digest>-<start>-<end>.json. The digest is twelve hex characters after the algorithm separator, and every non-hex character is discarded, so nothing a registry could place in a digest string can introduce a path separator. A capture the controller could not tie to a build is namednodigestrather than given an empty segment — the state means the informer had not caught up, never that the container had not started, since sampling a container's CPU means it was running.This is truncation, and ADR 0019 refused to truncate. Different things: 0019 governed values that leave the cluster, where a prefix of an unexpected string ships looking legitimate. Here the full digest is in the payload and the short form is a local file name.
The provenance field carried a capture method
ADR 0011 §6 wrote
source=ebpfinto the key — meaning how the samples were taken — before ADR 0012 §2 madesourcethe provenance discriminator over a closed vocabulary (structural,measured,journal,sampled). Two concepts on one field name, and the older meaning stayed in the bytes.Everything else had already moved:
backend-requirements.mddocuments the four classes, ADR 0012's registry sayssampled, and ADR 0013 §4's "the registry is now fully realized in the bytes" was true of every kind but this one. The payload was the last holdout. It now declaressampled.No
capturefield is added. There is one profile source today, and a second arrives as its own kind — a/debug/pprofprofile has no container-ID join and different window semantics, so it cannot share this natural key anyway. A field duplicating whatkindalready states is a field that will eventually disagree with it.Scope
One golden changed, by one field; the other nine are byte-identical, which is the evidence the change is confined. The full-path capture e2e reads the spool listing and matched on the prefix through the container name, so the extra segment did not break it — luck rather than design, and recorded in the ADR as such.
Closes two of ADR 0022 §5's five open items. Still standing: the profiling eligible set enforced on the controller rather than the node, the label selector
security.mdpromises and no decision ever made, andsecurity.md's present tense about unbuilt things.Carried forward: profile filenames embed unescaped Kubernetes names and are unbounded in length, exactly as ADR 0020 recorded for
oom_kill. This adds thirteen characters to them. Neither is a traversal risk; a naming scheme that bounds length is one decision for both kinds, not two.make testandmake lintgreen.🤖 Generated with Claude Code