Skip to content

test: the chart's absences are asserted, not reviewed - #76

Merged
kfeofantov merged 1 commit into
mainfrom
test/chart-guardrails
Aug 27, 2026
Merged

kfeofantov merged 1 commit into
mainfrom
test/chart-guardrails

Conversation

@kfeofantov

Copy link
Copy Markdown
Contributor

What was wrong

chart_test.go asserts that the chart contains what it must — every grant the informers need, the capabilities the profiler asks for, the policy on the receiver's port. Those tests enumerate what is present, and a test that enumerates presence is indifferent to an addition standing next to it.

Almost every promise in docs/security.md is about absence. Measured, one mutation at a time, against the full ./internal/chartrender/ suite on main:

change to the chart result
resources: [pods, namespaces, nodes, secrets, configmaps] PASS
hostPID: true + hostNetwork: true on the controller PASS
capabilities.add: [NET_ADMIN, SYS_ADMIN] on the controller PASS
privileged initContainer in the Deployment PASS
hostPath: / mounted writable into the controller PASS

secrets with verb get is a read-only ClusterRole, so TestTheClusterRoleGrantsNoWriteVerb is satisfied; TestTheClusterRoleGrantsEveryCacheTheAgentOpens checks the required set is a subset, so a word beside it is invisible. drop: [ALL] next to add: [SYS_ADMIN] still drops ALL. allContainers walked Spec.Containers only. TestEveryHostMountIsReadOnly walked DaemonSets only.

None of these had to defeat a test. They walked past tests aimed elsewhere.

What this adds

internal/chartrender/guardrail_test.go — the same promises written the other way round: name the complete permitted set, fail on anything outside it, per rendered pod, in every install profile.

  • TestTheClusterRoleNamesNothingBeyondThisList — the resource list is closed, not a floor. Spelled out in the test rather than derived from the chart, because an expectation read from the thing under test asserts nothing.
  • TestTheOnlyNonResourceURLsAreTheOnesNodeTokensNeed — exactly the two OIDC reads when a node is installed, and neither when it is not.
  • TestOnlyTheNodeSharesTheHostsNamespaces — hostPID asserted in both directions: absent from the controller, present on the node, because losing it there makes the scanner collect nothing and say nothing about why. hostNetwork and hostIPC on no pod anywhere.
  • TestOnlyTheNodeContainerAddsACapability — measured on 1.36.1: a container with drop: [ALL] and add: [SYS_ADMIN, NET_ADMIN] gets CapEff: 0000000000201000, bits 21 and 12, exactly the two added. The add wins; the securityContext reads as careful.
  • TestTheControllerReachesNothingOutsideItsOwnPod — no host path (read-only included), no Secret, no projected volume. §7's blast-radius paragraphs are written about the DaemonSet on the understanding that the controller has no such reach.
  • TestNoWorkloadKindEscapesThePodSpecWalk — a guard on the guards. Every assertion above is per pod spec, so it is only as complete as allPodSpecs, which knows two workload kinds. A Job added later would otherwise be silently exempt from all of them.

allContainers now includes init containers, which is what catches the privileged one above — via the existing test, in chart_test.go.

Negative check

All nine mutations re-run against this branch. Each fails, with a message naming the promise it breaks:

guardrail_test.go:70:  the ClusterRole names "secrets" in apiGroup "", which is not on the
                       permitted list; the agent opens no informer on it and
                       docs/security.md §9 says it is not requested
guardrail_test.go:130: Deployment runtime-agent-controller sets hostPID; the controller reads
                       the API server, not the node (docs/security.md §2)
guardrail_test.go:162: Deployment runtime-agent-controller: container "controller" adds
                       [NET_ADMIN SYS_ADMIN]; only the node role adds capabilities
chart_test.go:205:     container "setup" is privileged
guardrail_test.go:192: Deployment runtime-agent-controller mounts host path "/"; the controller
                       reads the API server, not a node (docs/security.md §7)
guardrail_test.go:134: DaemonSet runtime-agent-node does not set hostPID; without it the scanner
                       sees only its own /proc and collects nothing (ADR 0009)
guardrail_test.go:101: nonResourceURLs = [... /metrics ...], want exactly [...]
guardrail_test.go:224: the chart renders a Job, which no guardrail in this file walks

Plus a read-only hostPath: /etc on the controller, caught by the same test as the writable one.

Docs

  • docs/security.md §2 — the host boundary runs between the two roles, and that this is asserted per rendered pod.
  • docs/security.md §9 — the "no secrets, no configmaps" row now says the resource list is closed by a test rather than by review.
  • docs/development.md — a testing-strategy bullet on why absences need their own tests, and the note that widening the chart means widening a list there.
  • Two chart comments pointed at charts/runtime-agent/render_test.go, which does not exist. Now internal/chartrender/chart_test.go.

Not in scope

No ADR: this records no decision, it holds decisions already recorded (0009, 0010, 0036, 0037, 0039). No agent behaviour changes and no chart behaviour changes — the only template edits are comments.

Gates

go build ./..., go test -race -count=1 ./..., go vet -tags e2e ./..., make lint (0 issues), make chart-lint (3 profiles, 0 failed), gofmt clean. No e2e run: nothing outside test files and comments changed.

The chart tests enumerated what must be present, and a test that enumerates
presence is indifferent to an addition standing next to it. Five mutations were
measured passing the whole suite green:

  resources: [pods, namespaces, nodes, secrets, configmaps]   PASS
  hostPID: true, hostNetwork: true on the controller          PASS
  capabilities.add: [NET_ADMIN, SYS_ADMIN] on the controller  PASS
  a privileged initContainer in the Deployment                PASS
  hostPath: / mounted writable into the controller            PASS

Each breaks a promise docs/security.md makes in plain words, and none of them
had to defeat a test to get in.

internal/chartrender/guardrail_test.go names the complete permitted set instead
and fails on anything outside it, per rendered pod, in every install profile:
the ClusterRole's resources and non-resource URLs, host namespaces (hostPID
exactly on the node DaemonSet, hostNetwork and hostIPC nowhere), capabilities
added by anything other than the node container, and any host path, Secret or
projected volume on the controller. A last test fails if the chart renders a
workload kind the pod-spec walk does not know, so a Job added later cannot be
silently exempt from all of it.

allContainers now includes init containers. An init container carries its own
securityContext and can hold privileges the containers beside it do not, so
walking only Spec.Containers left the existing privileged/drop-ALL assertions
with a hole; the chart renders none today, which is why the hole was invisible.

Also fixes two chart comments pointing at charts/runtime-agent/render_test.go,
a file that does not exist.

All nine mutations (the five above plus a third non-resource URL, hostPID
dropped from the node, and a read-only host path on the controller) were
re-run and each fails with a message naming the promise it breaks.

No agent behaviour changes, no ADR: this adds no decision, it holds decisions
already recorded.
@kfeofantov
kfeofantov merged commit d245a05 into main Aug 27, 2026
3 checks passed
@kfeofantov
kfeofantov deleted the test/chart-guardrails branch August 27, 2026 13:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant