Skip to content

fix: env, args and command never enter the informer cache (ADR 0046) - #79

Merged
kfeofantov merged 1 commit into
mainfrom
fix/informer-cache-drops-env
Aug 27, 2026
Merged

kfeofantov merged 1 commit into
mainfrom
fix/informer-cache-drops-env

Conversation

@kfeofantov

Copy link
Copy Markdown
Contributor

docs/security.md §8 promised that env, args and command are "discarded
before anything is stored or transported". They were discarded at the payload
seam, which ADR 0031 §7 checks on the encoded bytes. Upstream of that seam the
factory was informers.NewSharedInformerFactory(clientset, 0) — no transform,
no field selector — so every Pod, ReplicaSet, Deployment, StatefulSet, DaemonSet,
Job and CronJob in the cluster sat in memory as the API server returned it, for
the lifetime of the process. Invariant 4 ranks that second: not collected beats
collected and dropped.

  • One transform on the factory, so every informer gets it. env, envFrom,
    args and command go from every container list of every pod spec an object
    carries — a pod's own containers, init containers and ephemeral containers, and
    the template inside each workload kind.
  • managedFields and kubectl.kubernetes.io/last-applied-configuration go
    from every object, whatever its kind.
    The annotation is a verbatim copy of
    the object as applied: stripping containers while leaving it would have moved
    the values, not dropped them. A field selector cannot help here — the pod
    selectors the API server supports are spec.nodeName, status.phase, name and
    namespace.
  • Not done: the informers are not namespace-scoped. That would cut more, but
    the namespace filter supports a deny-list, and the annotation opt-outs are
    properties of objects the agent would then no longer watch.

Measured on kind 1.36.1 by reading /api/v1/pods rather than through kubectl,
which has hidden managedFields since 1.21 — so the objects an operator
inspects are not the objects an informer receives:

pod cache, 9 pods bytes
as received 77,463
without managedFields 46,917
without env / args / command 73,180
without both 42,634

45% of what the cache held was material nothing reads, against a 256Mi limit.

Both guards were checked by making them fail:

--- FAIL: TestTheStrippedObjectIsWhatTheListersServe
    the cache holds env=[{DATABASE_URL postgres://svc:hunter2@db.internal/prod nil}] args=[--token=t0ps3cret] for "app"
    the cache holds the applied-configuration copy
--- FAIL: TestNoWatchedKindEntersTheCacheCarryingEnvArgsOrCommand   (CronJob case removed)
    CronJob: container "app" reached the cache with env=[...] command=[/bin/sh -c] args=[...]

The wiring test earns its place: with the transform unregistered, the per-kind
test still passes — a transform registered with nothing strips nothing.

A 75-second run against the kind cluster: 0 errors, 9 pods observed,
workload_not_cached: 0, all four usage signals, and workload names resolved
through the owner chain — that path reads the stripped ReplicaSets and
Deployments.

docs/security.md §8 states the new boundary and shrank by the lines it grew
(ADR 0044): the mechanism sentences it carried about ADR 0030's byte check and
about why PVC names are read moved back to their ADRs.

go build ./..., go test -race -count=1 ./..., make lint (0 issues),
go vet -tags e2e ./..., gofmt — all green.

🤖 Generated with Claude Code

Invariant 4 ranks "not collected" above "collected and dropped", and
security.md §8 promised env, args and command were "discarded before anything
is stored". They were dropped at the payload seam (ADR 0031 §7) — upstream of
it, every Pod, ReplicaSet, Deployment, StatefulSet, DaemonSet, Job and CronJob
in the cluster sat in memory exactly as the API server returned it.

- One transform on the factory, so it applies to every informer: env, envFrom,
  args and command go from every container list of every pod spec an object
  carries, including init and ephemeral containers and every workload template.
- managedFields and kubectl.kubernetes.io/last-applied-configuration go from
  every object of every kind. The annotation is a verbatim copy of the object
  as applied, so stripping containers alone would have moved the values rather
  than dropped them.
- Not done: informers are not namespace-scoped. The namespace filter supports a
  deny-list, and the opt-out annotations live on objects the agent would then
  stop watching.

Measured on kind 1.36.1 against /api/v1/pods, which is what a watch delivers
and not what `kubectl get -o json` shows: 9 pods, 77,463 B as received, 42,634 B
after the transform. managedFields alone was 39% of it.

Negative checks: unwiring the transform fails the lister test with the env value
printed; removing the CronJob case fails the per-kind test. A 75 s run against
the kind cluster reports 0 errors, 9 pods observed, workload_not_cached 0, and
all four usage signals — the owner chain resolves through the stripped objects.

§8 shrank by the lines it grew, per ADR 0044.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@kfeofantov
kfeofantov merged commit 7e965cb into main Aug 27, 2026
3 checks passed
@kfeofantov
kfeofantov deleted the fix/informer-cache-drops-env branch August 27, 2026 19:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant