Skip to content

feat(k8s): generate NetworkPolicies and per-module ServiceAccounts (#75) - #1022

Merged
RonaldHensbergen merged 6 commits into
mainfrom
feat/75-k8s-networkpolicy-serviceaccounts
Oct 30, 2026
Merged

RonaldHensbergen merged 6 commits into
mainfrom
feat/75-k8s-networkpolicy-serviceaccounts

Conversation

@RonaldHensbergen

Copy link
Copy Markdown
Owner

Closes #75 (remaining scope after the 2026-10-25 re-scope: the Helm renderer already shipped Secret/ConfigMap generation and StatefulSet/PVC support).

What this adds

NetworkPolicy generation from contract boundaries. Every module in a rendered chart now gets templates/<module>-network-policy.yaml:

  • policyTypes: [Ingress, Egress] — default-deny both directions.
  • Allow rules derived from the plan's contract bindings: a binding from consumer to provider yields egress on the consumer's pods and matching ingress on the provider's pods, both scoped to the contract's port. No policy allows traffic a binding does not justify: an unrelated module's policy names no other module.
  • Same-module pods may talk freely (a module is the unit of composition and declares its own internal wiring); cluster DNS egress (kube-system/kube-dns :53 UDP+TCP) is the only other unconditional rule.
  • NodePort/LoadBalancer services are intended entry points: ingress opens to all sources on exactly their exposed ports (and CDS-K8S-007 flags that exposure).
  • A binding whose contract carries no numeric port falls back to the provider's advertised container ports; when no port evidence exists at all the rule stays module-scoped and W128 asks for the port to be declared.

Service accounts and least-privilege RBAC. Every module gets templates/<module>-service-account.yaml and every workload pod references it (serviceAccountName). Least privilege is the default: without declared rules there is no Role, no RoleBinding, and no mounted API token (automountServiceAccountToken: false). Modules opt in via the new implementation.kubernetes.serviceAccount.rbac schema section (rules rendered verbatim into a Role + RoleBinding); malformed rules fail closed as E147 because a plan loaded from a file bypasses the module schema.

Extended security checks. scan_k8s_security gains CDS-K8S-006 (wildcard verbs/resources/apiGroups in RBAC rules, medium) and CDS-K8S-007 (NodePort/LoadBalancer ingress exposure, low).

Verification

  • Full suite via scripts/run_tests_with_deprecation_gate.py: 1942 tests, OK.
  • Coverage gate: 97% overall (cli/k8s_renderer.py 91%, cli/k8s_security.py 98%).
  • markdownlint, yamllint, ruff, pre-commit: pass. Renovate validates with the pinned renovate@44.65.5; the unpinned make lint-renovate (latest Renovate) rejects renovate.json identically on main — pre-existing, untouched here.
  • Golden chart hashes regenerated for the new template files; real-profile render and cds test --target helm produce no new diagnostics versus main.

Follow-ups for the maintainer

  • CI (incl. the k3s e2e lane, if configured to run the helm target) should be green; the chart now contains 8 extra template files for the demo profile.
  • Consider documenting the network-isolation baseline in the docs (the issue notes captured it); nothing in docs/ was updated in this PR.
  • The unpinned-renovate lint failure on main is worth a separate chore (pin the validator version in the Makefile like AGENTS.md does).

Complete the Kubernetes production resources for the helm render target:
- One NetworkPolicy per module, default-deny for ingress and egress, with
  allow rules derived from the plan's contract bindings: consumer egress
  and matching provider ingress on the contract port, same-module traffic,
  cluster DNS egress, and ingress from all sources on exactly the exposed
  ports of NodePort/LoadBalancer services. A binding without port evidence
  falls back to the provider's advertised ports, or warns W128 when the
  rule cannot be port-scoped.
- One ServiceAccount per module; every workload references it. Least
  privilege by default: no Role, no RoleBinding, and no mounted API token
  unless the module declares implementation.kubernetes.serviceAccount.
  rbac rules (new schema section); malformed rules fail closed as E147.
- Extend scan_k8s_security with CDS-K8S-006 (wildcard RBAC grants) and
  CDS-K8S-007 (NodePort/LoadBalancer ingress exposure notes).
- Regenerate the golden chart hashes; add renderer and security tests.

Generated by Mistral Vibe.
Co-Authored-By: Mistral Vibe <vibe@mistral.ai>
…e core API group in RBAC rules (#75)

Self-review fixes for the NetworkPolicy/ServiceAccount PR:
- A plan loaded from a file bypasses the module schema, so a non-mapping
  implementation.kubernetes.serviceAccount block crashed both the helm
  renderer and scan_k8s_security with a raw AttributeError. The renderer
  now fails closed with E147, and the scan ignores the malformed block
  (shape errors are the renderer's diagnostics to report). The
  NetworkPolicy binding derivation and workload surface collection skip
  non-dict consumes entries and service blocks for the same reason.
- module.schema.json rejected apiGroups: [""] (the Kubernetes core
  group, which the schema's own description uses as the example) because
  the items carried minLength: 1; the constraint is dropped for apiGroups
  only, and a schema regression test pins the spelling.

Generated by Mistral Vibe.
Co-Authored-By: Mistral Vibe <vibe@mistral.ai>

@RonaldHensbergen RonaldHensbergen left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

Reviewed the whole diff: the NetworkPolicy generator (_network_policy_manifests + _binding_ports), the ServiceAccount/RBAC path (_rbac_rules, _service_account_manifest), the module schema section, the two new security checks, and the tests. The design is right and the execution is careful: module-granularity policies are the correct unit (superset-init → postgres works without per-workload edges), RBAC fails closed for plans loaded from files, automountServiceAccountToken: false by default, the DNS rule is correct (kube-system + k8s-app: kube-dns, UDP+TCP), edges are sorted for deterministic output, and the test coverage is genuinely thorough. One mechanical blocker (E-code collision with main) and one design gap (no external-egress escape hatch) before merge; the rest is minor.

Blocking: E147 collides with main

cli/planner.py:672 on main (merged in #1020 yesterday) already uses E147 for the object-storage backup same-destination guard. This branch was cut from aa3076b, before that merge, and independently claimed E147 for the malformed-RBAC shape error at cli/k8s_renderer.py:1334/1368. Merging as-is puts two unrelated diagnostics on the same code — the same collision class we just untangled between #951/#952. E148 is free on both main and this branch: please renumber the RBAC shape error (and its tests) to E148 before merge. The CHANGELOG entry doesn't name the code, so nothing else moves.

Major: default-deny egress has no escape hatch for external endpoints

The egress allowlist is derived exclusively from contract bindings + same-module + DNS. Anything else is silently denied at runtime: a dagster user-code asset reading an external API or database (the module's core real-world job), a Superset instance pointed at an external warehouse, or any future helm-renderable module with an external endpoint (the backup modules are compose-only today, so they're unaffected — but the pattern repeats the moment they gain a kubernetes target). Render stays green; workloads time out with no diagnostic to point at the cause. The bundled demo profile passes because it never leaves the cluster, which is exactly why CI won't catch a customer hitting this.

Suggested paths, pick at least (a) for this PR: (a) document the network-isolation baseline and this limitation loudly (docs/kubernetes.md + the README follow-up already noted in the PR description); (b) add an opt-in passthrough, e.g. implementation.kubernetes.networkPolicy.egress rendered verbatim or an allowExternalEgress: true escape; (c) once #951 merges, derive proxy-route targets as additional egress edges so an in-chart Traefik can actually reach what it routes.

Major-adjacent: CDS-K8S-007's recommendation contradicts the generated policies

The finding recommends "prefer ClusterIP with an explicit ingress controller otherwise" — but the NetworkPolicies deny ingress from any controller that is not a bound module peer (nothing can name an external ingress-controller namespace), so the recommended configuration is exactly the one this PR breaks silently. NodePort/LoadBalancer is the only ingress path that works. Either adjust the recommendation text to match the behavior, or provide the ingress-side escape hatch.

Minor

  • cli/k8s_renderer.py:1571 — _binding_ports uses the contract's port (Service-facing) as the NetworkPolicy port, but NetworkPolicy ports match the destination pod port. Bundled modules all have Service port == targetPort == container port, so it works today; a Service with port: X, targetPort: Y remap would generate a rule matching X while pods listen on Y. The fallback path (_container_port) already does this correctly via targetPort. Consider mapping the contract port through the provider's Service (port → targetPort) when one declares it, or document the constraint.
  • cli/k8s_renderer.py:1428 — _workload_surfaces deep-copies and substitutes the whole kubernetes block a second time per module, duplicating the substitution and enabledFrom handling that _render_module already performs. Correct today; just a second code path that has to stay in sync if the workload shape rules change.
  • The RBAC shape error (post-renumber E148) isn't in the README troubleshooting table — the repo convention documents new user-facing E-codes there (E128, E133–E138 rows).

Questions

  • A binding whose contractRef names a compose-only module is silently skipped (provider_id not in module_ids → continue). That's correct for this chart, but should it warn? A consumer that expects to reach a provider which simply isn't rendered will fail at runtime with nothing pointing at the gap.
  • Same-module traffic is unrestricted by design and I agree with the rationale — worth one README line stating it explicitly (a reader will ask why a module's pods can reach each other on any port while everything else is denied).

Notes for the maintainer

  • Merge cleanliness: the branch is based on aa3076b; everything merged since (#1020, #1021, #1024/#1025) touches no demo-chart inputs, so the golden hashes stay valid and a plain merge should be conflict-free — but re-run CI on the merged result.
  • W128 is free on main (main has W127 only) — no collision there.
  • SonarCloud gate: passed.

Comment thread cli/k8s_renderer.py Resolved
Comment thread cli/k8s_renderer.py Resolved
Comment thread cli/k8s_security.py Resolved
Comment thread cli/k8s_renderer.py Resolved
Resolves the PR review findings on #75 in one pass:

- E147 renumbered to E148: main (via #1020) already uses E147 for the
  object-storage backup same-destination guard; two unrelated
  diagnostics must not share a code.
- External-egress/ingress escape hatch: modules can declare custom
  NetworkPolicy allow rules via
  implementation.kubernetes.networkPolicy.ingress/.egress, rendered
  verbatim after the binding-derived rules, for destinations no binding
  can name (an external API a job reads from, a cross-namespace ingress
  controller). Malformed shapes fail closed with E149 (plan-file bypass
  of the module schema), and CDS-K8S-008 records every external-egress
  rule as a low finding so intentional openings stay visible in the
  security scan.
- The contract's Service-facing port is now mapped through the
  provider's Service targetPort, so a `port: X, targetPort: Y` remap
  can no longer silently break a port-scoped rule.
- CDS-K8S-007's recommendation now matches the behavior: an external
  ingress controller needs explicit networkPolicy.ingress rules because
  the generated policies deny it by default.
- Documented the network-isolation baseline in docs/kubernetes.md
  (default-deny posture, binding-derived allows, same-module rationale,
  DNS egress, NodePort/LB exposure, the passthrough and its security
  visibility), updated the stale "Network policy not emitted" boundary
  row, added E148/E149/W128 to the README troubleshooting table, and
  extended the CHANGELOG entry.
- W129 (binding to an unrendered provider) was considered and rejected
  as dead code: E072 already fails the whole Helm render for any plan
  module lacking a Kubernetes implementation, so the silent-skip
  cannot occur.

Generated by Mistral Vibe.
Co-Authored-By: Mistral Vibe <vibe@mistral.ai>
@RonaldHensbergen

Copy link
Copy Markdown
Owner Author

All review findings are resolved in f69bf9f (inline threads replied and resolved individually):

Finding Resolution
Blocking: E147 collides with main (#1020's backup same-destination guard) Renumbered the RBAC shape error to E148 (verified free on both main and this branch); tests updated.
Major: default-deny egress has no escape hatch for external endpoints New implementation.kubernetes.networkPolicy.ingress/.egress opt-in passthrough, rendered verbatim after the binding-derived rules; malformed shapes fail closed with E149 (plan-file bypass class, same as RBAC); CDS-K8S-008 records every external-egress rule as a low finding so intentional openings stay visible in cds security; the baseline and the limitation are documented in docs/kubernetes.md ("Network isolation and least privilege").
Major-adjacent: CDS-K8S-007's recommendation contradicts the policies Recommendation now says an external ingress controller needs explicit networkPolicy.ingress rules "since the generated NetworkPolicy denies it by default" -- and with the passthrough implemented, that recommendation is actionable.
Minor: contract port (Service-facing) vs pod port in NetworkPolicy rules _workload_surfaces now collects each Service's port -> targetPort mapping and _binding_ports maps the contract port through it; a port: 80, targetPort: 8080 remap generates rules for 8080. Golden hashes unchanged (bundled modules keep port == targetPort).
Minor: RBAC E-code missing from README troubleshooting E148, E149, and W128 rows added.
Question: silent skip of bindings to non-rendered providers Investigated and rejected as dead code: E072 already fails the whole Helm render for any plan module lacking a Kubernetes implementation, so the scenario cannot occur -- no W129 was added, and the generator docstring records why.
Question: same-module traffic rationale undocumented Stated explicitly in the new docs/kubernetes.md section.

Notes:

  • One review observation was wrong on inspection and has been corrected: I initially drafted a W129 warning for unrendered providers, but the E072 check makes it unreachable -- the finding that motivated the question is already covered by existing behavior.
  • The stale "Network policy: not emitted" row in docs/kubernetes.md's translation-boundaries table was updated to match the PR's behavior.
  • The _workload_surfaces duplicate-substitution maintainability note is intentionally left as-is (correct today; noted in the review).
  • Follow-up for a future PR: once feat(security): reverse-proxy TLS metadata and contract-driven dynamic routes #951 merges, derive reverse-proxy route targets as additional egress edges so an in-chart Traefik reaches what it routes without a passthrough.

Verification: full offline suite 1953 tests with only the documented pre-existing local SQLAlchemy error (cds-runtime extra absent locally; skips cleanly in CI) and 43 Docker-gated skips; ruff, yamllint, markdownlint clean; golden chart hashes unchanged (the port mapping and passthrough are no-ops for the demo profile); new tests cover passthrough rendering, both E149 shapes, the targetPort remap, selector-scoped vs recorded external egress, and the scan's tolerance of malformed blocks.

RonaldHensbergen added a commit that referenced this pull request Oct 30, 2026
…les, add README row

Review findings on PR #1028, all fixed in this commit:

- Blocking: E147 collides with open PR #1022 (k8s serviceAccount/RBAC
  checks use E147-E149; main stops at E146). Renamed the new
  provided-contract host diagnostic to E150 everywhere (renderer,
  utils docstring, docs/modules.md, CHANGELOG, tests).
- Major: the single-service migration missed modules whose compose
  services include auxiliary job containers (postgres: postgres +
  postgres-backup/postgres-restore; superset: superset + superset-init),
  because module_dns_host only handled len(services)==1. Extended the
  rule: when a namesake compose service matches the module's own name,
  the dns placeholder resolves to its renderer compose name
  (<id>-<namesake> for a non-convention id). Migrated the postgres
  sql-database host/connectionUri and the superset http-service host to
  the dns placeholder; verified end-to-end that id: analytics-db now
  yields host analytics-db-postgres and renders with zero errors.
  Convention ids are unaffected (all checked-in profiles use
  id == module name).
- Minor: added the E150 row to the README troubleshooting table next
  to E136.

Tests: 21 tests in test_contract_host_resolution.py (namesake-rule
unit tests, postgres-style resolution test, module.yaml guard test
asserting checked contract kinds use the dns placeholder); full suite
1911 passed / 43 env-gated skips; ruff, yamllint, markdownlint clean.

Generated by Mistral Vibe.
Co-Authored-By: Mistral Vibe <vibe@mistral.ai>
The entry read as purely additive, but existing --target helm
deployments pick up the default-deny baseline on their next render,
silently cutting external and cross-namespace traffic until
passthrough rules are declared. Marks the entry per the CHANGELOG's
existing **Breaking:** convention and adds the migration pointer to
docs/kubernetes.md.

Generated by Mistral Vibe.
Co-Authored-By: Mistral Vibe <vibe@mistral.ai>
@RonaldHensbergen

Copy link
Copy Markdown
Owner Author

Follow-ups from the review are now tracked as issues instead of comment notes, per the repo's loose-end convention (all inherit #75's milestone):

Also in bfa7c9d: the CHANGELOG entry is now marked Breaking (for existing --target helm deployments) with the migration pointer, since existing charts pick up the default-deny baseline on their next render.

RonaldHensbergen added a commit that referenced this pull request Oct 30, 2026
…rule (#1028)

* fix(contracts): resolve contract DNS hosts via the renderer's naming rule (#1010)

Provided contracts substituted ${service.host} with the module instance
id while the renderer names each compose service through its
service-naming rule, so any profile instantiating a module with an id
different from its service name produced contract endpoints no sibling
container could resolve. The interpolation vocabulary now separates the
two roles: ${service.host} keeps resolving to the instance id (name
prefixes such as backup volume names depend on it), while the new
${service.dns} resolves to the Compose DNS name (the renderer's compose
service name for single-service modules; the explicit serviceName
mechanism keeps covering multi-service modules). The single-service
modules advertising network endpoints (cache, identity provider,
reverse proxy, object storage) build them from ${service.dns}, and the
render cross-checks every provided http-service, sql-database,
cache-service, and object-storage contract's advertised host against
the compose services it actually produced, failing with E147 instead
of shipping a broken endpoint.

Generated by Mistral Vibe.
Co-Authored-By: Mistral Vibe <vibe@mistral.ai>

* review(#1028): rename diagnostic to E150, cover namesake-service modules, add README row

Review findings on PR #1028, all fixed in this commit:

- Blocking: E147 collides with open PR #1022 (k8s serviceAccount/RBAC
  checks use E147-E149; main stops at E146). Renamed the new
  provided-contract host diagnostic to E150 everywhere (renderer,
  utils docstring, docs/modules.md, CHANGELOG, tests).
- Major: the single-service migration missed modules whose compose
  services include auxiliary job containers (postgres: postgres +
  postgres-backup/postgres-restore; superset: superset + superset-init),
  because module_dns_host only handled len(services)==1. Extended the
  rule: when a namesake compose service matches the module's own name,
  the dns placeholder resolves to its renderer compose name
  (<id>-<namesake> for a non-convention id). Migrated the postgres
  sql-database host/connectionUri and the superset http-service host to
  the dns placeholder; verified end-to-end that id: analytics-db now
  yields host analytics-db-postgres and renders with zero errors.
  Convention ids are unaffected (all checked-in profiles use
  id == module name).
- Minor: added the E150 row to the README troubleshooting table next
  to E136.

Tests: 21 tests in test_contract_host_resolution.py (namesake-rule
unit tests, postgres-style resolution test, module.yaml guard test
asserting checked contract kinds use the dns placeholder); full suite
1911 passed / 43 env-gated skips; ruff, yamllint, markdownlint clean.

Generated by Mistral Vibe.
Co-Authored-By: Mistral Vibe <vibe@mistral.ai>

---------

Co-authored-by: Mistral Vibe <vibe@mistral.ai>
@RonaldHensbergen

Copy link
Copy Markdown
Owner Author

Heads-up na de merge van #1028: PR #1020 (nu op main als 751fa72) heeft inmiddels E147 geclaimd in cli/planner.py (backup same-source object-storage destination check). Jullie PR gebruikt E147-E149 voor de k8s serviceAccount/RBAC-checks, dus E147 botst nu met main. Bij rebase van deze branch zal de k8s-check naar een vrij code moeten (E151+ lijkt vrij op main; #1028 gebruikt E150).

@sonarqubecloud

Copy link
Copy Markdown

@RonaldHensbergen
RonaldHensbergen merged commit 215e825 into main Oct 30, 2026
15 checks passed
@RonaldHensbergen
RonaldHensbergen deleted the feat/75-k8s-networkpolicy-serviceaccounts branch October 30, 2026 22:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implement K8s secrets, configmaps, network policies, and stateful resources

1 participant