Delivery keycloak-bundle final 2026-04-02 (review fixed 2024-04-17)

[dincali]

No description provided.

[Copilot]

Pull request overview

This PR packages a “final” Keycloak bundle delivery by adding OCM signing key material and OCM helper scripts, introducing a Go-based Keycloak CRD operator (with Helm chart + CRDs), and adding deployment/validation utilities, fixtures, and documentation for hardening/operations.

Changes:

• Add OCM signing/validation/transfer tooling plus a committed public verification key under security/.
• Introduce a Keycloak configuration operator (Go + controller-runtime) with Helm chart templates and generated CRDs/RBAC.
• Add Kubernetes manifests, deploy/cleanup utilities, monitoring resources, CI fixtures, and accompanying documentation/contribution metadata.

Reviewed changes

Copilot reviewed 122 out of 124 changed files in this pull request and generated 11 comments.

Show a summary per file

File	Description
security/README.md	Documents OCM signature key usage and rotation.
security/ocm-signing-public-key.pub	Adds the public key used to verify OCM signatures.
scripts/utils/status.sh	Utility to inspect Keycloak namespace, pods, services, DB cluster.
scripts/utils/portforward.sh	Utility to port-forward Keycloak service locally with flexible args.
scripts/utils/logs.sh	Utility to stream logs from Keycloak or CNPG pods by selector.
scripts/utils/common.sh	Shared bash helpers (logging, fail, registry default, suffix generator).
scripts/tests/test-security.sh	Adds static manifest securityContext assertions for CI.
scripts/tests/test-crd.sh	Compatibility wrapper delegating to modular CRD suite runner.
scripts/tests/test-crd-suite.sh	Orchestrates smoke/lifecycle CRD integration suites.
scripts/tests/test-crd-smoke.sh	Smoke suite applying fixtures and verifying resources exist in Keycloak.
scripts/tests/test-crd-lifecycle.sh	Lifecycle suite deleting CRs and verifying removal in Keycloak.
scripts/tests/fixtures/user-ci-test-user.yaml	CI fixture for User CR.
scripts/tests/fixtures/secret-ci-test-user-password.yaml	CI fixture secret for User initial password.
scripts/tests/fixtures/realm-master.yaml	CI fixture for base Realm CR.
scripts/tests/fixtures/README.md	Explains fixture purpose/naming conventions.
scripts/tests/fixtures/identityprovider-ci-test-oidc.yaml	CI fixture for IdentityProvider CR.
scripts/tests/fixtures/group-ci-test-group.yaml	CI fixture for Group CR.
scripts/tests/fixtures/clientscope-ci-test-scope.yaml	CI fixture for ClientScope CR.
scripts/tests/fixtures/client-odc-showcase.yaml	CI fixture for Client CR.
scripts/tests/fixtures/authflow-ci-test-browser-mfa.yaml	CI fixture for AuthFlow CR.
scripts/ocm/ocm-verify.sh	Verifies signed OCM component archives using a public key.
scripts/ocm/ocm-validate.sh	Validates archive contents and optionally verifies signatures/describes.
scripts/ocm/ocm-transfer.sh	Transfers CTF to an OCI registry with overwrite/immutable modes.
scripts/ocm/ocm-sign.sh	Signs component archive (and verifies) using RSA keys.
scripts/ocm/ocm-create.sh	Creates component archive/CTF tarball and generates SBOM.
scripts/deploy/install-prometheus-operator.sh	Installs Prometheus Operator bundle and waits for readiness.
scripts/deploy/install-jaeger.sh	Deploys Jaeger all-in-one into an observability namespace.
scripts/deploy/install-cnpg.sh	Installs CloudNativePG operator and waits for readiness.
scripts/deploy/deploy-postgres.sh	Deploys CNPG cluster and waits for primary readiness.
scripts/deploy/deploy-operator.sh	Installs CRDs and deploys the operator chart, with optional OCM verify gate.
scripts/deploy/deploy-keycloak.sh	Applies Keycloak manifests and waits for readiness.
scripts/deploy/cleanup.sh	Deletes instance namespace with pre-delete CR/finalizer cleanup attempts.
renovate.json	Adds Renovate configuration for images, actions, and custom regex managers.
operator/internal/wrapper/job_runner.go	Adds Job-based config-cli runner and secret/job management logic.
operator/internal/controller/user_controller.go	User controller with finalizer + realm sync trigger behavior.
operator/internal/controller/realm_controller.go	Realm controller running export + config-cli job and status updates.
operator/internal/controller/identityprovider_controller.go	IdentityProvider controller with finalizer + realm sync triggers.
operator/internal/controller/helper.go	Shared finalizer helpers, status setters, and realm sync trigger logic.
operator/internal/controller/group_controller.go	Group controller with finalizer + realm sync triggers.
operator/internal/controller/clientscope_controller.go	ClientScope controller with finalizer + realm sync triggers.
operator/internal/controller/client_controller.go	Client controller (includes confidential client secret generation).
operator/internal/controller/authflow_controller.go	AuthFlow controller with finalizer + realm sync triggers.
operator/hack/boilerplate.go.txt	Adds boilerplate header for generated Go files.
operator/go.mod	Defines operator module and dependencies.
operator/Dockerfile	Builds a distroless, non-root operator container image.
operator/config/rbac/role.yaml	Generated manager ClusterRole rules for operator controllers.
operator/cmd/main.go	Operator entrypoint: manager setup, env config, controller wiring.
operator/api/v1alpha1/user_types.go	Defines User CRD Go types + deepcopy methods and markers.
operator/api/v1alpha1/realm_types.go	Defines Realm CRD Go types + deepcopy methods and markers.
operator/api/v1alpha1/identityprovider_types.go	Defines IdentityProvider CRD Go types + markers.
operator/api/v1alpha1/groupversion_info.go	Registers CRD types with scheme builder.
operator/api/v1alpha1/group_types.go	Defines Group CRD Go types + markers.
operator/api/v1alpha1/common_types.go	CommonStatus type shared across CRDs.
operator/api/v1alpha1/clientscope_types.go	Defines ClientScope CRD Go types + markers.
operator/api/v1alpha1/client_types.go	Defines Client CRD Go types + markers.
operator/api/v1alpha1/authflow_types.go	Defines AuthFlow CRD Go types + markers.
manifests/postgres/cluster.yaml	CNPG Cluster manifest for Keycloak DB.
manifests/monitoring/prometheus.yaml	Prometheus instance, service, and SA for per-namespace monitoring.
manifests/monitoring/keycloak-service-monitor.yaml	ServiceMonitor scraping Keycloak metrics endpoint.
manifests/monitoring/keycloak-prometheus-rules.yaml	PrometheusRule alerts for Keycloak and CNPG metrics.
manifests/monitoring/cnpg-pod-monitor.yaml	PodMonitor scraping CNPG pod metrics.
manifests/keycloak/keycloak-service.yaml	Keycloak Service exposing http and management ports.
manifests/keycloak/keycloak-secret.yaml	Admin credentials Secret for Keycloak bootstrap.
manifests/keycloak/keycloak-sa.yaml	ServiceAccount for Keycloak pods.
manifests/keycloak/keycloak-rbac.yaml	Role/RoleBinding for pod discovery (KUBE_PING).
manifests/keycloak/keycloak-pdb.yaml	PodDisruptionBudget for Keycloak availability during disruptions.
manifests/keycloak/keycloak-deployment.yaml	Keycloak Deployment with securityContext, init wait-for-db, probes, env.
Makefile	Adds generation/build/test targets for operator + chart artifacts.
LICENSE	Adjusts Apache 2.0 boilerplate/footer content.
examples/user-example.yaml	Adds example User CR.
examples/restore-cluster-example.yaml	Adds example CNPG restore cluster manifest.
examples/realm-example.yaml	Adds example Realm CR.
examples/README.md	Describes examples scope and guidance.
examples/keycloak-instance.yaml	Adds example KRO KeycloakInstance manifest.
examples/identityprovider-example.yaml	Adds example IdentityProvider CR.
examples/group-example.yaml	Adds example Group CR.
examples/clientscope-example.yaml	Adds example ClientScope CR.
examples/client-public.yaml	Adds example public Client CR.
examples/client-example.yaml	Adds example confidential Client CR.
examples/backup-example.yaml	Adds example CNPG backup/scheduled backup/objectstore manifests.
examples/authflow-example.yaml	Adds example AuthFlow CR.
docs/HARDENING.md	Adds security hardening reference and accepted deviations.
docs/DATABASE.md	Documents database/operator decision and usage model.
docs/CLIENT.md	Documents declarative configuration approach and operator architecture decisions.
CONTRIBUTING.md	Adds contribution process, branch conventions, and CI gates.
component-constructor.yaml	Defines OCM component resources (images, CRDs, chart, manifests, SBOM).
charts/keycloak-operator/values.yaml	Default Helm values for operator deployment.
charts/keycloak-operator/templates/serviceaccount.yaml	Helm template for operator ServiceAccount.
charts/keycloak-operator/templates/rbac.yaml	Helm template installing RBAC from generated role.yaml.
charts/keycloak-operator/templates/deployment.yaml	Helm template for operator Deployment and env wiring.
charts/keycloak-operator/templates/_helpers.tpl	Helper templates for naming/labels/service account name.
charts/keycloak-operator/files/role.yaml	Bundled RBAC rules file consumed by Helm template.
charts/keycloak-operator/crds/keycloak.opendefense.cloud_users.yaml	Generated User CRD manifest.
charts/keycloak-operator/crds/keycloak.opendefense.cloud_realms.yaml	Generated Realm CRD manifest.
charts/keycloak-operator/crds/keycloak.opendefense.cloud_identityproviders.yaml	Generated IdentityProvider CRD manifest.
charts/keycloak-operator/crds/keycloak.opendefense.cloud_groups.yaml	Generated Group CRD manifest.
charts/keycloak-operator/crds/keycloak.opendefense.cloud_clientscopes.yaml	Generated ClientScope CRD manifest.
charts/keycloak-operator/crds/keycloak.opendefense.cloud_clients.yaml	Generated Client CRD manifest.
charts/keycloak-operator/crds/keycloak.opendefense.cloud_authflows.yaml	Generated AuthFlow CRD manifest.
charts/keycloak-operator/Chart.yaml	Helm chart metadata for operator.
CHANGELOG.md	Adds changelog entries for new features and delivery scope.
.gitignore	Updates ignored files for build artifacts, OCM outputs, secrets, temp files.
.github/workflows/security.yml	Adds scheduled + PR security scanning (Trivy) workflow.
.gitattributes	Enforces LF endings for public key and shell scripts.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[themue]

Fixed findings by Copilot after delivery of final on 2026-04-02.

[trevex]

Puh, sorry for the late review it was hard to find time to go through this huge PR.

I added some comments for potential improvements,.

[dincali]

Thanks for the review.
The requested changes and suggestions for improvment have been implemented in latest commit 5f7e8c4.
See also the corresponding GitHub Actions run.

[christianhuening]

This image is not present in the component-descriptor therefore making a fully ocm based and potentially air-gapped deployment not possible.

[dincali]

Addressed in commit ff23f9c.

The BusyBox image used by the wait-for-db init container is now included in the component descriptor as a digest-pinned external OCM ociImage resource:

• busybox-image OCM resource definition
• Corresponding wait-for-db init-container image reference

To prevent this issue from recurring, an automated air-gapped OCM image-coverage test was added:

• Extraction of runtime image references from manifests and the KRO RGD
• Digest-pinning validation for OCM image resources
• Validation that every static runtime image is covered by an OCM resource
• Integration of the check into the CI quality-check job

The quality-check job, including Air-gapped OCM Image Coverage, passed for this commit:

• CI quality-check result for ff23f9c

This ensures that the BusyBox image is part of the OCM transport set and that missing or non-digest-pinned static runtime images are detected automatically.

[christianhuening]

The RGD does not use KRO CEL Expressions thus treating all {...} as literal static string values. I cannot see how that would even work for a kro based deployment.
Also if I see it correctly, the KRO based deployment path is never tested via the provided test scripts. This needs to be done and the KRO path verified.

[dincali]

Addressed in commit ff23f9c.

The previous Go/Helm-style placeholders, for example {{ .spec.namespace }}, were replaced throughout the RGD with KRO CEL expressions based on schema, for example ${schema.spec.namespace}.

The relevant changes include:

• Namespace and instance-label expressions
• CNPG namespace, instance-count and storage-size expressions
• Keycloak deployment namespace and replica expressions
• Admin-secret CEL expressions in the Keycloak deployment
• Operator image and secret-related CEL expressions
• KRO status aggregation using resource and schema expressions

A dedicated KRO deployment-path verification script was also added:

• Complete KRO deployment test
• Rejection of remaining Go/Helm-style delimiters
• Application of the RGD and creation of a KeycloakInstance
• Verification that KRO creates all expected backing resources
• Validation of dynamically rendered schema values
• Final check of the rendered resources for unresolved delimiters

The test applies the actual RGD to a cluster, creates a minimal KeycloakInstance, waits for the KRO-managed resources and validates that the supplied values were rendered into those resources.

Because this test requires an existing cluster with both KRO and CloudNativePG installed, it is currently a conditional integration test. Its prerequisites and invocation are documented here:

• KRO verification instructions

[shawnsarwar]

@christianhuening the latest commit should address your open issues

[shawnsarwar]

@dincali can you please help make this more comprehensible by tagging the fixes in the latest commit? It's difficult to parse what's been done as the commit is quite noisy.

[dincali]

@shawnsarwar I added the concrete mapping for the fixes from commit ff23f9c to the two review findings:

1. BusyBox/OCM: busybox-image was added to the component descriptor, together with an automated OCM image-coverage test that is executed in CI.

2. KRO: The RGD now consistently uses KRO CEL expressions, including CEL-based status aggregation. A dedicated KRO deployment-path integration test was added as well.

I have provided the detailed line-level references in the corresponding review threads so that each finding can be traced directly to its implementation and verification.