Skip to content

feat(openebs): make the Loki/MinIO/Alloy observability stack toggleable - #155

Merged
patrick-hermann-sva merged 1 commit into
mainfrom
feat/openebs-observability-toggles
Aug 11, 2026
Merged

patrick-hermann-sva merged 1 commit into
mainfrom
feat/openebs-observability-toggles

Conversation

@patrick-hermann-sva

Copy link
Copy Markdown
Contributor

The openebs chart ships an observability stack enabled by default: a Loki StatefulSet, the MinIO StatefulSet backing it, an Alloy DaemonSet, and two extra StorageClasses (openebs-loki-localpv, openebs-minio-localpv). The localpv profile disabled the lvm/zfs/mayastor engines but left all of that on, so every consumer of this helmfile got seven pods of logging nobody asked for.

On a small cluster it is worse than unnecessary: the Loki StatefulSet wants three replicas with pod anti-affinity, so on a 1+2 kind cluster the two workers take replicas 1 and 2, the control-plane node is tainted, and replica 0 sits Pending forever — a permanently non-Running pod that has to be explained to everyone who looks at the cluster. Seen building the u26-kind3 machinery cluster.

Both toggles default to true

Deliberately — this changes nothing for anyone today. This helmfile is consumed unpinned (git::...helm.git, no ref), so flipping a default would reach every cluster on its next apply rather than when someone chose it.

helmfile apply -f infra/openebs.yaml.gotmpl \
  --state-values-set openebs_loki_enabled=false,openebs_alloy_enabled=false

Notes

  • MinIO has no toggle of its own — it is Loki's object store and a sub-dependency, so loki.enabled=false is what removes it.
  • Alloy is turned off alongside rather than separately by default in callers: leaving the log shipper running with no Loki ships logs at nothing.
  • openebs-hostpath is unaffected by either toggle — verified by rendering the helmfile both ways.
default (unchanged) both toggles off
StatefulSets etcd, loki, minio, nats etcd, nats
DaemonSets …, alloy, … (alloy gone)
StorageClasses hostpath, single-replica, mayastor-etcd, loki-localpv, minio-localpv hostpath, single-replica, mayastor-etcd

🤖 Generated with Claude Code

https://claude.ai/code/session_01FSYX5dsdVdzLCuB6xYhaLK

The openebs chart ships an observability stack enabled by default: a Loki
StatefulSet, the MinIO StatefulSet backing it, an Alloy DaemonSet, and two extra
StorageClasses (openebs-loki-localpv, openebs-minio-localpv). The localpv
profile disabled the lvm/zfs/mayastor engines but left all of that on, so every
consumer of this helmfile got seven pods of logging nobody asked for.

On a small cluster it is worse than unnecessary. The Loki StatefulSet wants
three replicas with pod anti-affinity, so on a 1+2 kind cluster the two workers
take replicas 1 and 2, the control-plane node is tainted, and replica 0 sits
Pending forever — a permanently non-Running pod that has to be explained to
everyone who looks at the cluster. Seen on the u26-kind3 machinery cluster.

Both toggles default to TRUE, so this changes nothing for anyone today. That is
deliberate: this helmfile is consumed unpinned (`git::...helm.git`, no ref), so
a default flip would reach every cluster on its next apply rather than when
someone chose it.

Turn off with:
  helmfile apply -f infra/openebs.yaml.gotmpl \
    --state-values-set openebs_loki_enabled=false,openebs_alloy_enabled=false

MinIO has no toggle of its own — it is Loki's object store and a sub-dependency,
so loki.enabled=false is what removes it. openebs-hostpath, the StorageClass the
fleet actually uses, is unaffected by either toggle; verified by rendering the
helmfile both ways.
@patrick-hermann-sva
patrick-hermann-sva merged commit ed1aa3f into main Aug 11, 2026
4 checks passed
patrick-hermann-sva added a commit to stuttgart-things/ansible that referenced this pull request Aug 11, 2026
…y clusters

The openebs chart ships an observability stack enabled by default: a Loki
StatefulSet, the MinIO StatefulSet backing it, an Alloy DaemonSet, and two extra
StorageClasses. The localpv profile this play uses disabled the lvm/zfs/mayastor
engines but left all of that on, so every machinery cluster ran seven pods of
logging nobody asked for.

On a 1+2 kind cluster — which is what every machinery cluster is — it cannot
even run. Loki asks for three replicas with pod anti-affinity: the two workers
take replicas 1 and 2, the control-plane node is tainted, and replica 0 sits
Pending forever. u26-kind3 carried that permanently non-Running pod from the day
it was built, and kind1 has the same one.

MinIO has no toggle of its own — it is Loki's object store and a sub-dependency,
so loki.enabled=false is what removes it. Alloy goes too, since a log shipper
with no Loki ships logs at nothing. openebs-hostpath, the StorageClass
everything here actually uses, is untouched.

To pass the toggles, pre_crossplane_helmfiles entries become dicts with an
optional state_values map rendered into --state-values-set. A plain string still
works, so a caller overriding the list is unaffected.

Requires stuttgart-things/helm#155, which adds the state values (defaulting to
true there, so nothing else changes). Against an older helmfile the flags are
simply ignored and the stack stays up.

Verified on u26-kind3: after the run the openebs namespace holds only
openebs-localpv-provisioner, the StorageClass list is down to openebs-hostpath,
and the cluster has no non-Running pod left. The three MinIO StatefulSet PVCs
survive a Helm removal and were deleted separately.

This branch was previously deployed

1 inactive deployment
k8s — d5474166 Deployed Aug 11, 2026 by patrick-hermann-sva via Analyze-Changed-Charts #246
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants