deploy.sh: pinned Istio 1.24.0 is EOL and breaks on Kubernetes 1.31+ (image volume rejection)

[markjbrown]

Summary

documentdb-playground/multi-cloud-deployment/deploy.sh currently pins:

ISTIO_VERSION="1.24.0"

This release is end-of-life (Istio supported releases) and its bundled Helm charts no longer render cleanly on current istioctl versions. A fresh istioctl install against a Kubernetes 1.33+ cluster fails with:

WARNING: Istio 1.24.0 may be out of support (EOL) already
Error: generate config: helm render: load chart: component does not exist

In addition, the Istio 1.23.x sidecar webhook (which is what gets installed if a user already has istioctl on their PATH from before this script was written) does not understand the Kubernetes OCI image volume source (alpha 1.31, beta 1.33, GA 1.35). When the DocumentDB cnpg-i sidecar plugin attaches the extension binaries via an image: volume, the older Istio injector mutates the spec into something the API server rejects:

spec.volumes[3].image: Forbidden: may not specify more than 1 volume type
spec.containers[1].volumeMounts[5].name: Not found: "istio-envoy"

This silently breaks the whole multi-cloud deployment on K8s 1.31+ ΓÇö CNPG sits in Setting up primary forever because init/join Job pods never get created, and the operator log just loops on Selected PVC is not ready yet.

Repro

1. Provision a multi-cluster setup on Kubernetes 1.33 or newer (e.g. AKS + EKS, both on K8s 1.35).
2. Run deploy.sh end-to-end.
3. Apply documentdb-cluster.yaml.
4. Observe that kubectl get documentdb stays in Setting up primary indefinitely; the underlying cluster.postgresql.cnpg.io reaches Cluster is unrecoverable; events on the cluster show the Forbidden: may not specify more than 1 volume type rejection.

Suggested fix

Bump ISTIO_VERSION in deploy.sh to a current supported release. Istio 1.29.2 is what we verified works end-to-end on AKS + EKS / K8s 1.35 with the canonical IstioOperator spec already in deploy.sh. (1.24.x charts no longer render at all; 1.25.x and 1.26.x are also EOL.)

Happy to send the bump as a PR if helpful.

Workaround (already documented)

I sent #366 alongside this issue, adding a troubleshooting entry to documentdb-playground/multi-cloud-deployment/README.md that walks users through the manual upgrade path until deploy.sh itself is bumped.

Environment

• Kubernetes: AKS 1.35 (eastus2) + EKS 1.35 (us-west-2)
• Fleet hub: KubeFleet
• DocumentDB operator: ghcr.io/documentdb/documentdb-kubernetes-operator/operator:0.2.0
• Istio (broken): 1.23.4 (left over from older install)
• Istio (verified working): 1.29.2

[documentdb-triage-tool]

🤖 Auto-triaged by documentdb-triage-tool.

Applied: ecosystem, bug
Project fields suggested: Component playground · Priority P1 · Effort S · Status Triage
Confidence: 0.92 (llm)

Reasoning

LLM: Pinned EOL Istio version actively breaks multi-cloud deployments on Kubernetes 1.31+ with a clear fix (single-line version bump in deploy.sh).

If a label is wrong, remove it manually and ping @patty-chow so the rules can be tuned. The bot will not re-label items that already have component labels.

[documentdb-triage-tool]

👋 @alaye-ms — this issue has been assigned for 17 days with no activity. Is it still in progress? If you've made progress, please leave a quick update. Otherwise it'll be flagged for re-assignment at the next triage meeting.

[documentdb-triage-tool]

👋 @alaye-ms — this issue has been assigned for 14 days with no activity. Is it still in progress? If you've made progress, please leave a quick update. Otherwise it'll be flagged for re-assignment at the next triage meeting.

[documentdb-triage-tool]

👋 @alaye-ms — this issue has been assigned for 14 days with no activity. Is it still in progress? If you've made progress, please leave a quick update. Otherwise it'll be flagged for re-assignment at the next triage meeting.