From 2dc6897b4c2e3a25aad8d26c26fd68f70f5c93d3 Mon Sep 17 00:00:00 2001 From: Paul Wright Date: Wed, 8 Apr 2026 14:54:39 +0100 Subject: [PATCH 1/5] certs draft --- input/kube-yaml/site-linking.md | 158 +++++++++++++++++++++++++++++++- 1 file changed, 157 insertions(+), 1 deletion(-) diff --git a/input/kube-yaml/site-linking.md b/input/kube-yaml/site-linking.md index 698abe09..748b3565 100644 --- a/input/kube-yaml/site-linking.md +++ b/input/kube-yaml/site-linking.md @@ -113,4 +113,160 @@ A connecting site redeems this token for a `Link` resource to establish a link t kubectl get link NAME STATUS REMOTE SITE MESSAGE token-to-west Ready my-site OK - ``` \ No newline at end of file + ``` + + +## Linking sites using `Link` resources and custom certificates + +By default, the site CA `skupper-site-ca` issues the `skupper-site-server` certificate that authenticates incoming links. +If you need to use your own server certificate, you can provide custom TLS credentials and then create a `Link` resource for remote sites. + +**Prerequisites** + +* Two sites +* The listening site must have `link-access` enabled +* A server certificate and key for the listening site +* `jq` and `yq` installed if you want to use the command examples to generate the `Link` resource + +To link sites using custom certificates, you provide a custom server certificate on the listening site and create a `Link` resource on the connecting site that references matching client credentials. + +**Procedure** + +1. On the listening site, create a secret named `skupper-site-server`, for example: + ```yaml + apiVersion: v1 + kind: Secret + metadata: + name: skupper-site-server + data: + ca.crt: LS0tLS1C...redacted + tls.crt: LS0tLS1C...redacted + tls.key: LS0tLS1C...redacted + ``` + Apply the secret: + ```shell + kubectl apply -f skupper-site-server.yaml + ``` + The certificate specified in `tls.crt` must be valid for the hostname or IP address that connecting sites use to connect to this site. + +2. Determine the hostname or IP address for the listening site: + ```shell + kubectl get site -o json | jq -r '.status.endpoints[0].host' + ``` + For example, the output might be: + ``` + skupper.public.host + ``` + If the site is already running, you can confirm that Skupper detected the custom server certificate: + ```shell + kubectl get certificate skupper-site-server -o json | jq -r '.status.conditions[].message' + ``` + The output includes a message similar to: + ``` + Secret exists but is not controlled by skupper + ``` + +3. On the listening site, create client credentials for the connecting site. + + If the listening site provides the `skupper-site-ca` issuer, create a `Certificate` resource so that Skupper generates a client secret named `skupper-link`: + ```yaml + apiVersion: skupper.io/v2alpha1 + kind: Certificate + metadata: + name: skupper-link + spec: + ca: skupper-site-ca + client: true + subject: skupper.public.host + ``` + Apply the resource: + ```shell + kubectl apply -f skupper-link-certificate.yaml + ``` + Save the generated secret to a local file for use in the next step: + ```shell + kubectl get secret skupper-link -o yaml | yq -y 'del(.metadata.namespace, .metadata.creationTimestamp, .metadata.resourceVersion, .metadata.uid, .metadata.managedFields)' > client-secret.yaml + ``` + + If you are providing the client certificate yourself, create a secret named `skupper-link`, for example: + ```yaml + apiVersion: v1 + kind: Secret + metadata: + name: skupper-link + data: + ca.crt: LS0tLS1C...redacted + tls.crt: LS0tLS1C...redacted + tls.key: LS0tLS1C...redacted + ``` + Save this resource locally as `client-secret.yaml` if you want to combine it with a `Link` resource in a single file. + +4. On the listening site, create a `Link` resource YAML file. + + To generate the `Link` resource using `kubectl`, `jq`, and `yq`, retrieve the site endpoints and write the YAML file: + ```shell + endpoints="$(kubectl get site -o json | jq -c '.status.endpoints')" + cat < west-link.yaml + apiVersion: skupper.io/v2alpha1 + kind: Link + metadata: + name: skupper-link + spec: + cost: 1 + tlsCredentials: skupper-link + EOF + ``` + If you created a local client secret file, combine it with the `Link` resource: + ```shell + printf '%s\n' '---' >> west-link.yaml + cat client-secret.yaml >> west-link.yaml + ``` + + To create the YAML manually, first inspect the endpoints: + ```shell + kubectl get site -o yaml | yq -y '.status.endpoints' + ``` + Then create a file that contains both the `Link` resource and the client `Secret`, for example: + ```yaml + --- + apiVersion: skupper.io/v2alpha1 + kind: Link + metadata: + name: skupper-link + spec: + cost: 1 + tlsCredentials: skupper-link + endpoints: + - group: skupper-router + host: skupper.public.host + name: inter-router + port: "55671" + - group: skupper-router + host: skupper.public.host + name: edge + port: "45671" + --- + apiVersion: v1 + kind: Secret + metadata: + name: skupper-link + data: + ca.crt: LS0tLS1C...redacted + tls.crt: LS0tLS1C...redacted + tls.key: LS0tLS1C...redacted + ``` + +5. Securely transfer the `Link` resource YAML file to the context of the connecting site. + If you have both sites available from your terminal session, this step is not required. + + **📌 NOTE** + Access to this file provides access to the application network. + Protect it appropriately. + +6. On the connecting site, apply the YAML file and check status: + ```shell + kubectl apply -f west-link.yaml + kubectl get link + NAME STATUS REMOTE SITE MESSAGE + skupper-link Ready my-site OK + ``` From 969b8d1b0a9b7cfe10875a0ee759c1a358a36aa9 Mon Sep 17 00:00:00 2001 From: Paul Wright Date: Thu, 9 Apr 2026 10:00:07 +0100 Subject: [PATCH 2/5] update --- input/overview/security.md | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/input/overview/security.md b/input/overview/security.md index 740e7936..737af5f2 100644 --- a/input/overview/security.md +++ b/input/overview/security.md @@ -22,3 +22,21 @@ Each router is uniquely identified by its own certificate. This means that the Skupper network is isolated from external access, preventing security risks such as lateral attacks, malware infestations, and data exfiltration. +**Certificates and trust between sites** + +On Kubernetes, each site has its own certificate authority for inter-site communication. +By default, Skupper generates the certificates needed to identify the router and to authenticate incoming and outgoing links. + +When two sites are linked, each router verifies the certificate presented by the remote router and checks that it is signed by a trusted CA. +This mutual TLS exchange ensures that only sites with valid credentials can join the application network. + +If your environment requires certificates issued by your own PKI, you can replace the default server certificate that a site uses for incoming links and provide matching client credentials for connecting sites. +This allows you to keep Skupper's mutual TLS model while aligning certificate management with your organization's security requirements. + +**Service level TLS** + +Skupper always encrypts traffic between sites. +This protects traffic carried across the application network, even when workloads run in different clusters or clouds. + +This inter-site encryption is separate from any TLS configuration used by your application services. +If a backend service also requires TLS, you can configure listener and connector resources with appropriate credentials for the application side of the connection. From 9ac8bdd3141c4f5b25eaaff8bb939f99607f38c4 Mon Sep 17 00:00:00 2001 From: Paul Wright Date: Tue, 14 Apr 2026 11:34:24 +0100 Subject: [PATCH 3/5] test --- input/kube-yaml/site-linking.md | 24 +- tests/README.md | 98 +++++ tests/minikube-custom-linking.sh | 687 +++++++++++++++++++++++++++++++ 3 files changed, 799 insertions(+), 10 deletions(-) create mode 100644 tests/README.md create mode 100755 tests/minikube-custom-linking.sh diff --git a/input/kube-yaml/site-linking.md b/input/kube-yaml/site-linking.md index 748b3565..babf778e 100644 --- a/input/kube-yaml/site-linking.md +++ b/input/kube-yaml/site-linking.md @@ -185,7 +185,7 @@ To link sites using custom certificates, you provide a custom server certificate ``` Save the generated secret to a local file for use in the next step: ```shell - kubectl get secret skupper-link -o yaml | yq -y 'del(.metadata.namespace, .metadata.creationTimestamp, .metadata.resourceVersion, .metadata.uid, .metadata.managedFields)' > client-secret.yaml + kubectl get secret skupper-link -o yaml | yq eval -o=yaml 'del(.metadata.namespace, .metadata.creationTimestamp, .metadata.resourceVersion, .metadata.uid, .metadata.managedFields)' - > client-secret.yaml ``` If you are providing the client certificate yourself, create a secret named `skupper-link`, for example: @@ -206,14 +206,18 @@ To link sites using custom certificates, you provide a custom server certificate To generate the `Link` resource using `kubectl`, `jq`, and `yq`, retrieve the site endpoints and write the YAML file: ```shell endpoints="$(kubectl get site -o json | jq -c '.status.endpoints')" - cat < west-link.yaml - apiVersion: skupper.io/v2alpha1 - kind: Link - metadata: - name: skupper-link - spec: - cost: 1 - tlsCredentials: skupper-link + cat < west-link.yaml + { + "apiVersion": "skupper.io/v2alpha1", + "kind": "Link", + "metadata": { + "name": "skupper-link" + }, + "spec": { + "cost": 1, + "tlsCredentials": "skupper-link" + } + } EOF ``` If you created a local client secret file, combine it with the `Link` resource: @@ -224,7 +228,7 @@ To link sites using custom certificates, you provide a custom server certificate To create the YAML manually, first inspect the endpoints: ```shell - kubectl get site -o yaml | yq -y '.status.endpoints' + kubectl get site -o yaml | yq eval -o=yaml '.status.endpoints' - ``` Then create a file that contains both the `Link` resource and the client `Secret`, for example: ```yaml diff --git a/tests/README.md b/tests/README.md new file mode 100644 index 00000000..87c24fe6 --- /dev/null +++ b/tests/README.md @@ -0,0 +1,98 @@ +# Tests + +This directory contains ad hoc test helpers for validating documentation procedures locally. + +## Minikube custom-link test + +`minikube-custom-linking.sh` exercises the Kubernetes YAML workflow for linking two Skupper sites with: + +* two Minikube profiles +* a custom `skupper-site-server` certificate on the listening site +* a manually generated client `Secret` +* a `Link` YAML file assembled with `kubectl`, `jq`, and `yq` +* a V2 `Connector` and `Listener` pair to validate service exposure + +The script is intended for Fedora and assumes: + +* `minikube` +* `kubectl` +* `jq` +* `mikefarah/yq` v4 +* `openssl` +* access to download the Skupper install manifest unless Skupper is already installed + +### What it does + +1. Starts Minikube profiles `west` and `east` unless `START_PROFILES=0`. +2. Installs Skupper automatically with `kubectl apply -f https://skupper.io/install.yaml` if the CRDs are missing. +3. Creates `Site` resources in namespaces `west` and `east`. +4. Waits for both sites to reach the `Running` condition. +5. Starts `minikube tunnel -p west` automatically and writes its output to a log file. +6. Waits for the `west` site to become fully `Ready` and expose a listening endpoint. +7. Generates a temporary CA, server certificate, and client certificate. +8. Applies a custom `skupper-site-server` secret on the listening site. +9. Builds `west-link.yaml` using the same `jq` and `yq` pattern described in the docs. +10. Applies the generated link bundle in the `east` namespace and waits for the link to become `Ready`. +11. Deploys `quay.io/skupper/hello-world-backend` on the `east` site. +12. Applies a V2 `Connector` on `east` and a V2 `Listener` on `west`. +13. Starts a probe pod on `west` and fetches `http://east-backend:8080/` through the application network. +14. Pauses at a few checkpoints so you can inspect the clusters before continuing. + +### Usage + +Start the test: + +```bash +./tests/minikube-custom-linking.sh +``` + +If the Minikube profiles already exist: + +```bash +START_PROFILES=0 ./tests/minikube-custom-linking.sh +``` + +Skip automatic tunnel startup: + +```bash +START_TUNNEL=0 ./tests/minikube-custom-linking.sh +``` + +Disable the interactive checkpoints: + +```bash +PAUSE_ON_CHECKPOINTS=0 ./tests/minikube-custom-linking.sh +``` + +Disable the pause before exit on failures: + +```bash +PAUSE_ON_ERROR=0 ./tests/minikube-custom-linking.sh +``` + +Skip automatic Skupper installation: + +```bash +INSTALL_SKUPPER=0 ./tests/minikube-custom-linking.sh +``` + +Clean up the namespaces, profiles, and generated files: + +```bash +./tests/minikube-custom-linking.sh cleanup +``` + +### Notes + +* By default the script starts `minikube tunnel -p west` itself without `sudo`. Set `TUNNEL_WITH_SUDO=1` only if your local Minikube setup requires it. +* By default the script lets Minikube auto-detect the driver. Set `MINIKUBE_DRIVER` only if you need to force a specific driver on your machine. +* By default the script installs Skupper from `https://skupper.io/install.yaml` if the CRDs are missing. Override `SKUPPER_INSTALL_URL` if you want to use a different manifest. +* The script writes generated YAML manifests to `output/tests/minikube-custom-linking` unless `OUTPUT_DIR` is set. +* The script also saves the `Link` generation intermediates there, including the raw endpoint JSON, the JSON template, and the rendered JSON before conversion to YAML. +* The script writes transient files such as certificates, logs, and traffic output to `/tmp/skupper-docs-minikube-custom-linking` unless `WORKDIR` is set. +* The tunnel output is written to `${WORKDIR}/minikube-tunnel.log`. +* By default the script pauses for Enter at a few Skupper-related checkpoints, including after both sites are running, after the link is ready, and after service traffic is working. Set `PAUSE_ON_CHECKPOINTS=0` to disable that behavior. +* By default the script also pauses before exiting on an error so you can inspect the cluster state while the resources and tunnel are still up. Set `PAUSE_ON_ERROR=0` to disable that behavior. +* The script validates the manual client-secret path from the YAML documentation. It does not exercise the `Certificate` resource variant. +* By default the traffic check uses `busybox:1.36` and `wget`. Override `PROBE_POD_IMAGE` if you need a different probe image in your environment. +* `cleanup` removes the Minikube profiles and the temporary work directory, but it does not remove the saved YAML files under `output/`. diff --git a/tests/minikube-custom-linking.sh b/tests/minikube-custom-linking.sh new file mode 100755 index 00000000..dc4d4e79 --- /dev/null +++ b/tests/minikube-custom-linking.sh @@ -0,0 +1,687 @@ +#!/usr/bin/env bash + +set -euo pipefail + +WEST_PROFILE="${WEST_PROFILE:-west}" +EAST_PROFILE="${EAST_PROFILE:-east}" +WEST_NAMESPACE="${WEST_NAMESPACE:-west}" +EAST_NAMESPACE="${EAST_NAMESPACE:-east}" +WORKDIR="${WORKDIR:-/tmp/skupper-docs-minikube-custom-linking}" +OUTPUT_DIR="${OUTPUT_DIR:-output/tests/minikube-custom-linking}" +START_PROFILES="${START_PROFILES:-1}" +START_TUNNEL="${START_TUNNEL:-1}" +TUNNEL_WITH_SUDO="${TUNNEL_WITH_SUDO:-0}" +PAUSE_ON_CHECKPOINTS="${PAUSE_ON_CHECKPOINTS:-1}" +PAUSE_ON_ERROR="${PAUSE_ON_ERROR:-1}" +INSTALL_SKUPPER="${INSTALL_SKUPPER:-1}" +SKUPPER_INSTALL_URL="${SKUPPER_INSTALL_URL:-https://skupper.io/install.yaml}" +MINIKUBE_DRIVER="${MINIKUBE_DRIVER:-}" +MINIKUBE_CPUS="${MINIKUBE_CPUS:-4}" +MINIKUBE_MEMORY="${MINIKUBE_MEMORY:-8192}" +TUNNEL_PID="" +TUNNEL_LOG="" +PROBE_POD_IMAGE="${PROBE_POD_IMAGE:-busybox:1.36}" + +log() { + printf '==> %s\n' "$*" +} + +fail() { + printf 'ERROR: %s\n' "$*" >&2 + + if [[ "$PAUSE_ON_ERROR" == "1" && -t 0 ]]; then + printf 'Press Enter to exit so you can inspect the cluster state... ' >&2 + read -r _ + fi + + exit 1 +} + +need_cmd() { + command -v "$1" >/dev/null 2>&1 || fail "Missing required command: $1" +} + +pause_checkpoint() { + local message="$1" + + if [[ "$PAUSE_ON_CHECKPOINTS" != "1" || ! -t 0 ]]; then + return + fi + + printf '\nCheckpoint: %s\n' "$message" + printf 'Press Enter to continue... ' + read -r _ +} + +west_kubectl() { + kubectl --context="$WEST_PROFILE" --namespace="$WEST_NAMESPACE" "$@" +} + +east_kubectl() { + kubectl --context="$EAST_PROFILE" --namespace="$EAST_NAMESPACE" "$@" +} + +skupper_api_available() { + local context="$1" + kubectl --context="$context" api-resources --api-group=skupper.io 2>/dev/null | grep -q '^sites' +} + +start_profiles() { + if [[ "$START_PROFILES" != "1" ]]; then + log "Skipping minikube start because START_PROFILES=$START_PROFILES" + return + fi + + local west_start_args=( + -p "$WEST_PROFILE" + --cpus="$MINIKUBE_CPUS" + --memory="$MINIKUBE_MEMORY" + ) + local east_start_args=( + -p "$EAST_PROFILE" + --cpus="$MINIKUBE_CPUS" + --memory="$MINIKUBE_MEMORY" + ) + + if [[ -n "$MINIKUBE_DRIVER" ]]; then + west_start_args+=(--driver="$MINIKUBE_DRIVER") + east_start_args+=(--driver="$MINIKUBE_DRIVER") + fi + + log "Starting minikube profile $WEST_PROFILE" + minikube start "${west_start_args[@]}" + + log "Starting minikube profile $EAST_PROFILE" + minikube start "${east_start_args[@]}" +} + +check_skupper_api() { + log "Checking that the Skupper API is available on both clusters" + + if ! skupper_api_available "$WEST_PROFILE"; then + if [[ "$INSTALL_SKUPPER" == "1" ]]; then + install_skupper "$WEST_PROFILE" + else + fail "Skupper CRDs not found in context $WEST_PROFILE. Install the Skupper V2 controller in that cluster first." + fi + fi + + if ! skupper_api_available "$EAST_PROFILE"; then + if [[ "$INSTALL_SKUPPER" == "1" ]]; then + install_skupper "$EAST_PROFILE" + else + fail "Skupper CRDs not found in context $EAST_PROFILE. Install the Skupper V2 controller in that cluster first." + fi + fi +} + +install_skupper() { + local context="$1" + local deadline + deadline=$((SECONDS + 300)) + + log "Installing Skupper in context $context from $SKUPPER_INSTALL_URL" + kubectl --context="$context" apply -f "$SKUPPER_INSTALL_URL" + + log "Waiting for Skupper CRDs in context $context" + while (( SECONDS < deadline )); do + if skupper_api_available "$context"; then + return + fi + sleep 5 + done + + fail "Timed out waiting for Skupper CRDs after install in context $context" +} + +prepare_workdir() { + rm -rf "$WORKDIR" + mkdir -p "$WORKDIR" + mkdir -p "$OUTPUT_DIR" +} + +deploy_sites() { + log "Creating namespaces" + kubectl --context="$WEST_PROFILE" create namespace "$WEST_NAMESPACE" --dry-run=client -o yaml | kubectl --context="$WEST_PROFILE" apply -f - + kubectl --context="$EAST_PROFILE" create namespace "$EAST_NAMESPACE" --dry-run=client -o yaml | kubectl --context="$EAST_PROFILE" apply -f - + + cat > "$OUTPUT_DIR/west-site.yaml" < "$OUTPUT_DIR/east-site.yaml" <&2 + kubectl --context="$context" --namespace="$namespace" get site -o yaml >&2 || true + kubectl --context="$context" --namespace="$namespace" get pods >&2 || true + kubectl --context="$context" --namespace="$namespace" describe site "$namespace" >&2 || true + kubectl --context="$context" --namespace="$namespace" get events --sort-by=.lastTimestamp >&2 || true +} + +wait_for_site() { + local context="$1" + local namespace="$2" + local deadline + deadline=$((SECONDS + 300)) + + log "Waiting for site $namespace in context $context" + while (( SECONDS < deadline )); do + if kubectl --context="$context" --namespace="$namespace" get site "$namespace" >/dev/null 2>&1; then + local status + status="$(kubectl --context="$context" --namespace="$namespace" get site "$namespace" --no-headers 2>/dev/null | awk '{print $2}')" + if [[ "$status" == "Ready" ]]; then + return + fi + fi + sleep 5 + done + + dump_site_diagnostics "$context" "$namespace" + fail "Timed out waiting for site $namespace in context $context" +} + +wait_for_site_running() { + local context="$1" + local namespace="$2" + local deadline + deadline=$((SECONDS + 300)) + + log "Waiting for site $namespace in context $context to be Running" + while (( SECONDS < deadline )); do + if kubectl --context="$context" --namespace="$namespace" get site "$namespace" >/dev/null 2>&1; then + local running + running="$(kubectl --context="$context" --namespace="$namespace" get site "$namespace" -o json | jq -r '.status.conditions[]? | select(.type=="Running") | .status')" + if [[ "$running" == "True" ]]; then + return + fi + fi + sleep 5 + done + + dump_site_diagnostics "$context" "$namespace" + fail "Timed out waiting for site $namespace in context $context to be Running" +} + +prompt_for_tunnel() { + if [[ "$START_TUNNEL" != "1" ]]; then + return + fi + + if [[ "$TUNNEL_WITH_SUDO" == "1" && "$EUID" -ne 0 ]]; then + log "Requesting sudo access for minikube tunnel" + sudo -v + fi + + TUNNEL_LOG="$WORKDIR/minikube-tunnel.log" + + log "Starting minikube tunnel for profile $WEST_PROFILE" + if [[ "$TUNNEL_WITH_SUDO" == "1" && "$EUID" -ne 0 ]]; then + sudo --preserve-env=HOME,PATH minikube tunnel -p "$WEST_PROFILE" >"$TUNNEL_LOG" 2>&1 & + else + minikube tunnel -p "$WEST_PROFILE" >"$TUNNEL_LOG" 2>&1 & + fi + TUNNEL_PID=$! + + sleep 3 + if ! kill -0 "$TUNNEL_PID" >/dev/null 2>&1; then + cat "$TUNNEL_LOG" >&2 || true + fail "Failed to start minikube tunnel for profile $WEST_PROFILE" + fi + + cat </dev/null 2>&1; then + log "Stopping minikube tunnel" + kill "$TUNNEL_PID" >/dev/null 2>&1 || true + wait "$TUNNEL_PID" 2>/dev/null || true + fi +} + +cleanup_probe_pod() { + west_kubectl delete pod traffic-check --ignore-not-found=true >/dev/null 2>&1 || true +} + +wait_for_endpoint_host() { + local deadline + deadline=$((SECONDS + 300)) + + log "Waiting for a listening endpoint on ${WEST_PROFILE}/${WEST_NAMESPACE}" + while (( SECONDS < deadline )); do + WEST_HOST="$(west_kubectl get site "$WEST_NAMESPACE" -o json | jq -r '.status.endpoints[0].host // empty')" + if [[ -n "$WEST_HOST" && "$WEST_HOST" != "null" ]]; then + export WEST_HOST + log "Using listening endpoint host: $WEST_HOST" + return + fi + sleep 5 + done + + west_kubectl get site "$WEST_NAMESPACE" -o yaml || true + fail "Timed out waiting for the listening endpoint. Ensure minikube tunnel is running for profile $WEST_PROFILE." +} + +write_openssl_config() { + local san_type="$1" + local san_value="$2" + + cat > "$WORKDIR/server-ext.cnf" < "$WORKDIR/client-ext.cnf" </dev/null 2>&1 + openssl req -x509 -new -nodes -key "$WORKDIR/ca.key" -sha256 -days 365 \ + -subj "/CN=skupper-docs-test-ca" \ + -out "$WORKDIR/ca.crt" >/dev/null 2>&1 + + openssl genrsa -out "$WORKDIR/server.key" 2048 >/dev/null 2>&1 + openssl req -new -key "$WORKDIR/server.key" \ + -config "$WORKDIR/server-ext.cnf" \ + -out "$WORKDIR/server.csr" >/dev/null 2>&1 + openssl x509 -req -in "$WORKDIR/server.csr" \ + -CA "$WORKDIR/ca.crt" -CAkey "$WORKDIR/ca.key" -CAcreateserial \ + -out "$WORKDIR/server.crt" -days 365 -sha256 \ + -extfile "$WORKDIR/server-ext.cnf" -extensions v3_server >/dev/null 2>&1 + + openssl genrsa -out "$WORKDIR/client.key" 2048 >/dev/null 2>&1 + openssl req -new -key "$WORKDIR/client.key" \ + -config "$WORKDIR/client-ext.cnf" \ + -out "$WORKDIR/client.csr" >/dev/null 2>&1 + openssl x509 -req -in "$WORKDIR/client.csr" \ + -CA "$WORKDIR/ca.crt" -CAkey "$WORKDIR/ca.key" -CAcreateserial \ + -out "$WORKDIR/client.crt" -days 365 -sha256 \ + -extfile "$WORKDIR/client-ext.cnf" -extensions v3_client >/dev/null 2>&1 +} + +write_secret_manifests() { + local ca_b64 server_crt_b64 server_key_b64 client_crt_b64 client_key_b64 + ca_b64="$(base64 -w0 < "$WORKDIR/ca.crt")" + server_crt_b64="$(base64 -w0 < "$WORKDIR/server.crt")" + server_key_b64="$(base64 -w0 < "$WORKDIR/server.key")" + client_crt_b64="$(base64 -w0 < "$WORKDIR/client.crt")" + client_key_b64="$(base64 -w0 < "$WORKDIR/client.key")" + + cat > "$OUTPUT_DIR/skupper-site-server.yaml" < "$OUTPUT_DIR/client-secret.yaml" </dev/null 2>&1 || true + kubectl --context="$EAST_PROFILE" --namespace=default delete secret skupper-link --ignore-not-found=true >/dev/null 2>&1 || true +} + +build_link_bundle() { + log "Building the Link YAML using kubectl, jq, and yq" + local endpoints + endpoints="$(west_kubectl get site "$WEST_NAMESPACE" -o json | jq -c '.status.endpoints')" + printf '%s\n' "$endpoints" | jq '.' > "$OUTPUT_DIR/west-endpoints.json" + + cat > "$OUTPUT_DIR/west-link-template.json" < "$OUTPUT_DIR/west-link-rendered.json" + + yq eval -P -o=yaml "$OUTPUT_DIR/west-link-rendered.json" > "$OUTPUT_DIR/west-link.yaml" + + printf '%s\n' '---' >> "$OUTPUT_DIR/west-link.yaml" + cat "$OUTPUT_DIR/client-secret.yaml" >> "$OUTPUT_DIR/west-link.yaml" +} + +apply_link_bundle() { + log "Applying the Link bundle on the connecting site" + cleanup_stale_default_link_resources + east_kubectl apply -f "$OUTPUT_DIR/west-link.yaml" +} + +deploy_backend_workload() { + log "Deploying a backend workload on the connecting site" + cat > "$OUTPUT_DIR/backend-deployment.yaml" < "$OUTPUT_DIR/backend-connector.yaml" < "$OUTPUT_DIR/backend-listener.yaml" </dev/null 2>&1; then + local status + status="$(east_kubectl get link skupper-link --no-headers 2>/dev/null | awk '{print $2}')" + if [[ "$status" == "Ready" ]]; then + east_kubectl get link + return + fi + fi + sleep 5 + done + + printf '\nDiagnostics for link readiness in context=%s namespace=%s\n' "$EAST_PROFILE" "$EAST_NAMESPACE" >&2 + east_kubectl get link -o yaml >&2 || true + kubectl --context="$EAST_PROFILE" get link -A >&2 || true + east_kubectl get secret skupper-link -o yaml >&2 || true + east_kubectl get events --sort-by=.lastTimestamp >&2 || true + fail "Timed out waiting for link skupper-link to become Ready" +} + +wait_for_service_exposure_ready() { + local deadline + deadline=$((SECONDS + 300)) + + log "Waiting for Connector and Listener resources to become Ready" + while (( SECONDS < deadline )); do + local connector_status listener_status + connector_status="$(east_kubectl get connector backend --no-headers 2>/dev/null | awk '{print $2}')" + listener_status="$(west_kubectl get listener backend --no-headers 2>/dev/null | awk '{print $5}')" + + if [[ "$connector_status" == "Ready" && "$listener_status" == "Ready" ]]; then + east_kubectl get connector + west_kubectl get listener + return + fi + sleep 5 + done + + east_kubectl get connector || true + west_kubectl get listener || true + fail "Timed out waiting for Connector and Listener resources to become Ready" +} + +verify_remote_traffic() { + local deadline + deadline=$((SECONDS + 300)) + + log "Verifying cross-site traffic through the Listener service" + cleanup_probe_pod + west_kubectl run traffic-check --image="$PROBE_POD_IMAGE" --restart=Never --command -- sh -c 'sleep 3600' + west_kubectl wait --for=condition=Ready pod/traffic-check --timeout=180s + + while (( SECONDS < deadline )); do + if west_kubectl exec traffic-check -- wget -qO- http://east-backend:8080/ > "$WORKDIR/traffic-check-response.txt" 2>/dev/null; then + log "Traffic check succeeded" + cat "$WORKDIR/traffic-check-response.txt" + cleanup_probe_pod + return + fi + sleep 5 + done + + west_kubectl get svc east-backend || true + cleanup_probe_pod + fail "Timed out verifying traffic through service east-backend" +} + +print_summary() { + cat < Date: Wed, 10 Jun 2026 15:45:37 +0100 Subject: [PATCH 4/5] update --- input/kube-cli/site-linking.md | 9 + input/kube-yaml/site-linking.md | 110 +++-- tests/README.md | 98 ----- tests/minikube-custom-linking.sh | 687 ------------------------------- 4 files changed, 83 insertions(+), 821 deletions(-) delete mode 100644 tests/README.md delete mode 100755 tests/minikube-custom-linking.sh diff --git a/input/kube-cli/site-linking.md b/input/kube-cli/site-linking.md index cc9602a9..3dd14abe 100644 --- a/input/kube-cli/site-linking.md +++ b/input/kube-cli/site-linking.md @@ -125,3 +125,12 @@ To link sites, you create a `link` resource YAML file on one site and apply that There are many options to consider when linking sites using the CLI, see [CLI Reference][cli-ref], including *frequently used* options. +## Using custom certificates + +For information about linking sites using custom certificates instead of the default Skupper-generated certificates, see [Linking sites using custom certificates][custom-certs-yaml]. + +**📌 NOTE** +The YAML approach is recommended for custom certificate workflows as it provides more control over certificate management and integration with existing PKI infrastructure. + +[custom-certs-yaml]: ../kube-yaml/site-linking.html#kube-link-custom-certs-yaml + diff --git a/input/kube-yaml/site-linking.md b/input/kube-yaml/site-linking.md index babf778e..40ed8151 100644 --- a/input/kube-yaml/site-linking.md +++ b/input/kube-yaml/site-linking.md @@ -118,15 +118,22 @@ A connecting site redeems this token for a `Link` resource to establish a link t ## Linking sites using `Link` resources and custom certificates -By default, the site CA `skupper-site-ca` issues the `skupper-site-server` certificate that authenticates incoming links. -If you need to use your own server certificate, you can provide custom TLS credentials and then create a `Link` resource for remote sites. +By default, the Skupper V2 controller generates internal Certificate Authorities (CAs) and self-signed certificates. +For example, it creates certificates to authenticate incoming Skupper links from external Skupper sites. + +The CA and server certificate used for this authentication are named `skupper-site-ca` (default issuer for a Skupper Site) and `skupper-site-server`, respectively. + +The self-signed server certificate (`skupper-site-server`) is issued for the public hostname or IP address associated with the `skupper-router` service. This depends on the ingress method used, for example an OpenShift Route or a Kubernetes LoadBalancer Service. + +Although this behavior is automatic, you can override it by providing your own custom server certificate or even your own CA. **Prerequisites** * Two sites * The listening site must have `link-access` enabled * A server certificate and key for the listening site -* `jq` and `yq` installed if you want to use the command examples to generate the `Link` resource +* `jq` and `yq` installed if using the kubectl method to generate the `Link` resource +* `skupper` CLI installed if using the CLI method to generate the `Link` resource To link sites using custom certificates, you provide a custom server certificate on the listening site and create a `Link` resource on the connecting site that references matching client credentials. @@ -147,27 +154,36 @@ To link sites using custom certificates, you provide a custom server certificate ```shell kubectl apply -f skupper-site-server.yaml ``` - The certificate specified in `tls.crt` must be valid for the hostname or IP address that connecting sites use to connect to this site. + + Make sure the certificate specified in `tls.crt` is valid for the hostname or IP address that will be referenced in your `Link` resource. In this example, consider the server certificate as being valid for the hostname: `skupper.public.host`. -2. Determine the hostname or IP address for the listening site: +2. Determine the hostname or IP address for the listening site. + + In case your Skupper Site is already created, you can find the appropriate Hostname or IP by looking at the Endpoint element of a Site status: ```shell kubectl get site -o json | jq -r '.status.endpoints[0].host' ``` - For example, the output might be: + And you will see something like: ``` skupper.public.host ``` - If the site is already running, you can confirm that Skupper detected the custom server certificate: + + **_Note:_** Make sure you specify the appropriate namespace when running the commands from this example. + + Again, if your Site has already been created, Skupper will recognize your custom secret. You can confirm this with the following command: ```shell kubectl get certificate skupper-site-server -o json | jq -r '.status.conditions[].message' ``` - The output includes a message similar to: + You should see a message like: ``` Secret exists but is not controlled by skupper ``` + This confirms that Skupper has detected your certificate but is not managing it. 3. On the listening site, create client credentials for the connecting site. + Once your Skupper site is configured to use your custom server certificate, you can create a `Link` resource and an associated client `Secret`. If the `skupper-site-ca` Issuer has been provided, you can create a Certificate resource and Skupper will generate the client Secret to be used, but it is only possible if the Issuer has been provided, otherwise, you have to provide the client Secret yourself. + If the listening site provides the `skupper-site-ca` issuer, create a `Certificate` resource so that Skupper generates a client secret named `skupper-link`: ```yaml apiVersion: skupper.io/v2alpha1 @@ -183,6 +199,12 @@ To link sites using custom certificates, you provide a custom server certificate ```shell kubectl apply -f skupper-link-certificate.yaml ``` + You should see an output like: + ``` + certificate.skupper.io/skupper-link created + ``` + Then a Secret named `skupper-link` should be created and it can be used to compose your link file. + Save the generated secret to a local file for use in the next step: ```shell kubectl get secret skupper-link -o yaml | yq eval -o=yaml 'del(.metadata.namespace, .metadata.creationTimestamp, .metadata.resourceVersion, .metadata.uid, .metadata.managedFields)' - > client-secret.yaml @@ -203,34 +225,50 @@ To link sites using custom certificates, you provide a custom server certificate 4. On the listening site, create a `Link` resource YAML file. - To generate the `Link` resource using `kubectl`, `jq`, and `yq`, retrieve the site endpoints and write the YAML file: + Now, regardless of whether Skupper generated your client certificate Secret or if you generated it yourself, to define a Skupper Link, that can be shared with remote sites allowing them to initiate a secure outgoing link to your site, you will need to write a YAML file that contains both documents: a `Link`, and a Client `Secret`. + + You can generate a Link using `kubectl` or manually (as long as retrieve the list of endpoints). Here are these two methods: + + **Option A: Using kubectl with jq and yq** + + The following command uses `kubectl`, `yq`, `jq` and `tee` to extract and compose the information needed to define a Link, storing the Link document into `skupper-link.yaml`: ```shell - endpoints="$(kubectl get site -o json | jq -c '.status.endpoints')" - cat < west-link.yaml - { - "apiVersion": "skupper.io/v2alpha1", - "kind": "Link", - "metadata": { - "name": "skupper-link" - }, - "spec": { - "cost": 1, - "tlsCredentials": "skupper-link" - } - } + endpoints=$(kubectl get site -o json | jq -r '.items[].status.endpoints') + cat << EOF | yq -y --argjson endpoints "${endpoints}" '.spec.endpoints = $endpoints' | tee skupper-link.yaml + apiVersion: skupper.io/v2alpha1 + kind: Link + metadata: + name: skupper-link + spec: + cost: 1 + tlsCredentials: skupper-link EOF ``` - If you created a local client secret file, combine it with the `Link` resource: + Then you need to combine it in a YAML file that contains both the Link above and the client Secret. Suppose your client Secret is stored in a file named `client-secret.yaml`, you could run: ```shell - printf '%s\n' '---' >> west-link.yaml - cat client-secret.yaml >> west-link.yaml + echo "---" >> skupper-link.yaml + cat client-secret.yaml >> skupper-link.yaml ``` - To create the YAML manually, first inspect the endpoints: + **Option B: Manual generation** + + You can retrieve the list of endpoints from the Site definition, using: ```shell - kubectl get site -o yaml | yq eval -o=yaml '.status.endpoints' - + kubectl get site -o yaml | yq -y .items[].status.endpoints + ``` + The command above uses both `kubectl` and `yq`. And the output should be something like: + ```yaml + - group: skupper-router + host: skupper.public.host + name: inter-router + port: '55671' + - group: skupper-router + host: skupper.public.host + name: edge + port: '45671' ``` - Then create a file that contains both the `Link` resource and the client `Secret`, for example: + + Then you can create your Link file with: ```yaml --- apiVersion: skupper.io/v2alpha1 @@ -242,22 +280,22 @@ To link sites using custom certificates, you provide a custom server certificate tlsCredentials: skupper-link endpoints: - group: skupper-router - host: skupper.public.host + host: skupper.public.host # The hostname or IP specified in your custom certificate name: inter-router - port: "55671" + port: '55671' - group: skupper-router - host: skupper.public.host + host: skupper.public.host # The same hostname or IP name: edge - port: "45671" + port: '45671' --- apiVersion: v1 kind: Secret metadata: name: skupper-link data: - ca.crt: LS0tLS1C...redacted - tls.crt: LS0tLS1C...redacted - tls.key: LS0tLS1C...redacted + ca.crt: LS0tLS1C...redacted # The trusted CA certificate + tls.crt: LS0tLS1C...redacted # The client certificate + tls.key: LS0tLS1C...redacted # The corresponding private key ``` 5. Securely transfer the `Link` resource YAML file to the context of the connecting site. @@ -269,7 +307,7 @@ To link sites using custom certificates, you provide a custom server certificate 6. On the connecting site, apply the YAML file and check status: ```shell - kubectl apply -f west-link.yaml + kubectl apply -f skupper-link.yaml kubectl get link NAME STATUS REMOTE SITE MESSAGE skupper-link Ready my-site OK diff --git a/tests/README.md b/tests/README.md deleted file mode 100644 index 87c24fe6..00000000 --- a/tests/README.md +++ /dev/null @@ -1,98 +0,0 @@ -# Tests - -This directory contains ad hoc test helpers for validating documentation procedures locally. - -## Minikube custom-link test - -`minikube-custom-linking.sh` exercises the Kubernetes YAML workflow for linking two Skupper sites with: - -* two Minikube profiles -* a custom `skupper-site-server` certificate on the listening site -* a manually generated client `Secret` -* a `Link` YAML file assembled with `kubectl`, `jq`, and `yq` -* a V2 `Connector` and `Listener` pair to validate service exposure - -The script is intended for Fedora and assumes: - -* `minikube` -* `kubectl` -* `jq` -* `mikefarah/yq` v4 -* `openssl` -* access to download the Skupper install manifest unless Skupper is already installed - -### What it does - -1. Starts Minikube profiles `west` and `east` unless `START_PROFILES=0`. -2. Installs Skupper automatically with `kubectl apply -f https://skupper.io/install.yaml` if the CRDs are missing. -3. Creates `Site` resources in namespaces `west` and `east`. -4. Waits for both sites to reach the `Running` condition. -5. Starts `minikube tunnel -p west` automatically and writes its output to a log file. -6. Waits for the `west` site to become fully `Ready` and expose a listening endpoint. -7. Generates a temporary CA, server certificate, and client certificate. -8. Applies a custom `skupper-site-server` secret on the listening site. -9. Builds `west-link.yaml` using the same `jq` and `yq` pattern described in the docs. -10. Applies the generated link bundle in the `east` namespace and waits for the link to become `Ready`. -11. Deploys `quay.io/skupper/hello-world-backend` on the `east` site. -12. Applies a V2 `Connector` on `east` and a V2 `Listener` on `west`. -13. Starts a probe pod on `west` and fetches `http://east-backend:8080/` through the application network. -14. Pauses at a few checkpoints so you can inspect the clusters before continuing. - -### Usage - -Start the test: - -```bash -./tests/minikube-custom-linking.sh -``` - -If the Minikube profiles already exist: - -```bash -START_PROFILES=0 ./tests/minikube-custom-linking.sh -``` - -Skip automatic tunnel startup: - -```bash -START_TUNNEL=0 ./tests/minikube-custom-linking.sh -``` - -Disable the interactive checkpoints: - -```bash -PAUSE_ON_CHECKPOINTS=0 ./tests/minikube-custom-linking.sh -``` - -Disable the pause before exit on failures: - -```bash -PAUSE_ON_ERROR=0 ./tests/minikube-custom-linking.sh -``` - -Skip automatic Skupper installation: - -```bash -INSTALL_SKUPPER=0 ./tests/minikube-custom-linking.sh -``` - -Clean up the namespaces, profiles, and generated files: - -```bash -./tests/minikube-custom-linking.sh cleanup -``` - -### Notes - -* By default the script starts `minikube tunnel -p west` itself without `sudo`. Set `TUNNEL_WITH_SUDO=1` only if your local Minikube setup requires it. -* By default the script lets Minikube auto-detect the driver. Set `MINIKUBE_DRIVER` only if you need to force a specific driver on your machine. -* By default the script installs Skupper from `https://skupper.io/install.yaml` if the CRDs are missing. Override `SKUPPER_INSTALL_URL` if you want to use a different manifest. -* The script writes generated YAML manifests to `output/tests/minikube-custom-linking` unless `OUTPUT_DIR` is set. -* The script also saves the `Link` generation intermediates there, including the raw endpoint JSON, the JSON template, and the rendered JSON before conversion to YAML. -* The script writes transient files such as certificates, logs, and traffic output to `/tmp/skupper-docs-minikube-custom-linking` unless `WORKDIR` is set. -* The tunnel output is written to `${WORKDIR}/minikube-tunnel.log`. -* By default the script pauses for Enter at a few Skupper-related checkpoints, including after both sites are running, after the link is ready, and after service traffic is working. Set `PAUSE_ON_CHECKPOINTS=0` to disable that behavior. -* By default the script also pauses before exiting on an error so you can inspect the cluster state while the resources and tunnel are still up. Set `PAUSE_ON_ERROR=0` to disable that behavior. -* The script validates the manual client-secret path from the YAML documentation. It does not exercise the `Certificate` resource variant. -* By default the traffic check uses `busybox:1.36` and `wget`. Override `PROBE_POD_IMAGE` if you need a different probe image in your environment. -* `cleanup` removes the Minikube profiles and the temporary work directory, but it does not remove the saved YAML files under `output/`. diff --git a/tests/minikube-custom-linking.sh b/tests/minikube-custom-linking.sh deleted file mode 100755 index dc4d4e79..00000000 --- a/tests/minikube-custom-linking.sh +++ /dev/null @@ -1,687 +0,0 @@ -#!/usr/bin/env bash - -set -euo pipefail - -WEST_PROFILE="${WEST_PROFILE:-west}" -EAST_PROFILE="${EAST_PROFILE:-east}" -WEST_NAMESPACE="${WEST_NAMESPACE:-west}" -EAST_NAMESPACE="${EAST_NAMESPACE:-east}" -WORKDIR="${WORKDIR:-/tmp/skupper-docs-minikube-custom-linking}" -OUTPUT_DIR="${OUTPUT_DIR:-output/tests/minikube-custom-linking}" -START_PROFILES="${START_PROFILES:-1}" -START_TUNNEL="${START_TUNNEL:-1}" -TUNNEL_WITH_SUDO="${TUNNEL_WITH_SUDO:-0}" -PAUSE_ON_CHECKPOINTS="${PAUSE_ON_CHECKPOINTS:-1}" -PAUSE_ON_ERROR="${PAUSE_ON_ERROR:-1}" -INSTALL_SKUPPER="${INSTALL_SKUPPER:-1}" -SKUPPER_INSTALL_URL="${SKUPPER_INSTALL_URL:-https://skupper.io/install.yaml}" -MINIKUBE_DRIVER="${MINIKUBE_DRIVER:-}" -MINIKUBE_CPUS="${MINIKUBE_CPUS:-4}" -MINIKUBE_MEMORY="${MINIKUBE_MEMORY:-8192}" -TUNNEL_PID="" -TUNNEL_LOG="" -PROBE_POD_IMAGE="${PROBE_POD_IMAGE:-busybox:1.36}" - -log() { - printf '==> %s\n' "$*" -} - -fail() { - printf 'ERROR: %s\n' "$*" >&2 - - if [[ "$PAUSE_ON_ERROR" == "1" && -t 0 ]]; then - printf 'Press Enter to exit so you can inspect the cluster state... ' >&2 - read -r _ - fi - - exit 1 -} - -need_cmd() { - command -v "$1" >/dev/null 2>&1 || fail "Missing required command: $1" -} - -pause_checkpoint() { - local message="$1" - - if [[ "$PAUSE_ON_CHECKPOINTS" != "1" || ! -t 0 ]]; then - return - fi - - printf '\nCheckpoint: %s\n' "$message" - printf 'Press Enter to continue... ' - read -r _ -} - -west_kubectl() { - kubectl --context="$WEST_PROFILE" --namespace="$WEST_NAMESPACE" "$@" -} - -east_kubectl() { - kubectl --context="$EAST_PROFILE" --namespace="$EAST_NAMESPACE" "$@" -} - -skupper_api_available() { - local context="$1" - kubectl --context="$context" api-resources --api-group=skupper.io 2>/dev/null | grep -q '^sites' -} - -start_profiles() { - if [[ "$START_PROFILES" != "1" ]]; then - log "Skipping minikube start because START_PROFILES=$START_PROFILES" - return - fi - - local west_start_args=( - -p "$WEST_PROFILE" - --cpus="$MINIKUBE_CPUS" - --memory="$MINIKUBE_MEMORY" - ) - local east_start_args=( - -p "$EAST_PROFILE" - --cpus="$MINIKUBE_CPUS" - --memory="$MINIKUBE_MEMORY" - ) - - if [[ -n "$MINIKUBE_DRIVER" ]]; then - west_start_args+=(--driver="$MINIKUBE_DRIVER") - east_start_args+=(--driver="$MINIKUBE_DRIVER") - fi - - log "Starting minikube profile $WEST_PROFILE" - minikube start "${west_start_args[@]}" - - log "Starting minikube profile $EAST_PROFILE" - minikube start "${east_start_args[@]}" -} - -check_skupper_api() { - log "Checking that the Skupper API is available on both clusters" - - if ! skupper_api_available "$WEST_PROFILE"; then - if [[ "$INSTALL_SKUPPER" == "1" ]]; then - install_skupper "$WEST_PROFILE" - else - fail "Skupper CRDs not found in context $WEST_PROFILE. Install the Skupper V2 controller in that cluster first." - fi - fi - - if ! skupper_api_available "$EAST_PROFILE"; then - if [[ "$INSTALL_SKUPPER" == "1" ]]; then - install_skupper "$EAST_PROFILE" - else - fail "Skupper CRDs not found in context $EAST_PROFILE. Install the Skupper V2 controller in that cluster first." - fi - fi -} - -install_skupper() { - local context="$1" - local deadline - deadline=$((SECONDS + 300)) - - log "Installing Skupper in context $context from $SKUPPER_INSTALL_URL" - kubectl --context="$context" apply -f "$SKUPPER_INSTALL_URL" - - log "Waiting for Skupper CRDs in context $context" - while (( SECONDS < deadline )); do - if skupper_api_available "$context"; then - return - fi - sleep 5 - done - - fail "Timed out waiting for Skupper CRDs after install in context $context" -} - -prepare_workdir() { - rm -rf "$WORKDIR" - mkdir -p "$WORKDIR" - mkdir -p "$OUTPUT_DIR" -} - -deploy_sites() { - log "Creating namespaces" - kubectl --context="$WEST_PROFILE" create namespace "$WEST_NAMESPACE" --dry-run=client -o yaml | kubectl --context="$WEST_PROFILE" apply -f - - kubectl --context="$EAST_PROFILE" create namespace "$EAST_NAMESPACE" --dry-run=client -o yaml | kubectl --context="$EAST_PROFILE" apply -f - - - cat > "$OUTPUT_DIR/west-site.yaml" < "$OUTPUT_DIR/east-site.yaml" <&2 - kubectl --context="$context" --namespace="$namespace" get site -o yaml >&2 || true - kubectl --context="$context" --namespace="$namespace" get pods >&2 || true - kubectl --context="$context" --namespace="$namespace" describe site "$namespace" >&2 || true - kubectl --context="$context" --namespace="$namespace" get events --sort-by=.lastTimestamp >&2 || true -} - -wait_for_site() { - local context="$1" - local namespace="$2" - local deadline - deadline=$((SECONDS + 300)) - - log "Waiting for site $namespace in context $context" - while (( SECONDS < deadline )); do - if kubectl --context="$context" --namespace="$namespace" get site "$namespace" >/dev/null 2>&1; then - local status - status="$(kubectl --context="$context" --namespace="$namespace" get site "$namespace" --no-headers 2>/dev/null | awk '{print $2}')" - if [[ "$status" == "Ready" ]]; then - return - fi - fi - sleep 5 - done - - dump_site_diagnostics "$context" "$namespace" - fail "Timed out waiting for site $namespace in context $context" -} - -wait_for_site_running() { - local context="$1" - local namespace="$2" - local deadline - deadline=$((SECONDS + 300)) - - log "Waiting for site $namespace in context $context to be Running" - while (( SECONDS < deadline )); do - if kubectl --context="$context" --namespace="$namespace" get site "$namespace" >/dev/null 2>&1; then - local running - running="$(kubectl --context="$context" --namespace="$namespace" get site "$namespace" -o json | jq -r '.status.conditions[]? | select(.type=="Running") | .status')" - if [[ "$running" == "True" ]]; then - return - fi - fi - sleep 5 - done - - dump_site_diagnostics "$context" "$namespace" - fail "Timed out waiting for site $namespace in context $context to be Running" -} - -prompt_for_tunnel() { - if [[ "$START_TUNNEL" != "1" ]]; then - return - fi - - if [[ "$TUNNEL_WITH_SUDO" == "1" && "$EUID" -ne 0 ]]; then - log "Requesting sudo access for minikube tunnel" - sudo -v - fi - - TUNNEL_LOG="$WORKDIR/minikube-tunnel.log" - - log "Starting minikube tunnel for profile $WEST_PROFILE" - if [[ "$TUNNEL_WITH_SUDO" == "1" && "$EUID" -ne 0 ]]; then - sudo --preserve-env=HOME,PATH minikube tunnel -p "$WEST_PROFILE" >"$TUNNEL_LOG" 2>&1 & - else - minikube tunnel -p "$WEST_PROFILE" >"$TUNNEL_LOG" 2>&1 & - fi - TUNNEL_PID=$! - - sleep 3 - if ! kill -0 "$TUNNEL_PID" >/dev/null 2>&1; then - cat "$TUNNEL_LOG" >&2 || true - fail "Failed to start minikube tunnel for profile $WEST_PROFILE" - fi - - cat </dev/null 2>&1; then - log "Stopping minikube tunnel" - kill "$TUNNEL_PID" >/dev/null 2>&1 || true - wait "$TUNNEL_PID" 2>/dev/null || true - fi -} - -cleanup_probe_pod() { - west_kubectl delete pod traffic-check --ignore-not-found=true >/dev/null 2>&1 || true -} - -wait_for_endpoint_host() { - local deadline - deadline=$((SECONDS + 300)) - - log "Waiting for a listening endpoint on ${WEST_PROFILE}/${WEST_NAMESPACE}" - while (( SECONDS < deadline )); do - WEST_HOST="$(west_kubectl get site "$WEST_NAMESPACE" -o json | jq -r '.status.endpoints[0].host // empty')" - if [[ -n "$WEST_HOST" && "$WEST_HOST" != "null" ]]; then - export WEST_HOST - log "Using listening endpoint host: $WEST_HOST" - return - fi - sleep 5 - done - - west_kubectl get site "$WEST_NAMESPACE" -o yaml || true - fail "Timed out waiting for the listening endpoint. Ensure minikube tunnel is running for profile $WEST_PROFILE." -} - -write_openssl_config() { - local san_type="$1" - local san_value="$2" - - cat > "$WORKDIR/server-ext.cnf" < "$WORKDIR/client-ext.cnf" </dev/null 2>&1 - openssl req -x509 -new -nodes -key "$WORKDIR/ca.key" -sha256 -days 365 \ - -subj "/CN=skupper-docs-test-ca" \ - -out "$WORKDIR/ca.crt" >/dev/null 2>&1 - - openssl genrsa -out "$WORKDIR/server.key" 2048 >/dev/null 2>&1 - openssl req -new -key "$WORKDIR/server.key" \ - -config "$WORKDIR/server-ext.cnf" \ - -out "$WORKDIR/server.csr" >/dev/null 2>&1 - openssl x509 -req -in "$WORKDIR/server.csr" \ - -CA "$WORKDIR/ca.crt" -CAkey "$WORKDIR/ca.key" -CAcreateserial \ - -out "$WORKDIR/server.crt" -days 365 -sha256 \ - -extfile "$WORKDIR/server-ext.cnf" -extensions v3_server >/dev/null 2>&1 - - openssl genrsa -out "$WORKDIR/client.key" 2048 >/dev/null 2>&1 - openssl req -new -key "$WORKDIR/client.key" \ - -config "$WORKDIR/client-ext.cnf" \ - -out "$WORKDIR/client.csr" >/dev/null 2>&1 - openssl x509 -req -in "$WORKDIR/client.csr" \ - -CA "$WORKDIR/ca.crt" -CAkey "$WORKDIR/ca.key" -CAcreateserial \ - -out "$WORKDIR/client.crt" -days 365 -sha256 \ - -extfile "$WORKDIR/client-ext.cnf" -extensions v3_client >/dev/null 2>&1 -} - -write_secret_manifests() { - local ca_b64 server_crt_b64 server_key_b64 client_crt_b64 client_key_b64 - ca_b64="$(base64 -w0 < "$WORKDIR/ca.crt")" - server_crt_b64="$(base64 -w0 < "$WORKDIR/server.crt")" - server_key_b64="$(base64 -w0 < "$WORKDIR/server.key")" - client_crt_b64="$(base64 -w0 < "$WORKDIR/client.crt")" - client_key_b64="$(base64 -w0 < "$WORKDIR/client.key")" - - cat > "$OUTPUT_DIR/skupper-site-server.yaml" < "$OUTPUT_DIR/client-secret.yaml" </dev/null 2>&1 || true - kubectl --context="$EAST_PROFILE" --namespace=default delete secret skupper-link --ignore-not-found=true >/dev/null 2>&1 || true -} - -build_link_bundle() { - log "Building the Link YAML using kubectl, jq, and yq" - local endpoints - endpoints="$(west_kubectl get site "$WEST_NAMESPACE" -o json | jq -c '.status.endpoints')" - printf '%s\n' "$endpoints" | jq '.' > "$OUTPUT_DIR/west-endpoints.json" - - cat > "$OUTPUT_DIR/west-link-template.json" < "$OUTPUT_DIR/west-link-rendered.json" - - yq eval -P -o=yaml "$OUTPUT_DIR/west-link-rendered.json" > "$OUTPUT_DIR/west-link.yaml" - - printf '%s\n' '---' >> "$OUTPUT_DIR/west-link.yaml" - cat "$OUTPUT_DIR/client-secret.yaml" >> "$OUTPUT_DIR/west-link.yaml" -} - -apply_link_bundle() { - log "Applying the Link bundle on the connecting site" - cleanup_stale_default_link_resources - east_kubectl apply -f "$OUTPUT_DIR/west-link.yaml" -} - -deploy_backend_workload() { - log "Deploying a backend workload on the connecting site" - cat > "$OUTPUT_DIR/backend-deployment.yaml" < "$OUTPUT_DIR/backend-connector.yaml" < "$OUTPUT_DIR/backend-listener.yaml" </dev/null 2>&1; then - local status - status="$(east_kubectl get link skupper-link --no-headers 2>/dev/null | awk '{print $2}')" - if [[ "$status" == "Ready" ]]; then - east_kubectl get link - return - fi - fi - sleep 5 - done - - printf '\nDiagnostics for link readiness in context=%s namespace=%s\n' "$EAST_PROFILE" "$EAST_NAMESPACE" >&2 - east_kubectl get link -o yaml >&2 || true - kubectl --context="$EAST_PROFILE" get link -A >&2 || true - east_kubectl get secret skupper-link -o yaml >&2 || true - east_kubectl get events --sort-by=.lastTimestamp >&2 || true - fail "Timed out waiting for link skupper-link to become Ready" -} - -wait_for_service_exposure_ready() { - local deadline - deadline=$((SECONDS + 300)) - - log "Waiting for Connector and Listener resources to become Ready" - while (( SECONDS < deadline )); do - local connector_status listener_status - connector_status="$(east_kubectl get connector backend --no-headers 2>/dev/null | awk '{print $2}')" - listener_status="$(west_kubectl get listener backend --no-headers 2>/dev/null | awk '{print $5}')" - - if [[ "$connector_status" == "Ready" && "$listener_status" == "Ready" ]]; then - east_kubectl get connector - west_kubectl get listener - return - fi - sleep 5 - done - - east_kubectl get connector || true - west_kubectl get listener || true - fail "Timed out waiting for Connector and Listener resources to become Ready" -} - -verify_remote_traffic() { - local deadline - deadline=$((SECONDS + 300)) - - log "Verifying cross-site traffic through the Listener service" - cleanup_probe_pod - west_kubectl run traffic-check --image="$PROBE_POD_IMAGE" --restart=Never --command -- sh -c 'sleep 3600' - west_kubectl wait --for=condition=Ready pod/traffic-check --timeout=180s - - while (( SECONDS < deadline )); do - if west_kubectl exec traffic-check -- wget -qO- http://east-backend:8080/ > "$WORKDIR/traffic-check-response.txt" 2>/dev/null; then - log "Traffic check succeeded" - cat "$WORKDIR/traffic-check-response.txt" - cleanup_probe_pod - return - fi - sleep 5 - done - - west_kubectl get svc east-backend || true - cleanup_probe_pod - fail "Timed out verifying traffic through service east-backend" -} - -print_summary() { - cat < Date: Wed, 10 Jun 2026 16:03:21 +0100 Subject: [PATCH 5/5] update --- input/kube-yaml/site-linking.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/input/kube-yaml/site-linking.md b/input/kube-yaml/site-linking.md index 40ed8151..cf01e0f7 100644 --- a/input/kube-yaml/site-linking.md +++ b/input/kube-yaml/site-linking.md @@ -121,9 +121,9 @@ A connecting site redeems this token for a `Link` resource to establish a link t By default, the Skupper V2 controller generates internal Certificate Authorities (CAs) and self-signed certificates. For example, it creates certificates to authenticate incoming Skupper links from external Skupper sites. -The CA and server certificate used for this authentication are named `skupper-site-ca` (default issuer for a Skupper Site) and `skupper-site-server`, respectively. +The CA and server certificate used for this authentication are named `skupper-site-ca` (default signing `Certificate` resource for a Skupper Site) and `skupper-site-server`, respectively. -The self-signed server certificate (`skupper-site-server`) is issued for the public hostname or IP address associated with the `skupper-router` service. This depends on the ingress method used, for example an OpenShift Route or a Kubernetes LoadBalancer Service. +The server certificate (`skupper-site-server`) issued by the self-signed CA `skupper-site-ca` is issued for the public hostname or IP address associated with the `skupper-router` service. This depends on the ingress method used, for example an OpenShift Route or a Kubernetes LoadBalancer Service. Although this behavior is automatic, you can override it by providing your own custom server certificate or even your own CA. @@ -182,7 +182,7 @@ To link sites using custom certificates, you provide a custom server certificate 3. On the listening site, create client credentials for the connecting site. - Once your Skupper site is configured to use your custom server certificate, you can create a `Link` resource and an associated client `Secret`. If the `skupper-site-ca` Issuer has been provided, you can create a Certificate resource and Skupper will generate the client Secret to be used, but it is only possible if the Issuer has been provided, otherwise, you have to provide the client Secret yourself. + Once your Skupper site is configured to use your custom server certificate, you can create a `Link` resource and an associated client `Secret`. If the `skupper-site-ca` signing `Certificate` resource has been provided, you can create a Certificate resource and Skupper will generate the client Secret to be used, but it is only possible if the signing `Certificate` resource has been provided, otherwise, you have to provide the client Secret yourself. If the listening site provides the `skupper-site-ca` issuer, create a `Certificate` resource so that Skupper generates a client secret named `skupper-link`: ```yaml @@ -233,7 +233,7 @@ To link sites using custom certificates, you provide a custom server certificate The following command uses `kubectl`, `yq`, `jq` and `tee` to extract and compose the information needed to define a Link, storing the Link document into `skupper-link.yaml`: ```shell - endpoints=$(kubectl get site -o json | jq -r '.items[].status.endpoints') + endpoints=$(kubectl get site -o json | jq -r '.status.endpoints') cat << EOF | yq -y --argjson endpoints "${endpoints}" '.spec.endpoints = $endpoints' | tee skupper-link.yaml apiVersion: skupper.io/v2alpha1 kind: Link @@ -254,7 +254,7 @@ To link sites using custom certificates, you provide a custom server certificate You can retrieve the list of endpoints from the Site definition, using: ```shell - kubectl get site -o yaml | yq -y .items[].status.endpoints + kubectl get site -o yaml | yq -y .status.endpoints ``` The command above uses both `kubectl` and `yq`. And the output should be something like: ```yaml