Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
65 changes: 65 additions & 0 deletions .github/workflows/ecr-mirror-check.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
name: ECR mirror check

"on":
schedule:
- cron: "17 6 * * *"
workflow_dispatch:
inputs:
request:
description: Dispatch mirror requests for out-of-sync releases.
required: false
type: boolean
default: false

permissions:
contents: read

concurrency:
group: ecr-mirror-check
cancel-in-progress: false

jobs:
check:
name: check ECR Public mirrors
runs-on: ubuntu-24.04
timeout-minutes: 120
steps:
- name: Checkout mirror scripts
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
fetch-depth: 1
persist-credentials: false

- name: Install regctl
shell: bash
run: |
set -euo pipefail
regctl_version="v0.11.5"
expected_sha256="c93aa7638749f5aaac1a8e01787321889c78f0101809bb2880343478d0ba0467"
curl -fsSL \
"https://github.com/regclient/regclient/releases/download/${regctl_version}/regctl-linux-amd64" \
-o "$RUNNER_TEMP/regctl"
actual_sha256="$(sha256sum "$RUNNER_TEMP/regctl" | cut -d' ' -f1)"
[[ "$actual_sha256" == "$expected_sha256" ]] || {
printf 'regctl checksum mismatch: expected %s, got %s\n' \
"$expected_sha256" "$actual_sha256" >&2
exit 1
}
chmod +x "$RUNNER_TEMP/regctl"
echo "$RUNNER_TEMP" >> "$GITHUB_PATH"

- name: Compare published releases against ECR Public
shell: bash
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
MIRROR_DISPATCH_TOKEN: ${{ secrets.CLI_MIRROR_DISPATCH_TOKEN }}
REQUEST: ${{ inputs.request }}
run: |
set -euo pipefail
args=()
if [[ "$REQUEST" == "true" ]]; then
args+=(--request)
else
unset MIRROR_DISPATCH_TOKEN
fi
scripts/ecr-mirror.sh sync "${args[@]}"
1 change: 1 addition & 0 deletions .github/workflows/repository-checks.yml
Original file line number Diff line number Diff line change
Expand Up @@ -57,5 +57,6 @@ jobs:
scripts/test-extract-upstream-archive.sh
scripts/test-upstream-artifact.sh
scripts/test-oci-mirror.sh
scripts/test-ecr-mirror.sh
scripts/test-upstream-runtime.sh
services/vector/test-smoke.sh
88 changes: 88 additions & 0 deletions .github/workflows/service-release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -738,12 +738,83 @@ jobs:
retention-days: 7
path: mirror-provenance.json

mirror-ecr:
name: mirror ${{ inputs.service }} image to ECR Public
needs:
- plan
- publish-image
if: needs.plan.outputs.publish == 'true'
runs-on: ubuntu-24.04
timeout-minutes: 45
outputs:
mirrored: ${{ steps.mirror.outputs.mirrored || 'false' }}
steps:
- name: Checkout mirror scripts
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
fetch-depth: 1
persist-credentials: false

- name: Check mirror dispatch configuration
id: config
shell: bash
env:
MIRROR_DISPATCH_TOKEN: ${{ secrets.CLI_MIRROR_DISPATCH_TOKEN }}
run: |
set -euo pipefail
if [[ -z "$MIRROR_DISPATCH_TOKEN" ]]; then
echo "::notice::CLI_MIRROR_DISPATCH_TOKEN is not configured; skipping the ECR Public mirror"
printf 'configured=false\n' >> "$GITHUB_OUTPUT"
else
printf 'configured=true\n' >> "$GITHUB_OUTPUT"
fi

- name: Download published image metadata
if: steps.config.outputs.configured == 'true'
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
name: published-image-${{ inputs.service }}-${{ inputs.version }}

- name: Install regctl
if: steps.config.outputs.configured == 'true'
shell: bash
run: |
set -euo pipefail
regctl_version="v0.11.5"
expected_sha256="c93aa7638749f5aaac1a8e01787321889c78f0101809bb2880343478d0ba0467"
curl -fsSL \
"https://github.com/regclient/regclient/releases/download/${regctl_version}/regctl-linux-amd64" \
-o "$RUNNER_TEMP/regctl"
actual_sha256="$(sha256sum "$RUNNER_TEMP/regctl" | cut -d' ' -f1)"
[[ "$actual_sha256" == "$expected_sha256" ]] || {
printf 'regctl checksum mismatch: expected %s, got %s\n' \
"$expected_sha256" "$actual_sha256" >&2
exit 1
}
chmod +x "$RUNNER_TEMP/regctl"
echo "$RUNNER_TEMP" >> "$GITHUB_PATH"

- name: Request and verify ECR Public mirror
id: mirror
if: steps.config.outputs.configured == 'true'
shell: bash
env:
MIRROR_DISPATCH_TOKEN: ${{ secrets.CLI_MIRROR_DISPATCH_TOKEN }}
SERVICE: ${{ inputs.service }}
VERSION: ${{ inputs.version }}
run: |
set -euo pipefail
digest="$(python3 -c 'import json; print(json.load(open("published-image.json"))["digest"])')"
scripts/ecr-mirror.sh request "$SERVICE" "$VERSION" "$digest"
printf 'mirrored=true\n' >> "$GITHUB_OUTPUT"

publish-release:
name: publish ${{ inputs.service }} release
needs:
- plan
- build
- publish-image
- mirror-ecr
if: needs.plan.outputs.publish == 'true'
runs-on: ubuntu-24.04
timeout-minutes: 20
Expand Down Expand Up @@ -799,6 +870,7 @@ jobs:
shell: bash
env:
IMAGE_RELEASE: ${{ needs.plan.outputs.image_release }}
MIRRORED: ${{ needs.mirror-ecr.outputs.mirrored }}
SERVICE: ${{ inputs.service }}
VERSION: ${{ inputs.version }}
run: |
Expand Down Expand Up @@ -856,6 +928,22 @@ jobs:
Verify a downloaded archive with the attached \`SHA256SUMS\` file.
EOF

if [[ "${MIRRORED:-}" == "true" ]]; then
ecr_image="$(
scripts/ecr-mirror.sh payload "$SERVICE" "$VERSION" "$digest" \
| python3 -c 'import json,sys; print(json.load(sys.stdin)["client_payload"]["destination"])'
)"
ecr_image="${ecr_image%%:*}"
cat >> release-notes.md <<EOF

AWS ECR Public mirror (same digests):

\`$ecr_image:$VERSION\`

\`$ecr_image@$digest\`
EOF
fi

if [[ "$IMAGE_RELEASE" == "mirror" ]]; then
python3 - "release-assets/$SERVICE-$VERSION.oci-provenance.json" "$image" >> release-notes.md <<'PY'
import json
Expand Down
91 changes: 91 additions & 0 deletions docs/design/ecr-mirror-dispatch.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
# ECR Public mirroring via supabase/cli dispatch

Slim images are published to `ghcr.io/supabase/cli/<service>:<version>` by
`.github/workflows/service-release.yml`. This document describes how those
images are also mirrored to AWS ECR Public, reusing the mirror machinery and
AWS credentials that already live in `supabase/cli`, so this repository needs
no AWS access of its own.

## Flow

1. The `publish-image` job publishes the multi-platform image to GHCR and
records its index digest in `published-image.json`.
2. The `mirror-ecr` job sends a `repository_dispatch` event to `supabase/cli`
(`scripts/ecr-mirror.sh request`), then polls the destination anonymously
until its index digest equals the GHCR index digest, or fails after a
timeout (15 minutes by default).
3. The `publish-release` job appends the ECR references to the release notes
when the mirror was verified.
4. `.github/workflows/ecr-mirror-check.yml` runs daily and fails when any
published non-draft/non-prerelease tag that maps to a configured service
(by `<service>-` prefix) is missing from ECR Public or resolves to a
different digest. The daily audit does not apply `tag_pattern`, so older
tags that no longer match the current pattern stay in the compare set.
Run it manually with `request: true` to re-request out-of-sync tags
(this is also the backfill path for releases that predate mirroring).

Release-time mirroring (`service-release.yml` `mirror-ecr`) is skipped,
with a workflow notice, until the `CLI_MIRROR_DISPATCH_TOKEN` secret
exists. Once the secret is set, a failed or unverified mirror fails the
release: a release is only done when both registries serve the same
digest. The daily audit always runs; it needs only `gh` and `regctl`.
Dispatch (`request: true`) still requires the token.

## Dispatch contract

The event sent to `POST /repos/supabase/cli/dispatches`:

```json
{
"event_type": "mirror-slim-image",
"client_payload": {
"service": "postgrest",
"version": "v16.2",
"source": "ghcr.io/supabase/cli/postgrest:v16.2",
"digest": "sha256:…",
"destination": "public.ecr.aws/supabase/cli/postgrest:v16.2"
}
}
```

The handling workflow in `supabase/cli` must:

- Trigger on `repository_dispatch` with `types: [mirror-slim-image]`.
- Copy `source` to `destination` with a digest-preserving tool
(`regctl image copy`, `crane cp`, or `akhilerm/tag-push-action`).
`docker buildx imagetools create` may rewrite the index and change its
digest; verification here would then fail the release.
- Reject a `source` outside `ghcr.io/supabase/cli/` and a `destination`
outside `public.ecr.aws/supabase/cli/`, and verify that `source` resolves
to `digest` before copying. The payload arrives with whatever authority
holds the dispatch token, so the handler validates it independently.
- Create the ECR Public repository when it does not exist, or the
`cli/<service>` repositories must be created up front. ECR does not create
repositories on push.

This repository treats the dispatch as fire-and-forget: success is defined
purely by the destination digest matching, which `scripts/ecr-mirror.sh`
verifies with anonymous pulls.

## Setup checklist

1. Land the `mirror-slim-image` handler in `supabase/cli` (see contract
above).
2. Create the `cli/<service>` ECR Public repositories for the services in
`.github/service-release-sources.json`, or grant the handler's role
`ecr-public:CreateRepository`.
3. Create a token that can send `repository_dispatch` to `supabase/cli`
(fine-grained PAT with contents read/write on `supabase/cli`, or a GitHub
App installation token) and store it in this repository as the
`CLI_MIRROR_DISPATCH_TOKEN` actions secret.
4. Backfill existing releases: run the `ECR mirror check` workflow with
`request: true`.

## Naming

The destination keeps the `cli/` namespace (`public.ecr.aws/supabase/cli/…`)
instead of joining the existing upstream mirrors at
`public.ecr.aws/supabase/<service>` because slim tags reuse upstream version
strings; `supabase/postgrest:v16.2` is already the upstream image. Keeping
the path identical to GHCR also lets consumers switch registries by swapping
only the registry host prefix.
Loading