Skip to content

fix: push can't cancel preview teardown (reliable cleanup)#278

Merged
nourshoreibah merged 1 commit into
mainfrom
fix/preview-teardown-concurrency
Jul 1, 2026
Merged

fix: push can't cancel preview teardown (reliable cleanup)#278
nourshoreibah merged 1 commit into
mainfrom
fix/preview-teardown-concurrency

Conversation

@nourshoreibah

Copy link
Copy Markdown
Collaborator

Bug

Remove the test-environment label, then immediately push a commit → the preview stack leaks (never torn down).

Why

preview-env.yml used one workflow-wide concurrency group with cancel-in-progress: true:

  1. Unlabel → teardown run starts.
  2. Push → new run in the same group → cancels the in-flight teardown.
  3. That push run's deploy job is skipped (label already gone) and its teardown job doesn't match synchronizeboth jobs skip.
  4. Teardown was cancelled and nothing re-runs it → resources leak.

Fix

Per-job concurrency instead of workflow-wide:

  • deploypreview-deploy-<pr>, cancel-in-progress: true (back-to-back commits still cancel the older deploy).
  • teardownpreview-teardown-<pr>, cancel-in-progress: false (a teardown is never cancelled; repeated teardowns queue and no-op on the already-deleted workspace).

Any overlapping terraform is serialized by the existing DynamoDB state lock, so cleanup is correct for every event ordering:

  • unlabel-then-push → push's deploy is skipped, teardown runs to completion ✅
  • push-then-unlabel → deploy runs (deploy group), teardown runs (teardown group); state lock serializes; end state = destroyed ✅
  • close while creating → create finishes, teardown then destroys ✅

Takes effect on merge to main (pull_request workflows run from the base branch's workflow file).

A single workflow-wide concurrency group with cancel-in-progress meant a
synchronize event (push) right after removing the test-environment label would
cancel the running teardown — and the push run itself skips both jobs (label
gone), so nothing re-ran teardown and the stack leaked.

Move concurrency to per-job groups:
- deploy: preview-deploy-<pr>, cancel-in-progress (back-to-back pushes still
  supersede each other).
- teardown: preview-teardown-<pr>, cancel-in-progress: false (never cancelled;
  repeated teardowns queue and no-op on the missing workspace).

Overlapping terraform is serialized by the existing DynamoDB state lock, so
cleanup is reliable regardless of event ordering.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
github-actions Bot added a commit that referenced this pull request Jul 1, 2026
@nourshoreibah nourshoreibah merged commit df1e9fb into main Jul 1, 2026
13 checks passed
@nourshoreibah nourshoreibah deleted the fix/preview-teardown-concurrency branch July 1, 2026 06:19
github-actions Bot added a commit that referenced this pull request Jul 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant