PKG-1331: codify Jenkins S3 buckets in IaC for CloudFormation import - #4083
Open
nogueiraanderson wants to merge 12 commits into
Open
PKG-1331: codify Jenkins S3 buckets in IaC for CloudFormation import#4083nogueiraanderson wants to merge 12 commits into
nogueiraanderson wants to merge 12 commits into
Conversation
Follow-up to PKG-1330. Adds PxbBuildCacheBucket to IaC/pxb.cd/S3.yaml. - Properties match the bucket's current state (AES256 default encryption, PublicAccessBlock all-false) so a CF IMPORT change set lands without triggering drift. - DeletionPolicy / UpdateReplacePolicy = Retain to protect build cache contents. - No tags added; current bucket has none. - Hardening (enable PublicAccessBlock, add billing tags) is intentionally out of scope. - IMPORT change set commands documented in the template header.
Follow-up to PKG-1330 / PR #4082. Adds CloudFormation templates that declare each persistent S3 bucket per master, plus the cross-fleet artifactory bucket as its own top-level stack. Properties match each bucket's current state so a CF IMPORT change set lands without drift. - IaC/pmm.cd/S3.yaml percona-vm (us-east-1), pmm-build-cache (us-east-2) - IaC/ps80.cd/S3.yaml ps-build-cache (us-east-2; shared by ps57.cd, ps80.cd, ps3.cd) - IaC/pxc.cd/S3.yaml pxc-build-cache (us-west-2) - IaC/rel.cd/S3.yaml jenkins-percona-jobs-backup (us-east-1), rel-repo-cache (eu-west-1) - IaC/JenkinsArtifactory.yml percona-jenkins-artifactory (us-east-1; cross-fleet) DeletionPolicy / UpdateReplacePolicy = Retain on every resource. Stale *-build-cache IAM grants in pg.cd / rel.cd / cloud.cd JenkinsStack.yml, hardening (PublicAccessBlock, billing tags), and the actual one-shot CF IMPORT executions are intentionally out of scope.
nogueiraanderson
changed the title
A CloudFormation stack is regional, so a single template cannot hold buckets that live in different AWS regions. Codex review flagged the two original templates as cross-region. - IaC/pmm.cd/S3.yaml -> us-east-2 only (pmm-build-cache) - IaC/pmm.cd/S3-us-east-1.yaml -> percona-vm - IaC/rel.cd/S3.yaml -> eu-west-1 only (rel-repo-cache) - IaC/rel.cd/S3-us-east-1.yaml -> jenkins-percona-jobs-backup Master-region buckets stay in the canonical S3.yaml; the cross-region buckets get an S3-<region>.yaml file deployed as a separate stack in that region.
nogueiraanderson
changed the title
CF refuses to modify or add outputs as part of an IMPORT change set. Revert S3.yaml outputs to match what pxb-backup-ci-bucket was deployed with (BucketName + BucketArn). New outputs can be added in a follow-up UPDATE.
CloudFormation rejects IMPORT change sets that include Outputs (or Parameters / Conditions) when creating a new stack. Strip those sections from the 7 templates that target brand-new stacks. They can be added back via update-stack after import.
CloudFormation drift detection treats template Prefix: '' as different from
actual unset Prefix (Filter={}). Omit the line entirely for catch-all rules
in pxc-build-cache (rule 3) and pmm-build-cache (rule 1).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Bug: 9 Jenkins S3 buckets were ad-hoc (no IaC tracking).
Fix: 8 CloudFormation templates: per-master
IaC/<master>.cd/S3.yamlplus top-levelIaC/JenkinsArtifactory.yml. All 9 buckets imported via CF IMPORT;detect-stack-driftreturnsIN_SYNCon every stack.DeletionPolicy: Retainon every resource.Tickets: PKG-1331 (this), PKG-1330 (origin).