Skip to content

indexer: add indexer_enabled teardown gate for internal-mainnet wind-down - #170

Open
UnbornAztecKing wants to merge 2 commits into
mainfrom
mattweeks/eng-3056-tear-down-internal-mainnet-indexer-apne-1-retain-restore
Open

indexer: add indexer_enabled teardown gate for internal-mainnet wind-down#170
UnbornAztecKing wants to merge 2 commits into
mainfrom
mattweeks/eng-3056-tear-down-internal-mainnet-indexer-apne-1-retain-restore

Conversation

@UnbornAztecKing

@UnbornAztecKing UnbornAztecKing commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

Winds down the internal-mainnet indexer to zero cost while keeping its Terraform state and config intact for a one-switch rebuild. The gate is derived from environment — mainnet is torn down by default and no errant apply can re-provision it — so every other workspace is untouched.

What it is

A teardown gate on the shared indexer/ root. Internal mainnet is the only environment = "mainnet" workspace, so the behavior is derived, not variable-driven:

Workspace Result
internal mainnet full teardown, effective on merge
dev / staging / testnet unchanged (no-op)
mainnet + indexer_enabled = true rebuild (restore)

Capabilities

Property Delivered
Isolation Non-mainnet workspaces see zero resource changes.
Teardown durability The torn-down state is intrinsic to the workspace; an apply can only re-confirm it, never re-provision.
State preservation Existing state migrates in place — no destroy+recreate — and config is retained.
Restore A single override switch plus a production DB snapshot rebuilds the topology.
Dependency safety External node images and service secrets are read-only and never destroyed.

Testing

  • terraform validate (tf 1.3.2) and fmt pass; validate exercises the non-mainnet path and proves every gated reference resolves.
  • terraform console confirms the derived values per environment — mainnet off, others on, override restores.
  • Structural sweep: no resource or module is left ungated; delete guards are all switch-driven.
  • Merge gate: speculative plans on dev / staging / testnet must each show 0 resource changes.

Reviewer brief

  • teardown.tf — the whole contract: two override variables and the environment-derived gate. Confirm it keys on environment == "mainnet".
  • locals.tf — the service/AZ source collections that empty out when disabled, collapsing every for_each.
  • teardown_moved.tf — 54 state-address migrations; a missing one makes a workspace propose destroy+recreate (caught by the merge gate).

Everything else is the mechanical gating those three drive.

Out of scope

  • metric_ingestor/ and amplitude_api_gateway/ workspaces.
  • External metadata and standalone full-node services (other accounts).
  • Deleting the TF Cloud workspace, its variables, or state history — retained for restore.

Test plan

  • terraform validate + fmt (tf 1.3.2)
  • terraform console — derived values per environment
  • Structural sweep — 0 ungated resources/modules, guards switch-driven
  • Speculative plan: dev / staging / testnet → 0 resource changes
  • Mainnet plan → destroy-only, no creates/updates

…down (ENG-3056)

Introduce two workspace-scoped switches that let the internal-mainnet indexer be
torn down while retaining Terraform state + config for an emergency rebuild:

  - indexer_enabled (default true): master existence gate. Every resource/module
    is count-gated on it, and the service/AZ source locals (services,
    lambda_services, service_names, azs) empty out when false so every for_each
    collapses. false => full destroy of all indexer + indexer-full-node infra.
  - indexer_teardown_disarm (default false): drops delete guards one apply before
    the destroy — RDS deletion_protection and the snapshot bucket force_destroy.

Both default to today's behavior, so dev/staging/testnet (which share this root
in separate workspaces) are unaffected. teardown_moved.tf carries 54 moved blocks
so adding count to previously count-less resources migrates existing state
(default indexer_enabled=true) with zero resource changes instead of
destroy+recreate.

Also drops the prevent_destroy lifecycle literals on the main ElastiCache group
and ACM cert (cannot be variable-driven) so the gated destroy can proceed.

Validated with `terraform validate` (tf 1.3.2). Speculative plans on
dev/staging/testnet (expected: 0 resource changes) gate the merge.
@linear-code

linear-code Bot commented Jul 22, 2026

Copy link
Copy Markdown

ENG-3056

@wiz-55df730c58

wiz-55df730c58 Bot commented Jul 22, 2026

Copy link
Copy Markdown

Wiz Scan Summary

Scanner Findings
Vulnerability Finding Vulnerabilities -
Data Finding Sensitive Data -
Secret Finding Secrets -
IaC Misconfiguration IaC Misconfigurations 1 Low
SAST Finding SAST Findings -
Software Management Finding Software Management Findings -
Total 1 Low

View scan details in Wiz

To detect these findings earlier in the dev lifecycle, try using Wiz Code VS Code Extension.

Make the internal-mainnet teardown intrinsic to the workspace instead of
dependent on a workspace variable that could be unset or forgotten. Internal
mainnet is the only workspace with environment = "mainnet", so derive the gate:

  local.indexer_enabled         = override ?? (environment != "mainnet")
  local.indexer_teardown_disarm = override ?? (environment == "mainnet")

The indexer_enabled / indexer_teardown_disarm variables become nullable overrides
(default null). All gating now references the locals. Effect:

  - mainnet: enabled=false, disarm=true -> full teardown, effective on merge, and
    an errant apply cannot re-provision (it can only re-confirm torn-down state).
  - dev/staging/testnet: values identical to before -> no-op (0 resource changes).
  - restore: set indexer_enabled=true (and indexer_teardown_disarm=false) on the
    mainnet workspace.

Verified derived values per environment with terraform console; terraform validate
(tf 1.3.2) passes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant