Skip to content

Feature/nctcog integration - #1

Merged
sclaw merged 3 commits into
masterfrom
feature/nctcog-integration
Oct 6, 2026
Merged

sclaw merged 3 commits into
masterfrom
feature/nctcog-integration

Conversation

@sclaw

@sclaw sclaw commented Oct 6, 2026

Copy link
Copy Markdown
Collaborator

Support NCTCOG integration with explicit FIM paths and ECR deployment

Decouple FIM discovery from a fixed directory layout so scenario data can point to maps stored in external library layouts, and add AWS ECR infrastructure and workflows to validate and publish the container image.

Additions

  • Add a fim_path scenario column to controls output, with actionable errors and SQL guidance for databases that do not yet provide the column.
  • Add AWS CDK infrastructure for an immutable, scan-on-push ECR repository, plus workflows to validate/deploy the infrastructure and publish multi-architecture images tagged by commit SHA.
  • Update reference controls files, database fixtures, and Linux/macOS test scripts for fim_path behavior.

Removals

  • Remove validate's -o_unexpected_fims output and the check that inferred unexpected scenarios from the legacy folder structure.
  • Remove FIM-path derivation from <reach_id>/z_<stage>/f_<flow>.tif for compositing and validation.

Changes

  • fim now reads paths from the controls file's fim_path column, resolving relative paths against -lib; absolute and GDAL VSI paths are used as-is.
  • validate matches library TIFFs against scenario fim_path values and updates its output columns. domain.tif retains its existing per-reach location when requested.
  • The change is breaking for older scenario databases and controls files; see Notes for the migration requirement.

Testing

  1. Updated Linux and macOS integration scripts to cover fim_path output, mapless scenarios, and rejection of controls files without the path column. These suites were not run while preparing this description.
  2. The infrastructure workflow runs Ruff, Pyright, and cdk synth for pull requests; these checks were not run locally.
  3. git diff --check origin/master...HEAD reports trailing whitespace in .github/workflows/infrastructure.yml; remove it and rerun the check.

Screenshots

Not applicable; this PR changes CLI, data, and infrastructure behavior without a user interface.

Notes

  • controls and validate require scenarios.fim_path. Use a compatible ripple1d-pipeline database or apply the SQL backfill printed by the command for libraries using the legacy layout.
  • The ECR workflows require the configured GitHub variables AWS_REGION, ECR_PUSH_ROLE_ARN, CDK_DEPLOY_ROLE_ARN, and CFN_EXECUTION_ROLE_ARN.
  • This checkout has master but no local main ref; the comparison and checks above use origin/master. Confirm the intended GitHub PR base.

Todos

  • Remove the reported trailing whitespace and rerun git diff --check.
  • Run the Linux/macOS integration suites and infrastructure validation in CI.
  • Confirm the PR base branch and required AWS GitHub variables are configured.

Checklist

  • PR has an informative and human-readable title
  • Changes are limited to a single goal (no scope creep)
  • Code can be automatically merged (no conflicts)
  • Code follows project standards (link if applicable)
  • Passes all existing automated tests
  • Any change in functionality is tested
  • New functions are documented (with a description, list of inputs, and expected output)
  • Placeholder code is flagged / future todos are captured in comments
  • Project documentation has been updated (including the "Unreleased" section of the CHANGELOG)
  • Reviewers requested with the Reviewers tool ➡️

Testing checklist

Target Environment support

  • Windows
  • Linux
  • Browser (not applicable to this CLI/infrastructure change)

Accessibility

  • Keyboard friendly (not applicable; no UI changes)
  • Screen reader friendly (not applicable; no UI changes)

Other

  • Is useable without CSS (not applicable; no web UI)
  • Is useable without JS (not applicable; no web UI)
  • Flexible from small to large screens (not applicable; no web UI)
  • No linting errors or warnings
  • JavaScript tests are passing (not applicable; no JavaScript changes)

@sclaw
sclaw merged commit acbc295 into master Oct 6, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant