Add nghost to namelist, validate nghost=2

[apcraig]

PR checklist

• Short (1 sentence) summary of your PR:
  Add nghost to namelist, validate nghost=2
• Developer(s):
  apcraig
• Suggest PR reviewers from list in the column to the right.
• Please copy the PR test results link or provide a summary of testing completed below.
  expect bit-for-bit, testing underway
• How much do the PR code changes differ from the unmodified code?
  • bit for bit
  • different at roundoff level
  • more substantial
• Does this PR create or have dependencies on Icepack or any other models?
  • Yes
  • No
• Does this PR update the Icepack submodule? If so, the Icepack submodule must point to a hash on Icepack's main branch.
  • Yes
  • No
• Does this PR add any new test cases?
  • Yes
  • No
• Is the documentation being updated? ("Documentation" includes information on the wiki or in the .rst files from doc/source/, which are used to create the online technical docs at https://readthedocs.org/projects/cice-consortium-cice/. A test build of the technical docs will be performed as part of the PR testing.)
  • Yes
  • No, does the documentation need to be updated at a later time?
    • Yes
    • No
• Please document the changes in detail, including why the changes are made. This will become part of the PR commit log.

Add support for nghost>1 and validate. This is needed for open boundary conditions (regional grids) using remap advection. The remap advection implementation needs two layers of halo cells. This is satisfied with nghost=1 with closed, cyclic, and tripole bcs, not not for open bcs.

Test and validate nghost=2, add fixes where needed

• Fix bug in gatherscatter, only impacts nghost > 1
• Modify remap advection departure points to only check where velocity is not zero
• Add a check in remap advection if open boundaries are used, nghost must be >= 2.

Add nghost=2 tests to test suites

Update full2nest.sh script and full/nest test to use nghost=2 for nest case

Modify order of domain_nml output and checks so output is first

Add check to ignore "buildincremental" in test setup

Update error message in ice boundary if northern-most tripole block is not large enough

Update documentation

Tested several cases to verify nghost=1 and nghost=2 are bit-for-bit. Added unit tests that check nghost=2. Verified nghost=2 now fixes the remap advection issues that occurred in the full/nest regional testcase.

[eclare108213]

This looks good to me. Nice debugging.
GitHub Actions is failing ... GitHub's AI helper says

The job is failing because the remap advection scheme requires nghost >= 2 when using regional boundary conditions (open, zero_gradient, or linear_extrap), but the configuration is set to nghost = 1.

The error occurs at line 291 in ice_transport_remap.F90

:)

[apcraig]

Thanks @eclare108213, doing comprehensive testing now. Looks like the displaced pole grids have open bcs in ns but there is land on both the north and south edge. OK, need to adjust for that in my check in remap advection.

[eclare108213]

What happens when a boundary type = 'open'? Does the code abort or provide a warning?

[apcraig]

'open' can only be used when there is land on the entire open boundary. That's what we use for the displaced pole grids in the ns direction. The 'open' boundary_type does nothing at the boundary in practice, halo values on the outer boundary are unset. If 'open' were to require nghost=2 for remap advection, then all our displaced pole grids would need to use nghost=2 or we'd have to change the boundary_type for this configuration.

We don't really have an obvious way to tell whether a case truly needs nghost=2 or not. It does if the boundary is not cyclic, not closed, not tripole and not all land. For now, I'm assuming 'open' goes with all land which means only the zero_gradient and linear_extrap cases need nghost=2 when remap advection is on. That's not a perfect check.