Adds the thermo coupling by alicebarthel · Pull Request #461 · E3SM-Project/Omega

alicebarthel · 2026-07-06T22:49:28Z

This PR adds the thermo coupling - from forcing terms to thickness and tracer tendencies.
The energy of mass fluxes and phase changes are hard-coded into the tendency conversions, like it was in MPAS-O. Hopefully the documentation (inline or otherwise) clarifies the meaning of each term.

This PR is 3/3 to make #418 more digestible.

Checklist

vanroekel · 2026-07-10T19:42:06Z

+         const Real PTopDb = PressureMid(ICell, KTop) * Pa2Db;
+         const Real SaTop  = SaltIndex >= 0
+                                 ? TracerCell(SaltIndex, ICell, KTop)
+                                 : 0.0_Real; // not sure we want zero here?


I think 0 is a fine limit. Does this form of code preclude logging when we go below this threshold?

@vanroekel note that this conditional is checking the validity of the tracer index (SaltIndex >= 0), not the value of salinity. For a check on salinity, it would be preferable to use Kokkos::clamp().
This was to make sure we are not using an invalid index, but we could do a clean fail rather than use 0 salinity...

I'm not sure that it's the job of this function to make sure that the tracer indices are initialized correctly. I'd be inclined to check for that elsewhere (and maybe not every timestep or every time forcing is updated) and remove the conditional. Thoughts?

Yea, I would agree. Are you thinking something like OceanInit, @cbegeman?

Yes, maybe Tracers::Init which is called during OceanInit?

@katsmith133 Should this be resolved or is this work still outstanding?

It seems like this is resolved looking at the code, but @katsmith133 can verify.

vanroekel · 2026-07-10T19:59:55Z

overall this looks quite good @alicebarthel I only had pretty minor comments

alicebarthel · 2026-07-13T22:53:30Z

One key test to do is to verify that omega_pr is BFB with develop baseline since we do not currently have a polaris test using the surface tracer forcing. @katsmith133

Here is my current test report:

Polaris `omega_pr` suite

Baseline workdir: /global/homes/a/abarthel/pscratch/polaris-scratch/baseline-current-develop-cpu
Baseline build: /global/homes/a/abarthel/pscratch/omega_scratch/develop-cpu
PR build: /global/homes/a/abarthel/pscratch/omega_scratch/add-thermo-forcing-cpu-gnu
PR workdir: /global/homes/a/abarthel/pscratch/polaris-scratch/add-thermo-cpu_vsbaseline
Machine: pm-cpu
Compiler: gnu
Build type: Debug
Log: not found
Result:
- Diffs (1 of 12):
  - ocean/planar/barotropic_gyre/munk/free-slip

alicebarthel · 2026-07-14T00:50:46Z

the log is at
/pscratch/sd/a/abarthel/polaris-scratch/add-thermo-cpu_vsbaseline/case_outputs/ocean_planar_barotropic_gyre_munk_free-slip.log
if that can be helpful

cbegeman · 2026-07-15T17:09:05Z

@alicebarthel Is this ready to test with standalone Omega (I'm thinking column cases) with the flux fields provided in the forcing stream?

alicebarthel · 2026-07-15T17:23:56Z

Yes @cbegeman additional testing with single-column polaris cases (and analytical equilibrated state) or more realistic cases could be performed at this stage. It may be good to resolve omega_pr fail above before then, though? (I will not be able to debug this myself, sorry).
If it can be helpful for the additional testing, I generated forcing files with realistic values (taken from an existing simulation) and the necessary variables for various meshes, stored on pm:
in /global/homes/a/abarthel/pscratch/omega_scratch/
forcing_nCells_7153.nc [forcing.nc]
forcing_nCells_2304.nc [forcingPlanar.nc]
forcing_nCells_2562.nc [forcingSphere.nc]
You may prefer to start with idealized forcing but thought I'd let you know.

katsmith133 · 2026-07-16T18:30:51Z

Pulled this PR down onto PM this morning and verified it passes all CTests on PM-CPU and GPU. Also, the polaris omega_pr suite passed BFB on PM-CPU for me so perhaps @alicebarthel you had something wonky with your set up.

Polaris `omega_pr` suite

Baseline workdir: /pscratch/sd/k/katsmith/polaris_PR461//baseline_omega_pr
Baseline build: /pscratch/sd/k/katsmith/polaris-main/omega_build
PR build: /pscratch/sd/k/katsmith/polaris-PR461/omega_build
PR workdir: /pscratch/sd/k/katsmith/polaris_PR461/pr461_omega_pr
Machine: pm-cpu
Compiler: gnu
Build type: Release
Log: not found
Result: All tests passed

@cbegeman, I think it should be safe to test with single-column now if you want

katsmith133 · 2026-07-16T18:47:53Z

Pretty minor comment for the PR. Seems like overall its pretty good, but I'd also like to see the results of the single-column test. @cbegeman can I help in any way with this?

katsmith133 · 2026-07-16T19:03:37Z

Was just checking the compile logs and saw this warning for PM-GPU:

TracerForcingVars.cpp(198): warning #20011-D: calling a __host__ function("OMEGA::Eos::calcPtFromCt(const double &, const double &) const") from \
a __host__ __device__ function("OMEGA::TracerForcingVars::computeSurfInsituTemp(const  ::Kokkos::View<double ***,  ::Kokkos::LayoutRight,  ::Kokkos::CudaSpace >  &, const  ::OMEGA::VertCoord *, const  ::OMEGA::Eos *) c\
onst::[lambda(int) (instance 1)]::operator () const") is not allowed

working on how to resolve this issue...

katsmith133 · 2026-07-16T22:50:38Z

Was just checking the compile logs and saw this warning for PM-GPU:

TracerForcingVars.cpp(198): warning #20011-D: calling a __host__ function("OMEGA::Eos::calcPtFromCt(const double &, const double &) const") from \
a __host__ __device__ function("OMEGA::TracerForcingVars::computeSurfInsituTemp(const  ::Kokkos::View<double ***,  ::Kokkos::LayoutRight,  ::Kokkos::CudaSpace >  &, const  ::OMEGA::VertCoord *, const  ::OMEGA::Eos *) c\
onst::[lambda(int) (instance 1)]::operator () const") is not allowed

working on how to resolve this issue...

Ok, I figured out a solution to this that removes the compile warning, but I am now adding in Linear and Constant EOS support to this PR, which primarily is just adding non-TEOS10 options to the calcCtFreezing function. I believe I have that finished, but am working on testing. Note: Alice suggests (on L355-356 of Eos.cpp) that we use the UNESCO 1983 polynomial for any of the non-TEOS10 options, but I think this would be inconsistent? So for now I've implemented a linear option and a constant option. I would like feedback on that once I have it tested and pushed to this PR.

This reverts commit d32ee4d.

katsmith133 · 2026-08-18T04:10:56Z

Retesting on Frontier after latest enthalpy calculation changes and rebasing to `develop`

CTest unit tests:

Machine: Frontier
Compiler: craygnu, craygnu-mphipcc
Build type: Release
Result: All tests passed

Polaris `omega_pr` suite

Baseline workdir: /lustre/orion/cli115/scratch/kmsmith/polaris_testing_pr461/main_omega_pr
Baseline build: /ccs/home/kmsmith/polaris-main/omega_build
PR build: /ccs/home/kmsmith/polaris-pr461/omega_build
PR workdir: /lustre/orion/cli115/scratch/kmsmith/polaris_testing_pr461/pr461_omega_pr
Machine: frontier
Partition: batch
Compiler: craygnu-mphipcc
Build type: Release
Log: not found
Result:
- Failures (1 of 22):
  - ocean/column/ekman

ocean/column/ekman test failure is expected.

@cbegeman I think we can finish up with updating polaris and doing the final conservation testing.

cbegeman · 2026-08-18T13:27:50Z

@katsmith133 Thanks for testing! I also did testing last night. Results located here: E3SM-Project/polaris#663 (comment)

cbegeman · 2026-08-19T14:23:49Z

Testing

Polaris `omega_pr` suite

Baseline workdir: /lus/flare/projects/E3SM_Dec/cbegeman/polaris-output/main-omega-submodule-20260818
Baseline build: /lus/flare/projects/E3SM_Dec/cbegeman/polaris-output/main-omega-submodule-20260818/build
PR build: /lus/flare/projects/E3SM_Dec/cbegeman/polaris-output/thermo-omega-thermo-20260818/build
PR workdir: /lus/flare/projects/E3SM_Dec/cbegeman/polaris-output/main-omega-thermo-20260819
Machine: aurora
Compiler: oneapi-ifxgpu
Build type: Release
Log: not found
Result:
- Failures (1 of 22):
  - ocean/column/ekman

cbegeman · 2026-08-28T22:27:53Z

Testing, aurora

All ctests pass on aurora, cpu
All ctests but 11 featuring SCYCL pass on aurora, gpu
Omega PR tests pass on aurora, cpu
Omega PR tests pass on aurora, gpu except cosine_bell decomp, unrelated to this PR

cbegeman · 2026-08-31T13:59:02Z

Testing, chrysalis

All ctests pass on gnu
All ctests pass on intel
Omega PR tests pass on gnu
Omga PR tests pass on intel

alicebarthel · 2026-08-31T16:39:07Z

As far as I know, this is ready for a merge? @cbegeman @sbrus89 ?
I'll do the modification in the KPP PR.

cbegeman · 2026-08-31T18:44:10Z

Teos-10 results reported here E3SM-Project/polaris#663 (comment). Looks good.

cbegeman

Approving on the basis of code inspection and testing. Thanks for the great work, @alicebarthel!

katsmith133

Approved based upon visual inspection and testing done by @cbegeman

sbrus89 · 2026-09-01T16:46:37Z

Local merge passes CTests and omega_pr on Frontier with craygnu. CTests pass with craygnu-mphipcc. omega_pr tests pass with craygnu-mphipcc except for two base mesh step baseline comparison diffs (forward steps pass baseline comparisons). These diffs are unrelated to this PR.

Polaris `omega_pr` suite

Baseline workdir: /ccs/home/brus/run/polaris_baseline_omega_pr_omega_craygnu_mpich/
Baseline build: /ccs/home/brus/run/polaris_baseline_omega_pr_omega_craygnu_mpich/build
PR build: /ccs/home/brus/run/polaris_thermo_forcing_omega_pr_omega_craygnu_mpich/build
PR workdir: /ccs/home/brus/run/polaris_thermo_forcing_omega_pr_omega_craygnu_mpich
Machine: frontier
Partition: batch
Compiler: craygnu
Build type: Release
Log: not found
Result: All tests passed

Polaris `omega_pr` suite

Baseline workdir: /ccs/home/brus/run/polaris_baseline_omega_pr_omega_craygnu-mphipcc_mpich/
Baseline build: /ccs/home/brus/run/polaris_baseline_omega_pr_omega_craygnu-mphipcc_mpich/build
PR build: /ccs/home/brus/run/polaris_thermo_forcing_omega_pr_omega_craygnu-mphipcc_mpich/build
PR workdir: /ccs/home/brus/run/polaris_thermo_forcing_omega_pr_omega_craygnu-mphipcc_mpich
Machine: frontier
Partition: batch
Compiler: craygnu-mphipcc
Build type: Release
Log: not found
Result:
- Diffs (2 of 22):
  - ocean/spherical/icos/cosine_bell/decomp
  - ocean/spherical/icos/cosine_bell/restart

ocean/spherical/icos/cosine_bell/decomp
  * step: icos_base_mesh_480km
          execution:        SUCCESS
          baseline comp.:   FAIL
          runtime:          0:00:13
  * step: icos_init_480km
          execution:        SUCCESS
          baseline comp.:   PASS
          runtime:          0:00:01
  * step: 12proc
          execution:        SUCCESS
          property checks:  PASS
          baseline comp.:   PASS
          runtime:          0:00:12
  * step: 24proc
          execution:        SUCCESS
          property checks:  PASS
          baseline comp.:   PASS
          runtime:          0:00:10
  * step: validate
          execution:        SUCCESS
          runtime:          0:00:00
  task execution:   SUCCESS
  baseline comp.:   FAIL
  task runtime:     0:00:37
ocean/spherical/icos/cosine_bell/restart
  * step: icos_base_mesh_480km
          already completed
          baseline comp.:   FAIL
  * step: icos_init_480km
          already completed
          baseline comp.:   PASS
  * step: full_run
          execution:        SUCCESS
          property checks:  PASS
          baseline comp.:   PASS
          runtime:          0:00:01
  * step: restart_run
          execution:        SUCCESS
          property checks:  PASS
          baseline comp.:   PASS
          runtime:          0:00:01
  * step: validate
          execution:        SUCCESS
          runtime:          0:00:00
  task execution:   SUCCESS
  baseline comp.:   FAIL
  task runtime:     0:00:03

This PR updates the Omega submodule to incorporate the addition of thermodynamic forcing from E3SM-Project/Omega#461. This hash is B4B for all current omega_pr tests, but is needed to for the new polaris thermo forcing tests in: #663.

alicebarthel requested a review from vanroekel July 6, 2026 22:49

vanroekel reviewed Jul 7, 2026

View reviewed changes

Comment thread components/omega/doc/devGuide/Forcing.md Outdated

vanroekel reviewed Jul 7, 2026

View reviewed changes

Comment thread components/omega/doc/userGuide/Forcing.md Outdated

vanroekel reviewed Jul 7, 2026

View reviewed changes

Comment thread components/omega/doc/userGuide/Forcing.md Outdated

vanroekel reviewed Jul 7, 2026

View reviewed changes

Comment thread components/omega/doc/userGuide/Forcing.md

vanroekel reviewed Jul 7, 2026

View reviewed changes

Comment thread components/omega/src/ocn/forcingVars/TracerForcingVars.cpp Outdated

vanroekel reviewed Jul 7, 2026

View reviewed changes

Comment thread components/omega/src/ocn/Forcing.cpp Outdated