Compilation error with development code using derecho_intel modules

[weiwilliam]

I'm trying to build SCM with -DCCPP_SUITES=ALL on Derecho.
I compiled it with 1 task and 128 GB of memory on a compute node.
When it comes to the end, it encountered

#0 0x00000000033517b1
#1 0x00000000033b6f87
#2 0x00000000033b70b5
#3 0x000014f8426c6900
#4 0x0000000002b2857a
#5 0x0000000002397bfe
#6 0x0000000002397ac6
#7 0x00000000026fc07d
#8 0x00000000026fefee
#9 0x000000000277b119
#10 0x00000000023834e8
#11 0x0000000002488b68
#12 0x00000000024885dc
#13 0x000000000248e350
#14 0x00000000032eef5e
#15 0x00000000032ec642
#16 0x0000000003295ef5
#17 0x0000000003476d9a
#18 0x000014f8426afe6c
#19 0x000014f8426aff35 __libc_start_main + 135
#20 0x00000000030d09ee

/var/tmp/pbs.6825927.desched1/ifx0509485308lB3Fdx/ifxGRI0ul.i90: error #5633: **Internal compiler error: segmentation violation signal raised** Please report this error along with the circumstances in which it occurred in a Software Problem Report.  Note: File and line given may not be explicit cause of this error.
compilation aborted for /glade/work/swei/projects/waterisotope/repos/scm-test1/scm/src/ccpp_static_api.F90 (code 3)
make[2]: *** [CMakeFiles/scm.dir/build.make:182: CMakeFiles/scm.dir/ccpp_static_api.F90.o] Error 3
make[1]: *** [CMakeFiles/Makefile2:194: CMakeFiles/scm.dir/all] Error 2
make: *** [Makefile:136: all] Error 2

It works fine with default suites (w/o specifying).

Does anyone know what happened and how to solve it?
I know sometimes it could happen when OOM, but it shouldn't be the case here given 1 task with 128 GB.

[scrasmussen]

@weiwilliam Hi, that might be occurring because the new Intel ifx compiler deals with internal representation differently than ifort, and that sometimes causes issues. We have had issues building all the suites with Intel, but they should work with GNU. I would recommend trying GNU to see if it will fit your needs.

Also, we recently modified suite_info.py and tort_test_cases.py to give better subsets of the suites, specifically because ifx was having problems compiling all of them. So you could use those to see sets of suites that work.

If you have any other questions, please let us know!

[weiwilliam]

@scrasmussen Thank you for the insights! The CCPP-SCM doc can note that GNU is recommended for compiling all the suites.
Also, I found the bomex test case needs _ps suite. It may be worth noting that a user cannot compile only one specific suite and test with it in the doc.