Add an option to setup coupled MALI-SLM runs in the ismip6 run testgroup

[hollyhan]

This PR adds to the /landice/ismip6_run/ismip_ais_proj2300 testcase the option to setup coupled MALI-Sea Level Model simulations using the ismip6-Antarctica-2300 experimental protocol.

With these changes, inclusion of the SLM can be activated through a test case cfg option. This will set up required namelist and input settings, as well as the namelist.sealevel file used by SLM. It will also set up an additional test case step that generates the required mapping files between MALI and SLM. This must be run before any of the experiments are run.

This PR requires addition of Pyproj string projections['ais-bedmap2-sphere'] = '+proj=stere +lat_ts=-71.0 +lat_0=-90 +lon_0=0.0 +k_0=1.0 +x_0=0.0 +y_0=0.0 +ellps=sphere' in the MPAS-Tools script set_lat_lon_fields_in_planar_grid.py to set the latitude and longitude values as if MALI was projected on sphere rather than ellipse. The requirement has been merged here (https://github.com/MPAS-Dev/MPAS-Tools/pull/559/files).

[hollyhan]

Hi @xylar and @matthewhoffman, the PR is ready to be reviewed.

The testing directory is in Chicoma (/lustre/scratch4/turquoise/hollyhan/folder_test_compass_ismip6run/landice/ismip6_run/ismip6_ais_proj2300/expAE01_04) where any climate group member can access.

Testing (done in Chicoma, with the compass conda environment 1.2.0-alpha.8 version):
In addition to editing and adding the namelist.landice and namelist.sealevel to the run folder setup by compass when sea-level model is coupled to MALI during compass setup, compass run in the work directory creates the mapping files mapfile_mali_to_slm.nc and mapfile_slm_to_mali.nc. Below is the logger message:

Loading conda environment
Done.

Loading Spack environment...

Lmod is automatically replacing "cce/16.0.0" with "gcc/12.2.0".

Done.

compass calling: compass.run.serial._run_test()
  in /lustre/scratch4/turquoise/.mdt4/hollyhan/compass/compass/run/serial.py


compass calling: compass.landice.tests.ismip6_run.ismip6_ais_proj2300.set_up_experiment.SetUpExperiment.runtime_setup()
  inherited from: compass.step.Step.runtime_setup()
  in /lustre/scratch4/turquoise/.mdt4/hollyhan/compass/compass/step.py


compass calling: compass.landice.tests.ismip6_run.ismip6_ais_proj2300.set_up_experiment.SetUpExperiment.run()
  in /lustre/scratch4/turquoise/.mdt4/hollyhan/compass/compass/landice/tests/ismip6_run/ismip6_ais_proj2300/set_up_experiment.py

creating scripfile for the SLM grid with 2048 Gauss-Legendre points in latitude
Running: ncremap -g slm_nglv2048scripfile.nc -G latlon=2048,4096#lat_typ=gss#lat_drc=n2s
ncremap: WARNING slm_nglv2048scripfile.nc already exists and will be overwritten by newly created grid

Running: set_lat_lon_fields_in_planar_grid.py --file mali_meshfile.nc --proj ais-bedmap2-sphere
== Gathering information.  (Invoke with --help for more details. All arguments are optional)

Using ais-bedmap2-sphere projection, defined as: +proj=stere +lat_ts=-71.0 +lat_0=-90 +lon_0=0.0 +k_0=1.0 +x_0=0.0 +y_0=0.0 +ellps=sphere
Input file xCell min/max values: -2545830.9970442397 2785860.086302431
Input file yCell min/max values: -2189345.7664010245 2281377.989714217
Calculated latCell min/max values (radians): -1.5695348162256555 -1.094701146681904
Calculated lonCell min/max values (radians): 7.204743572800302e-06 6.283164667290495
Lat/lon calculations completed.  File has been written.

creating scrip file for the mali mesh
 -- Landice Masks are disabled

 -- WARNING: sphereRadius<0 so setting sphereRadius = 6371220.0
 -- WARNING: 'on_a_sphere' attribute is 'NO', which means that there may be some disagreement regarding area between the planar (source) and spherical (target) mesh
Input latCell min/max values (radians): -1.5695348162256555, -1.094701146681904
Input lonCell min/max values (radians): 7.204743572800302e-06, 6.283164667290495
Calculated grid_center_lat min/max values (radians): -1.5695348162256555, -1.094701146681904
Calculated grid_center_lon min/max values (radians): 7.204743572800302e-06, 6.283164667290495
Calculated grid_area min/max values (sq radians): 2.6159873831700514e-07, 1.0476949449951053e-05
Creation of SCRIP file is complete.
creating MALI -> SLM grid mapfile with bilinear method
Running: srun ESMF_RegridWeightGen -s mali_scripfile.nc -d slm_nglv2048scripfile.nc -w mapfile_mali_to_slm.nc -m conserve -i -64bit_offset --netcdf4 --src_regional
 Starting weight generation with these inputs:
   Source File: mali_scripfile.nc
   Destination File: slm_nglv2048scripfile.nc
   Weight File: mapfile_mali_to_slm.nc
   Source File is in SCRIP format
   Source Grid is a regional grid
   Source Grid is an unstructured grid
   Use the center coordinates of the source grid to do the regrid
   Destination File is in SCRIP format
   Destination Grid is a global grid
   Destination Grid is a logically rectangular grid
   Use the center coordinates of the destination grid to do the regrid
   Regrid Method: conserve
   Pole option: NONE
   Ignore unmapped destination points
   Output weight file in NetCDF4 file format
   Line Type: greatcircle
   Norm Type: dstarea
   Extrap. Method: none

 Completed weight generation successfully.
  ...
...
SLM -> MALI mesh mapfile with blinear method
Running: srun ESMF_RegridWeightGen -s slm_nglv2048scripfile.nc -d mali_scripfile.nc -w mapfile_slm_to_mali.nc -m bilinear -i -64bit_offset --netcdf4 --dst_regional
 Starting weight generation with these inputs:
   Source File: slm_nglv2048scripfile.nc
   Destination File: mali_scripfile.nc
   Weight File: mapfile_slm_to_mali.nc
   Source File is in SCRIP format
   Source Grid is a global grid
   Source Grid is a logically rectangular grid
   Use the center coordinates of the source grid to do the regrid
   Destination File is in SCRIP format
   Destination Grid is a regional grid
   Destination Grid is an unstructured grid
   Use the center coordinates of the destination grid to do the regrid
   Regrid Method: bilinear
   Pole option: ALL
   Ignore unmapped destination points
   Output weight file in NetCDF4 file format
   Line Type: cartesian
   Norm Type: dstarea
   Extrap. Method: none

 Completed weight generation successfully.
  ...
  ...
Removing the temporary mesh and scripfiles...
Warning: replacing namelist options in namelist.landice
config_pio_num_iotasks = 4
config_pio_stride = 128
Running: gpmetis graph.info 512
.
.
.

The contents in the work directory look as follows:

hollyhan@ch-fe1:/lustre/scratch4/turquoise/hollyhan/folder_test_compass_ismip6run/landice/ismip6_run/ismip6_ais_proj2300/expAE01_04> ls
AIS_4to20km_r01_20220907.regionMask_ismip6.nc				      
ICELOAD_SLM
namelist.sealevel
AIS_4to20km_r01_20220907_smb_NorESM1-M_RCP26-repeat_2300_minus1_bare_land.nc  
ismip6_ais_proj2300.cfg  
OUTPUT_SLM
AIS_4to20km_r01_20220907_TF_NorESM1-M_RCP26-repeat_2300.nc		     
job_script.sh	       
restart_timestamp
albany_input.yaml							      
landice_model	      
 rst.2015-01-01.nc
basin_and_coeff_gamma0_DeltaT_quadratic_non_local_after_relaxation.nc	      
load_compass_env.sh      
step.pickle
expAE01.o10070767		
mapfile_mali_to_slm.nc   
streams.landice
graph.info								      
mapfile_slm_to_mali.nc
graph.info.part.512

Note that in addition to namelist.sealevel and the mapping files, the folders needed for the SLM outputs ICELOAD_SLM and OUTPUT_SLM are created as well.

[matthewhoffman]

@hollyhan , this is fantastic to get the SLM coupled configuration automated in compass! I have a number of requested changes, most of them small or straightforward, but there are a couple items that might require some further discussion to decide on the best solution.

[matthewhoffman]

@hollyhan , I've moved the mapping file generation to a new step and confirmed that mapping files are created when it is run. I still need to update the docs, but feel free to try it out and let me know if you run into any issues.

One thing I realized is that all the experiments are set up as restarts from the hist run, so we will need to handle the SLM restart files somehow (or modify the experiments to not be restarts from hist). Let me know if you have thoughts about that.

[matthewhoffman]

@hollyhan , thinking about it more, the easiest thing would be to add instructions that a user needs to manually copy the SLM output from the hist run into the projections runs prior to conducting them.

[matthewhoffman]

@hollyhan , I updated the docs with the requirement to manually copy the OUTPUT_SLM directory from the hist run for the projection runs. With that and the changes you added today, I think all the details are taken care of. Can you update the PR description and let me know if you have any feedback on the new step I added for creating the mapping files? Once you sign off, I'm happy to have this merged.

@xylar , let me know if you wanted to take another look or are happy to consider this complete.

[xylar]

@matthewhoffman and @hollyhan, I think you should proceed. I don't think I'll have time to take another look this week and I don't want to hold things up.

[matthewhoffman]

Thanks, @xylar . Holly said she'll take a final look today as she is setting up new runs on Chicoma. Assuming no new issues come up, I'll merge this after her thumb's up.

[hollyhan]

@matthewhoffman, thanks for updating the document! I confirmed that everything looks and works great. I think it's ready to be merged!