Skip to content

Add argument to write_netcdf for the NetCDF engine#434

Merged
xylar merged 2 commits intoMPAS-Dev:masterfrom
xylar:select_netcdf_engine
Jul 29, 2021
Merged

Add argument to write_netcdf for the NetCDF engine#434
xylar merged 2 commits intoMPAS-Dev:masterfrom
xylar:select_netcdf_engine

Conversation

@xylar
Copy link
Collaborator

@xylar xylar commented Jul 28, 2021

Performance for the default netcdf4 engine has found to be poor for large files on Chrysalis and Anvil, and scipy seems to perform better. For now, netcdf4 remains the default but the option to change is now provided.

To aid in this process, default values have been defined in the mpas_tools.io module that can be modified once, rather than
each time write_netcdf() is called.

The planar_hex tool and mask creation tools have been modified to take the format and engine as arguments, so that calling code can have more control of these if needed.

@xylar xylar self-assigned this Jul 28, 2021
@xylar
Copy link
Collaborator Author

xylar commented Jul 28, 2021

Python 3.6 testing is failing because xarray has dropped support. We'll do the same in #435

xylar added 2 commits July 28, 2021 21:49
Performance for the default 'netcdf4' engine has found to be poor
for large files, and 'scipy' seems to perform better.  For now,
'netcdf4' remains the default but the option to change is now
provided.

To aid in this process, default values have been defined in the
mpas_tools.io module that can be modified once, rather than
each time `write_netcdf` is called.
The planar_hex tool and mask creation tools have been modified
to take the format and engine as arguments, so that calling code
can have more control of these if needed.
@xylar
Copy link
Collaborator Author

xylar commented Jul 28, 2021

Testing

As reported here MPAS-Dev/compass#187 (comment), I ran the nightly ocean test suite successfully with this branch and MPAS-Dev/compass#187 on Ubuntu with gnu and Chrysalis with intel, comparing against a baseline of the current master.

The ssh_adjustment step with the SOwISC12to60 mesh now works like a charm, so this does seem to be a good solution to MPAS-Dev/compass#185.

@xylar xylar marked this pull request as ready for review July 29, 2021 06:09
@xylar
Copy link
Collaborator Author

xylar commented Jul 29, 2021

@mark-petersen and @matthewhoffman, I'm going to go ahead and merge this and make an MPAS-Tools release. You can then test these changes as part of reviewing MPAS-Dev/compass#187. It will be tricky to test both this PR and that one at the same time so I'd prefer to do it that way, since I'm pretty confident these changes are working as expected.

@xylar xylar merged commit 08f7151 into MPAS-Dev:master Jul 29, 2021
@xylar xylar deleted the select_netcdf_engine branch July 29, 2021 06:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant