#Creating the files needed for MPAS-Ocean/Seaice in EarthWorks Input files needed by EarthWorks: Required: Mesh and Input file - this has all the mpas metrics plus initial conditions for the ocean. Can be a restart file. ESMF mesh file. Graph decomposition file. Optional: Runoff remapping file - needed to properly couple to runoff models - these are archived and described in another reposity: mpas-ocean-runoff Surface salinity forcing file - for OMIP style spin up runs. Transect and basin file - defines regions for ocean diatnostics.

Initial conditions if not starting from a restart

Ocean temperature and salinity are provided by the mesh and input file. Velocities are zero. The sea surface height is flat. Ocean files have 100 layers and a maximum depth of 6000m. The nominal ocean layer depths are in file /glade/campaign/univ/ucsu0085/build-mpas-ocean-data/refBottomDepth. The model initializes the seaice to be disks of uniform thickness surrounding each pole. This is placed atop the flat ocean.

Overview of steps to creating these files:

Start with a global base mesh file - this can be an atmospheric initial condition file or one of the mesh files found at https://mpas-dev.github.io/atmosphere/atmosphere_meshes.html

Create a file with the bathymetry defined on the global base (mpasa) mesh - this will use a 15s bathymetry file that can be found at /glade/campaign/univ/ucsu0085/build-mpas-ocean-data/SRTM15_plus_earth_relief_15s_original.nc. Compile the bathymetry_to_mpasa.f90 file with the netCDF library and run. This will require the global base mesh file as input, and you will need to specify a name for the global base mesh bathymetry file that will be created. The bathymetry output by this program will have negative values where the ocean domain is defined, and zero for land. The program has an example block the user may use as a template for user-specified modifications to the ocean domain - look for '!15km - fill/dredge'.

Create a file defining mesh variables on the ocean domain, and basic initial state variables to start the ocean from rest (except T and S, which will be appended later), and to enable seaice to start as specified from the model namelist - build_culled_file_from_bathy.f90 is built with mpi, hdf5, netcdf, and pnetcdf libraries. It requires as input the global base mesh file, the global base mesh bathymetry file, and will write the ocean domain mesh file which name the user will specify. There is also a template block that allows the user to modify the domain - look for '! 15km adjustment'.

Create an ESMF mesh file for the ocean domain - mpas2esmf.f90 takes as input the ocean mesh file and creates an ocean domain ESMF mesh file specified by the user.

Create an ocean domain graph file for metis decomposition - create_graph.f90 takes as input the ocean mesh file and generates the ocean domain graph file specified by the user. NOTE! the file generated by this program requires further user intervention. The second number in line 1 needs correction (it is a text file). Run gpmetis on it.

gpmetis <graph_file_name> <number_of_PEs>

You will get something like:

***  I detected an error in your input file  ***

In the first line of the file, you specified that the graph contained
84444 edges. However, I only found 84244 edges in the file.
Please specify the correct number of edges in the first line of the file.

Edit the output graph file. Replace the second number of line one with the second value of edges specified in the warning message.

At this point you can choose to either 1) add Levitus temperature and salinity to the ocean mesh file or 2) interpolate existing ocean and seaice restarts to this mesh, modifying the ocean mesh file and generating ocean and seaice restart files.

1. Add Levitus T and S to the ocean mesh file - build_ocean_ts_levitus.f90 is compiled with mpi, hdf5, netcdf, and pnetcdf libraries. User enters the ocean mesh file name to be modifed. Levitus data can be found at /glade/campaign/univ/ucsu0085/build-mpas-ocean-data/.

or

2. create interpolated restarts from previous EarthWorks runs

Create an interpolation weights file - build_ocean_ice_interpolation.f90 is compiled with netcdf and openmp. The user specifies an input ocean restart file from another mesh, the current mesh ocean mesh file, and a file for the interpolation weights.

Interpolate a seaice restart - build_ice_restart_file.f90 will interpolate prognostic variables from a typically coarser mesh restart file to the target mesh. The user will need to specify the target ocean mesh file, the intepolation weights file generated above, an input seaice restart at the coarser resolution, and a name for the output seaice restart file.

Interpolate an ocean restart - build_ocn_restart_file.f90 will interpolate prognostic variables from a typically coarser mesh restart file to the target mesh. The user will need to specify the target oc ean mesh file, the intepolation weights file generated above, an input ocean restart at the coarser resolution, and a name for the output ocean restart file. This step still leaves the velocities at rest.

Interpolate the ocean currents - build_interpolated_velocities.f90 creates interpolated velocities for a restart on the new ocean mesh using velocities from an available restart. Because interpolating velocities is problematic, source mesh vorticity and divergence are computed and it is these scalars that are interpolated. Then an elliptic equation is solved by Jacobi iteration to get the desired mesh velocities. Because at higher resolutions convergence can be slow this program allows the user to write solution restarts at specified intervals. For an initial solution the user provides the source ocean restart file, the interpolation weights file, maximum permissible solution residual, maximum permissible solution iterations, number of iterations between restarts, and the output ocean restart file of the last step.

Finally, the interpolated restarts need to be managed further to prepare them for EarthWorks input. cp ocean_mesh_file to an oceanIC mesh file which will get updated interpolated data for prognostic variables. ncks -v temperature,salinity,layerThickness,normalVelocity,normalBarotropicVelocity new_ocean_restart_file oceanIC_mesh_file Build and run modify_ic.f90 with netcdf which will update derived mesh file variables using restart variables. User will specify the new ocean restart file and the oceanIC mesh file. Build and run modify_xtime.f90 with netcdf which will reset the time variables in the seaice restart for starting a new run. User will specify the new seaice restart file.

Optional input files

Monthly mean surface salinity forcing for OMIP runs - salinity_forcing.f90 will generate forcing files to enable restoring of the ocean surface salinity to monthly climatology, as required by the OMIP protocol. The user specifies the path to the input salinity files, whether to use 1 degree or 1/4 degree resolution input, the ocean mesh file name, and the name for the output salinity forcing file.

Transect and basin files for some diagnostics - mpaso_basins.f90 and mpaso_transects.f90 together with a template transect file generate a basins and transect file used in computing certain diagnostics in the mpas ocean. The user specifies the ocean mesh file name and the output transect file name to be created.

Runoff remapping files - generating runoff remappings is a complicated process and gets a github repository of its own dedicated to it. Without runoff remapping files a small fraction of the runoff generated by the data or prognostic model does not reach the ocean. In a long run ocean levels will drop noticably.

Some notes on the domain for MPAS ocean and seaice

Ocean domains generated by interpolation of the bathymetry and generate a small number of grid cells that lead to two pathologies.

1. An ocean grid cell that has only one edge shared with a neighbor is essentially a single column model with regards to seaice. Produced seaice cannot flow out, and the ocean beneath has only diffusion to share heat with its neighbor. In such a case, ice can grow without limit. Such cells can be tested for and are eliminated in the build_culled_file_from_bathy.f90 program poleward of 45 degrees.
2. An ocean cell where all vertices touch land cannot advect ice because ice velocities are defined at the vertices and vertices touching land have zero velocity. In certain portions of the Arctic at coarse resolutions straits responsible for significant ice transport will need intervention so that this pathology does not interfere with the transport.

Questions? donald.dazlich@colostate.edu, dondazlich@gmail.com