Upload
nguyendan
View
229
Download
0
Embed Size (px)
Citation preview
Climate, Ocean and Sea-Ice Modeling Projecthttp://public.lanl.gov/ringler/ringler.html
Todd RinglerTheoretical Division
Los Alamos National Laboratory
Status of MPAS-O
CESM Workshop: Ocean Model Working Group: June 21, 2011
What is MPAS?
1. MPAS is an unstructured-grid approach to climate system modeling.
2. MPAS supports both quasi-uniform and variable resolution meshing of the sphere using quadrilaterals, triangles or Voronoi tessellations.
3. MPAS is a software framework for the rapid prototyping of single-components of climate system models (atmosphere, ocean, land ice, etc.).
4. MPAS offers the potential to explore regional-scale climate change within the context of global climate system modeling. Multiple high-resolution regions are permitted.
5. MPAS is currently structured as a partnership between NCAR MMM and LANL COSIM, where we intend to distribute our models through open-source, 3rd-party facilities (e.g. Sourceforge).
CESM Workshop: Ocean Model Working Group: June 21, 2011
MPAS-Ocean Team:Todd Ringler, Mark Petersen, Mat Maltrud, Chris Newman, Bob Higdon, Doug Jacobsen,Rob Lowrie, Jonathan Graham, Qingshan Chen
MPAS Component Development Teams
MPAS-Atmos Hydrostatic Team: Bill Skamarock, Todd Ringler, Michael Duda, Sara Rauscher, Li Dong, Art Mirin, Chris Jeffery
MPAS-Land Ice Team:Bill Lipscomb, Steve Price, John Burkhart, Xylar Asay-Davis, Lili Ju, Max Gunzburger, Mauro Perego
MPAS-Atmos Non-Hydrostatic Team: Bill Skamarock, Michael Duda, Laura Fowler and others in MMM
All model components share a majority of their source code, are built from the same Makefile and will likely be used within the CESM.
CESM Workshop: Ocean Model Working Group: June 21, 2011
A summary of the MPAS-O capabilities
Choice of isopycnal or z-level vertical coordinate at runtime.Configurable with idealized or real-world boundary/forcing at runtime.Can run on quad grids (i.e. POP meshes) or Voronoi tessellations.Can run in quasi-uniform mode or with an arbitrary number of high-resolution regions.
CESM Workshop: Ocean Model Working Group: June 21, 2011
A summary of the MPAS-O capabilities
Choice of isopycnal or z-level vertical coordinate at runtime.Configurable with idealized or real-world boundary/forcing at runtime.Can run on quad grids (i.e. POP meshes) or Voronoi tessellations.Can run in quasi-uniform mode or with an arbitrary number of high-resolution regions.
High-order reconstruction for horizontal and vertical transport.Laplacian and/or bi-harmonic mixing of velocity and tracers.Nonlinear equation of state (Jackett and McDougall).Pacanowski and Philander Richardson Number based vertical mixing.Implicit solve for vertical mixing.Two-time level, split-explicit time-stepping scheme (implemented and being tested).Exact tracer conservation.
(Items discussed on following slides.)
CESM Workshop: Ocean Model Working Group: June 21, 2011
High-Order Vertical Tracer Advection
vertical advection
�
∂∂z
hϕw( ) = ϕktopwk
top −ϕk+1topwk+1
top
�
ϕktop ,wk
top 7/12
weights
7/12
-1/12
-1/12
4th order stencil3rd order stencil
7/12-3/12sign(wTop)
weights
7/12+3/12sign(wTop)
-1/12
-1/12
�
ϕk
1/2
weights
1/2
linear
�
ϕk+1
�
ϕk−2
cubic spline
Four methods are available to interpolate tracer values to cell interface:
Vertical tracer advection, requires tracer values at vertical cell edge.
CESM Workshop: Ocean Model Working Group: June 21, 2011
Richardson-Number based Vertical Mixing with an Implicit Solver
solve explicitly solve implicitly
new provisional value from explicit solve
Pacanowski-Philander vertical mixingBased on Richardson Number, so viscosity and tracer diffusion increase with vertical shear and weaker stratification.
Implicit vertical mixingAllows mixing to occur stably at fast timescales without constraining the model time step.
Operator splitting used on explicit and implicit tendency terms in the momentum and tracer equations.
The modular computation of vertical mixing coefficients allows for an easy migration to KPP or other vertical turbulence closure scheme.
CESM Workshop: Ocean Model Working Group: June 21, 2011
Design and Implementation of a two-time level, split-explicit scheme
removes barotropiccomponent from baroclinicsystem.
CESM Workshop: Ocean Model Working Group: June 21, 2011
Design and Implementation of a two-time level, split-explicit scheme
removes barotropiccomponent from baroclinicsystem.
∆t = tn+1 − tntn
�
′ u n +1 G,update baroclinic velocity
RHS based on mid-time state
slow mode
CESM Workshop: Ocean Model Working Group: June 21, 2011
Design and Implementation of a two-time level, split-explicit scheme
removes barotropiccomponent from baroclinicsystem.
∆t = tn+1 − tntn
�
′ u n +1 G,update baroclinic velocity
RHS based on mid-time state
slow mode
�
u n +1/ 2,ηn +1/ 2,update barotropic state
fast mode
CESM Workshop: Ocean Model Working Group: June 21, 2011
Design and Implementation of a two-time level, split-explicit scheme
removes barotropiccomponent from baroclinicsystem.
∆t = tn+1 − tntn
�
′ u n +1 G,update baroclinic velocity
RHS based on mid-time state
slow mode
�
ϕn+1,ρn+1
�
′ u n +1/ 2 + u n +1/ 2∆t = tn+1 − tn
update tracers
slow mode
�
u n +1/ 2,ηn +1/ 2,update barotropic state
fast mode
CESM Workshop: Ocean Model Working Group: June 21, 2011
Estimate mid-time state and repeator
Move to next time step
Design and Implementation of a two-time level, split-explicit scheme
removes barotropiccomponent from baroclinicsystem.
∆t = tn+1 − tntn
�
′ u n +1 G,update baroclinic velocity
RHS based on mid-time state
slow mode
�
ϕn+1,ρn+1
�
′ u n +1/ 2 + u n +1/ 2∆t = tn+1 − tn
update tracers
slow mode
�
u n +1/ 2,ηn +1/ 2,update barotropic state
fast mode
CESM Workshop: Ocean Model Working Group: June 21, 2011
Estimate mid-time state and repeator
Move to next time step
Design and Implementation of a two-time level, split-explicit scheme
A few nice properties of this approach:1. Only need time-level n information.2. Can run with or without the splitting (runtime choice).3. Maintain exact tracer conservation.4. Splitting approach is same for z-level or layer model.
removes barotropiccomponent from baroclinicsystem.
∆t = tn+1 − tntn
�
′ u n +1 G,update baroclinic velocity
RHS based on mid-time state
slow mode
�
ϕn+1,ρn+1
�
′ u n +1/ 2 + u n +1/ 2∆t = tn+1 − tn
update tracers
slow mode
�
u n +1/ 2,ηn +1/ 2,update barotropic state
fast mode
CESM Workshop: Ocean Model Working Group: June 21, 2011
At that last OMWG, focus was on variable resolution capability of MPAS-O.R60kmNA: local grid resolutionR60km: local grid resolution
See example of this same variable resolution approach in CAM at Sara Rauscher’s poster.
CESM Workshop: Ocean Model Working Group: June 21, 2011
The focus over the last development period has been on real-world,quasi-uniform simulations at 120 km, 60 km and 30 km resolutions.\
nominal 120 km, 60 km and 30 km horizontal resolutionsz-level model, 40 levels (same as POP gx1v3)topography taken from ETOPO, no smoothing, depth >= 2 levelsinitial conditions for T and S from annual mean WOCE global climatologytime-invariant wind stress forcing from annual mean NCEP 1958-2000
CESM Workshop: Ocean Model Working Group: June 21, 2011
The focus over the last development period has been on real-world,quasi-uniform simulations at 120 km, 60 km and 30 km resolutions.\
nominal 120 km, 60 km and 30 km horizontal resolutionsz-level model, 40 levels (same as POP gx1v3)topography taken from ETOPO, no smoothing, depth >= 2 levelsinitial conditions for T and S from annual mean WOCE global climatologytime-invariant wind stress forcing from annual mean NCEP 1958-2000
2nd-order, two-time level, predictor-corrector, unsplit time stepping(dt=200 s, 100 s and 50 s for 120 km, 60 km and 30 km resolutions)
3rd-order reconstruction for horizontal transport3rd-order reconstruction for vertical transportJackett and McDougall EOSvertical diffusivity/viscosity based on Pacanowski and Philander with implicit solve.
CESM Workshop: Ocean Model Working Group: June 21, 2011
The focus over the last development period has been on real-world,quasi-uniform simulations at 120 km, 60 km and 30 km resolutions.\
nominal 120 km, 60 km and 30 km horizontal resolutionsz-level model, 40 levels (same as POP gx1v3)topography taken from ETOPO, no smoothing, depth >= 2 levelsinitial conditions for T and S from annual mean WOCE global climatologytime-invariant wind stress forcing from annual mean NCEP 1958-2000
2nd-order, two-time level, predictor-corrector, unsplit time stepping(dt=200 s, 100 s and 50 s for 120 km, 60 km and 30 km resolutions)
3rd-order reconstruction for horizontal transport3rd-order reconstruction for vertical transportJackett and McDougall EOSvertical diffusivity/viscosity based on Pacanowski and Philander with implicit solve.
The simulations use bi-harmonic dissipation on u and tracers(visc = 1.0e14 m^4/s, 2.5e13 m^4/s and 5.0e12 m^4/s)
restoring to WOCE annual mean T and S with 45 day time scale
CESM Workshop: Ocean Model Working Group: June 21, 2011
Global 30 km Simulation: Snap-Shot of Surface Kinetic Energy
start of year 3
CESM Workshop: Ocean Model Working Group: June 21, 2011
Global 30 km Simulation: Snap-Shot of Surface Kinetic Energy
There are many promising aspects of this simulation .... all of which might disappear as we integrate the model longer.
start of year 3
CESM Workshop: Ocean Model Working Group: June 21, 2011
Time mean SSH, Year 3, contour interval = 0.15 m, peak-to-peak amplitude= 3.0 m.
Global 60 km Simulation: Sea-Surface Height
CESM Workshop: Ocean Model Working Group: June 21, 2011
Comparison of the 120 km and 60 km simulations.
120
km r
esol
utio
n60
km
res
olut
ion
CESM Workshop: Ocean Model Working Group: June 21, 2011
MPAS-POP comparisonEquatorial Undercurrent at 140W
dept
h (m
)
zonal velocity (m/s)
black: POP x1 (30 km at equator)red: MPAS 60 km (60 km at equator)
POPMPAS-O
CESM Workshop: Ocean Model Working Group: June 21, 2011
MPAS-POP comparisonEquatorial Undercurrent at 140W
dept
h (m
)
zonal velocity (m/s)
black: POP x1 (30 km at equator)red: MPAS 60 km (60 km at equator)
Overall, this figure does a great job of summarizing where we are at with MPAS-O. The raw features are present, sometimes with tantalizing hints of promise,
but often without the fidelity a highly polished model.
POPMPAS-O
CESM Workshop: Ocean Model Working Group: June 21, 2011
MPAS-POP comparisonComputational Efficiency
For a given simulation, we want to minimize wall clock time. We can reduce wall clock time by increasing the scaling efficiency and by reducing the computational time per degree of freedom. We have shown previously that model scales well out to at least 1000 processors.
CESM Workshop: Ocean Model Working Group: June 21, 2011
MPAS-POP comparisonComputational Efficiency
ModelWall Clock per
baroclinic time step
MPAS @ 120 km 1.7 s
POP @ x1 0.3 s
64 processors, MPAS @ 120 km, POP @ x1
For a given simulation, we want to minimize wall clock time. We can reduce wall clock time by increasing the scaling efficiency and by reducing the computational time per degree of freedom. We have shown previously that model scales well out to at least 1000 processors.
CESM Workshop: Ocean Model Working Group: June 21, 2011
MPAS-POP comparisonComputational Efficiency
ModelWall Clock per
baroclinic time step
MPAS @ 120 km 1.7 s
POP @ x1 0.3 s
64 processors, MPAS @ 120 km, POP @ x1
MPAS is currently 6x slower than POP (caveats abound here). This is due to the following:1. Have not invested any time in computational efficiency.2. Use of derived data types and pointers (expect 2x improvement here).3. Use of unstructured grids.
Our goal over the next six months is to bring MPAS on par with POP.
For a given simulation, we want to minimize wall clock time. We can reduce wall clock time by increasing the scaling efficiency and by reducing the computational time per degree of freedom. We have shown previously that model scales well out to at least 1000 processors.
CESM Workshop: Ocean Model Working Group: June 21, 2011
Plans for the next six-month cycle:
1. Finish testing and evaluation of two-time level, barotropic-baroclinic splitting scheme.
2. We plan to bring two manuscripts to the December meeting. Manuscript #1: Exploring a Multi-Resolution Approach for Global Ocean ModelingManuscript #2: A Comparison of MPAS-O to POP
3. MPAS-wide optimization.
4. Facilitate/Coordinate/Participate in the design of general CESM/COM software interfaces to allow the use of unstructured-mesh dynamical core(s). (Note: This is already done on the CESM/CAM side.)
5. Finish implementation of Prather-like, method-of-moments, horizontal transport scheme. (This scheme will be available across the MPAS framework.)
6. Begin design of Gent-McWilliams mixing schemes.
CESM Workshop: Ocean Model Working Group: June 21, 2011
Thanks!