NSF NCAR | NASA GSFC | DOE LANL ANL | NOAA NCEP GFDL | MIT | U MICH

www.esmf.ucar.edu

Emergence of the Earth System Modeling Framework

NSIPP Seasonal Forecast

NCAR/LANL CCSM

NCEP Forecast

GFDL FMS Suite

MITgcm

NASA GMAO Analysis

ClimateData Assimilation

Weather

Cecelia DeLucaSymposium on the 50th Anniversary of ONWP

NSF NCAR | NASA GSFC | DOE LANL ANL | NOAA NCEP GFDL | MIT | U MICH

www.esmf.ucar.edu

MotivationIn NWP, seasonal prediction and climate research...

increased emphasis on detailed representation of individual physical processes; requires many teams of specialists to contribute components to an overall modeling system

In computing technology... increase in hardware and software complexity in high-performance computing, shift toward the use of scalable computing architectures

In software …development of frameworks (integrated libraries), such as the GFDL Flexible Modeling System (FMS) and Goddard Earth Modeling System (GEMS) that encourage software reuse and interoperability

The ESMF is a community effort to tame the complexity of models and the computing environment. It leverages, unifies and extends existing software frameworks, breaking down institutional barriers and creating new opportunities for scientific contribution and collaboration.

NSF NCAR | NASA GSFC | DOE LANL ANL | NOAA NCEP GFDL | MIT | U MICH

www.esmf.ucar.edu

ESMF Project DescriptionGOALS: To increase software reuse, interoperability, ease of use and performance portability in

climate, weather, and data assimilation applications

PRODUCTS: • Core framework: Software for coupling geophysical components and utilities for building

components • Applications: Deployment of the ESMF in 15 of the nation’s leading climate and weather models,

assembly of 8 new science-motivated applications

METRICS:

RESOURCES and TIMELINE: $9.8M from NASA ESTO over 3 years, starting February 2002Ongoing support from the National Science Foundation

Reuse Interoperability Ease of Adoption Performance

15 applications use ESMF component coupling services and 3+ utilities

8 new applications comprised of never-before coupled components

2 codes adopt ESMF with < 2% lines of code changed, or within 120 FTE-hours

No more than 10% overhead in time to solution, no degradation in scaling

NSF NCAR | NASA GSFC | DOE LANL ANL | NOAA NCEP GFDL | MIT | U MICH

www.esmf.ucar.edu

ESMF Architecture

1. ESMF provides an environment for assembling geophysical components into applications, with support for ensembles and hierarchies.

2. ESMF provides a toolkit that components use to

i. increase interoperability

ii. improve performance portability

iii. abstract common services

ESMF InfrastructureData Classes: Bundle, Field, Grid, Array

Utility Classes: Clock, LogErr, DELayout, Machine

ESMF SuperstructureAppDriver

Component Classes: GridComp, CplComp, State

User Code

NSF NCAR | NASA GSFC | DOE LANL ANL | NOAA NCEP GFDL | MIT | U MICH

www.esmf.ucar.edu

Hierarchies and Ensembles

assim

sea iceocean

landatm

physics dycore

assim_atm

atmland

Seasonal Forecast

coupler

ESMF encourages applications to be assembled hierarchically and intuitively

Coupling interfaces are standard at each layer

Components can be used in different contexts

assim_atm

Ensemble Forecast

assim_atm assim_atm

ESMF supports ensembles with multiple instances of components running sequentially (and soon, concurrently)

NSF NCAR | NASA GSFC | DOE LANL ANL | NOAA NCEP GFDL | MIT | U MICH

www.esmf.ucar.edu

ESMF In More Detail…

• Fortran interfaces (mixed-language implementation)• Multi-platform support (SGI, Compaq, IBM, Linux - Intel, PGI, NAG, Absoft)• Clear, simple hierarchy of data objects (arrays, fields, bundles of fields,

components) for developing modeling codes• Toolkits for communications (redist, halo, etc.), time management, error

handling and logging, resource files, IO • Regridding library is on-line, parallelized version of SCRIP from LANL• Tools for multithreading, cache blocking, and load balancing integrated into

the architecture – development has been highly performance-aware!• Data objects and toolkits have consistent naming and behavior across the

framework

NSF NCAR | NASA GSFC | DOE LANL ANL | NOAA NCEP GFDL | MIT | U MICH

www.esmf.ucar.edu

The Benefits

• Standard interfaces to modeling components promote increased interoperability between centers, faster movement of modeling components from research to operations

• The ability to construct models hierarchically enables developers to add new modeling components more systematically and easily, facilitates development of complex coupled systems

• Multi-use objects mean that the same data structure can carry information about decomposition, communications, IO, coupling – this makes code smaller and simpler, and therefore less bug-prone and easier to maintain

• Shared utilities encourage efficient code development, higher quality tools, more robust codes

NSF NCAR | NASA GSFC | DOE LANL ANL | NOAA NCEP GFDL | MIT | U MICH

www.esmf.ucar.edu

Open Development• Open source• Currently ~800 unit tests, ~15 system tests are bundled with the ESMF

distribution, can be run in non-exhaustive or exhaustive modes• Results of nightly tests on many platforms are accessible on a Test and

Validation webpage• Test coverage, lines of code, requirements status are available on a Metrics

webpage• Exhaustive Reference Manual, including design and implementation notes, is

available on a Downloads and Documentation webpage

• Development is designed to allow users clear visibility into the workings and status of the system, to allow users to perform their own diagnostics, and to encourage community ownership

NSF NCAR | NASA GSFC | DOE LANL ANL | NOAA NCEP GFDL | MIT | U MICH

www.esmf.ucar.edu

ESMF Key Accomplishments• Public delivery of prototype ESMF v1.0 in May 2003• Monthly ESMF internal releases with steadily increasing functionality • Completion of first 3 coupling demonstrations using ESMF in March

2004– NCAR CAM with NCEP SSI– NCAR CAM with MITgcm ocean– GFDL B-grid atmosphere with MITgcm ocean– All codes above running as ESMF components and coupled using

the framework, codes available from Applications link on website– Other codes running as ESMF components: MOM4, GEOS-5– Less than 2% lines of source code change

• On track for delivery of ESMF v2.0 on 23 June 2004• 3rd Community Meeting to be held on 15 July 2004 at NCAR

NSF NCAR | NASA GSFC | DOE LANL ANL | NOAA NCEP GFDL | MIT | U MICH

www.esmf.ucar.edu

Interoperability Experiments Completed

NCAR Community Atmospheric Model (CAM) coupled to NCEP Spectral Statistical Interpolation (SSI) System, both set up as ESMF components

Experiment utilizes same observational stream used operationally at NCEP

3

NCAR Community Atmospheric Model (CAM) coupled to MITgcm ocean

Atmosphere, ocean, and coupler are set up as ESMF components

Uses ESMF regridding tools

1

GFDL B-grid atmosphere coupled to MITgcm ocean

Atmosphere, ocean, and coupler are set up as ESMF components

Uses ESMF regridding tools

2Temperature SSI import

Temperature SSI export

Temperature difference

NSF NCAR | NASA GSFC | DOE LANL ANL | NOAA NCEP GFDL | MIT | U MICH

www.esmf.ucar.edu

Next Steps– Integration with data archives and metadata standardization efforts,

anticipate collaboration with Earth System Grid (ESG) and European infrastructure project PRISM

– Integration with scientific model intercomparison projects (MIPs), anticipate collaboration with the Program for Climate Model Diagnosis and Intercomparison (PCMDI), other community efforts

– Integration with visualization and diagnostic tools for end-to-end modeling support, anticipate collaboration with the Earth Science Portal (ESP)

– ESMF “vision” for the future articulated in multi-agency white paper on the Publications and Talks webpage

NSF NCAR | NASA GSFC | DOE LANL ANL | NOAA NCEP GFDL | MIT | U MICH

www.esmf.ucar.edu

More Information

ESMF website: http://www.esmf.ucar.edu

3rd Community Meeting on 15 July at NCAR