The Realization of GRAPES Model on

China Meteorological Application Grid

Xuesheng YANGChinese Academy of Meteorological Sciences

China Meteorological Administrationyangxs@cma.gov.cn

2005/09/16

Characteristics of Numerical weather prediction

• Meteorology is one of the main components in the grid application– US:

• LEAD (Linked Environment for Atmospheric Discovery)• MEAD (Modeling Environment for Atmospheric Discovery)

– ECMWF: ECAccess

• NWP system includes preprocessing, analysis, numerical model, post processing, visualization:– execution processes are dynamically connected – enormously complex even if run individually – Involves in data management, remote collaborative research, hea

vy computation, massive data and intensive services• to ensure the proper operation of a NWP:

– massive computing resources needed

Existed issues

• tough issues meteorologists encountered:– organization of the NWP workflow and data– execution of model simulations– management of the resulting large volumes of data– subsequent data analysis– visualization

• model developers and computing resources stationed in different locations– telephone, mobile, e-mail, …

• inadequate for conducting a research project that involves in huge amount of simulations and extensive on-line discussions and consultations

– Computing resources can’t satisfy the research requirements• divert researchers’ focus on how to acquire computing resources

instead of addressing the scientific issues

Existed computing resources at CMA

• Available operational computing resources:– IBM Cluster 1600 – IBM SP

• Available computing resources for research:– IBM – Clusters: Sunway, Dawning, Oscar, Ocean

• CAMS• NMIC• Guangdong• Shanghai• Guangxi • etc.

• do not get full utilized– Computing resources– manpower

2. China Meteorological Application Grid(CMAG)

• grid portal

• GridWeather– a NWP workflow control interface

• GRAPES meso-scale model

• user management sub-system

• CVSExplorer – a code management system

• visualization

Tomcat5.28 Application Server

GridSphere Container

Base Portlets

LoginP

ortlet

ProfileM

anager

PortletA

pplication

LayoutM

anager

GridUser PortletsPortlets Modules

CredentialR

equest

CredentialS

ign

CredentialD

eleteG

ridUserM

ap

CMAGPortal Page

GridWeather portletsPortlets Modules

CredentialR

ecieve

JobSubm

ission

FileM

anager

FileT

ransfer

ResourceB

rowser

JobMonitoring

JobPosttreatm

ent

Forecast PortletsPortlets Modules

RainF

orecast

CityF

orecast

Elem

entForecast

PrecipitionO

bserved

CVSExplorerPortlets Modules

ProjectM

anager

CodeE

xplorer

CV

SM

anager

GridServices Myproxy GRAPES service

Grid Software(GT2,GT3,GT4,GOS)

Job Scheduler (OpenPBS,LSF,Conder)

Grid Resources(Cluster, expensive instrument)G

ang

lia

CVS ServerDatabase ServerMysql

IE browserarchitecture of CMAG Portal

Security and user management

• provides a single-log-on access interface for all grid services and grid resources

• authenticated users can access to computing, storage, data, software resources as well as all kinds of services

Security and user management

• SimpleCA of Globus toolkit is adopted to ensure the security of the grid resources– based on GSI and supports X.509 inter-certification mechanism

• CMAG Certification Authority signs each certificate

• In actual implementation, a CA center at CMA is set up and CA distribution packages are installed at all CMAG nodes or provincial meteorological bureaus

• User certifications managed by Myproxy library

CMAG Portal• Resource Access Portal

– to access geographically distributed computing resources

• NWP Products Browse Portal – to disseminate NWP products to the public and professional clients

• Remote Collaborative Portal (RCP)– to offer a universal platform for scientists to develop NWP modules, to share

source codes, to register modules within the grid and to perform NWP experiments.

CMAG Portal

• Globus 4.0 as the communication and tool-kit for collaborative research

• authentification and grid-job management is performed based on Gridsphere Portal Framework and Java COG tool-kits.

Infrastructure of CMAG Portal

Job Submission and resource monitoring

• based on the pre-wsrf-gram and wsrf-gram of Globus 4.0

• Each grid resource has its own job scheduling software:– SSC:

• PLATFORM’s LSF to perform job scheduling

– Others:• OpenPBS

• Ganglia to display dynamically the load of computing resource :– such as CPU load, availability of both memory and ha

rd-disk, as well as network throughput

The load report of grid resources displayed dynamically by Ganglia within CMAG

NWP executing control interface ---

GridWeather

• NWP system involves complicated process control and massive parameters setting

• traditional implementation method:– write a control routine – effective in local mode familiar with both every part of

NWP system and system environment.

• Gridweather:– based on Portlet including GRAM and GridFTP of Co

G as well as Globus toolkit

interface for research: GARPES model improvement

– to select the appropriate physical parameterization schemes

– to designate forecast domain, available numerical schemes, etc.

Flowchart of GRAPES model improvement on CMAG

job-submission strategy : firstly, select the computing resources already with special experimental data, secondly, select machines that can run the GRAPES model

NWP operational executing interface

Flowchart of operational GRAPES meso-scale model on CAMG

job-submitting strategy:

(1) selecting the computing resources that have had data already

(2) selecting idle machines, then local machine, or any available high-performance computing capabilities machines.

CVSExplorer

• NWP system upgrading• frequent modifications• diversity of program versions• intensive interactions between model devel

opers and inter-comparisons between numerical experiments

• it is necessary to create a suitable platform for joint NWP software development based on WinCVS

• Browse source codes

• GRAPES improvement

Configuration of CMAG

• portal server:– deployed on a DELL server at CAMS, includes:

• Web server Tomcat• Portal Framework Gridsphere• Web service software Java SOAP• grid application and development package CoG, GMETAD• web Front-end Ganglia

• management node:– implemented at OSCAR Cluster at CAMS

• Hierarchically structured MDS is installed– GIIS is available at each managing node and GRIS at

other nodes.

Computing resources

Location Grid Software

Security NWP software Job scheduler

Oscar CAMS GT4 CMA CA Center

GRAPES OpenPBS

Sunway48I CMAS GT4 CADP GRAPES OpenPBS

Sunway32I NMIC GT4 CADP GRAPES OpenPBS

Sunway32P NMIC GT4 CADP GRAPES OpenPBS

Ocean STI GT4 CADP GRAPES OpenPBS

Dawning GRMC GT4 CADP GRAPES OpenPBS

Dawn GXMB GT4 CADP GRAPES OpenPBS

PC Wuhan GT4 CADP GRAPES No

Dawning 4000A

SSC GT3 CADP GRAPES LSF

CADP: CMA CA Distributed Packages

STI: Shanghai Typhoon Research Inst.

Computing platform on CMAG

Real-time weather forecasts

• GRAPES meso-scale model 30KM/15KM– Mainly for rain forecast– runs every day on CMAG

• Real-time forecast products:– http://grid.cma.gov.cn:8080/

• Also computing resources and NWP technique comparatively less developed regions can utilize the grid resources or even run the NWP models to satisfy their forecast service requirement:– Yantai– Qinghai

GRAPES-Meso Scale model

• new NWP system developed by Research Center for Numerical Prediction at CAMS

• Features:– fully compressible primitive equations– switchable between hydrostatic and non-hydrostatic mode– semi-implicit and semi-Lagrangian advection schemes – latitude and longitude grid– horizontal Arakawa C grid – terrain-following coordinates – Charney-Phillips staggering vertically– physical parameterization package

ATOVS Data

Preprocess

Quality control

ConventionalData

preprocess

Quality control

3DVariation Anal

ysis

3DVariation Anal

ysis

Global model lateral B.C.

GRAPES Model real-timeforecasts

GRAPES Model real-timeforecasts

Post-processing

Post-processing

Forecast products

visualizationvisualization

prepocessing

Initial 48hrs forecast

6hrs cycling

Analysis Forecast Postprocessing

Flowchart of meso-scale GRAPES model on CMAG

500 hPa Geopotential height forecast over China (2005090900UTC)

forecast

Hourly rain forecast for Beijing, 2005080800UTC

Typhoon MTyphoon Matsaatsa

Conclusion

• Distributed computing resources at Beijing, Shanghai, Guangdong and Guangxi aggregated

• real-time GRAPES executions on CMAG feasible

• a virtual collaborative R/D center established:– provides researchers at different locations with a

common R/D environment aimed at further improvement of the GRAPES

Thanks for listening!