NASA Langley Research Center ASAP Science Meeting, Boulder, CO 13-14 Apr 2005 Louis Nguyen, Patrick Minnis, John Murray, and Rabindra Palikonda NASA Langley

NASA Langley Research Center ASAP Science Meeting, Boulder, CO 13-14 Apr 2005

Louis Nguyen, Patrick Minnis, John Murray, and Rabindra Palikonda

NASA Langley Research CenterHampton, Virginia, USA

Migration of LaRC Icing Products to Migration of LaRC Icing Products to Columbia SupercomputerColumbia Supercomputer

NASA Langley Research Center SPIE 49th, Denver, CO 2-6 Aug 2004

ObjectivesObjectives

• Improve processing system to produce reliable icing products

• Increase temporal and spatial resolution of these products

• Improve latency time

• Port and migrate code to Project Columbia Supercomputer

• Transition research code to operational code

Transition and Integrate the Langley’s Satellite Products to Operations in the operational National Weather Service Aviation Weather Center CIP product


OutlineOutline

• Overview of LaRC Icing processing environment

• Project Columbia Supercomputer

• Migration plan

• Utilize parallel processing

• Benefits from improved computer capability

• Summary and future plans


Overview of LaRC Processing EnvironmentOverview of LaRC Processing Environment

Real-time Data Acquisition SystemReal-time Data Acquisition System•• GOES-East & West data from via McIDAS ADDE server (SSEC)GOES-East & West data from via McIDAS ADDE server (SSEC)

•• RUC data obtained from SSEC and NOAA Forecast Systems LabRUC data obtained from SSEC and NOAA Forecast Systems Lab

•• Snow maps from NOAA via FTP siteSnow maps from NOAA via FTP site

•• Transition to Operations, utilize the NOAA NESDIS operational 24/7 ADDE servers to ingest Transition to Operations, utilize the NOAA NESDIS operational 24/7 ADDE servers to ingest satellite and model datasatellite and model data

Processing EnvironmentProcessing Environment•• Uses the Man computer Interactive Data Access System (McIDAS, Uses the Man computer Interactive Data Access System (McIDAS,

developed by SSEC Univ. Wisc)developed by SSEC Univ. Wisc)

•• McIDAS is a programmable suite of software and interactive data McIDAS is a programmable suite of software and interactive data visualization tool used for displaying, analyzing, and acquiring datasetsvisualization tool used for displaying, analyzing, and acquiring datasets

•• Runs on SGI Unix platformRuns on SGI Unix platform


Overview of LaRC Icing Processing (con’t)Overview of LaRC Icing Processing (con’t)

Source CodeSource Code•• Code Modularization (R. Palikonda)Code Modularization (R. Palikonda)

-- satellite modules for GOES (8-12), MSG, AVHRR, satellite modules for GOES (8-12), MSG, AVHRR, MODISMODIS

- sounding module for RUC, ECMWF, GFS/AVN- sounding module for RUC, ECMWF, GFS/AVN

- helps isolates sections of the algorithm to improve cloud mask, - helps isolates sections of the algorithm to improve cloud mask, corr-k, cloud parameterization, etc.corr-k, cloud parameterization, etc.

- most modules are written in C- most modules are written in C

•• Main driver program is in McIDAS environmentMain driver program is in McIDAS environment

•• Cloud algorithms are written in FortranCloud algorithms are written in Fortran

•• Data inputs and outputs utilizes McIDAS APIData inputs and outputs utilizes McIDAS API

•• Approximately 75,000 lines of codeApproximately 75,000 lines of code


Overview of LaRC Icing Processing (con’t)Overview of LaRC Icing Processing (con’t)

Current Computer Resources Used for ASAPCurrent Computer Resources Used for ASAP•• SGI Orgin 3800 with 40 CPUsSGI Orgin 3800 with 40 CPUs

•• Only 8 CPU is available for Icing (other CPUs are used by Only 8 CPU is available for Icing (other CPUs are used by CERES & CALIPSO project)CERES & CALIPSO project)

Limitation of Current Computer ResourcesLimitation of Current Computer Resources•• Run code every half hour during the day and hourly at nightRun code every half hour during the day and hourly at night

•• Process GOES-West (10) at 8km res: ~30 minsProcess GOES-West (10) at 8km res: ~30 mins

•• Process Process GOES-East (12) at 8km res: ~35 minsGOES-East (12) at 8km res: ~35 mins

Need Additional ResourcesNeed Additional Resources•• Run at full 4km resolution every 15 minsRun at full 4km resolution every 15 mins

•• Would take ~70 min to process GOES-West and ~80 min for Would take ~70 min to process GOES-West and ~80 min for GOES-East at full resolutionGOES-East at full resolution

•• Project Columbia Supercomputer offers needed resourceProject Columbia Supercomputer offers needed resource


• NASA’s newest supercomputer is one of the world’s fastestNASA’s newest supercomputer is one of the world’s fastest

• Supports the agency’s 4 mission directorates (Aeronautics, Exploration, Supports the agency’s 4 mission directorates (Aeronautics, Exploration, Science, & Space), NASA Engineering and Safety Center (NESC), and Science, & Space), NASA Engineering and Safety Center (NESC), and Vision for Space ExplorationVision for Space Exploration

• Support general science community: GSFC runs a General Circulation Support general science community: GSFC runs a General Circulation Model to generate real-time numerical weather prediction (NWP) to Model to generate real-time numerical weather prediction (NWP) to improve hurricane tracking and intensity forecast for Hurricane Centerimprove hurricane tracking and intensity forecast for Hurricane Center

• Funds for Columbia pre-paid for Funds for Columbia pre-paid for

thru FY05-07 thru FY05-07 from the 4 mission from the 4 mission

directorates and NESCdirectorates and NESC

• Requested 35,000 cpu hrs FY05-06 Requested 35,000 cpu hrs FY05-06

from Aviation Safety Program from Aviation Safety Program

allotmentallotment

Project Columbia SupercomputerProject Columbia Supercomputer


• 20 SGI Altix 3700 superclusters, 20 SGI Altix 3700 superclusters, each with 512 processorseach with 512 processors

• 10,240 Intel Itanium-2 processors 10,240 Intel Itanium-2 processors running at 1.5 GHzrunning at 1.5 GHz

• Capable of 51.9 teraflops, ranked Capable of 51.9 teraflops, ranked second behind DOE Blue Genesecond behind DOE Blue Gene

• 20 Terabyte of Memory 20 Terabyte of Memory

• Linix based operation systemLinix based operation system

• 40 TB of fiber channel RAID 40 TB of fiber channel RAID storagestorage

• Capable of storing 10 petabytes of Capable of storing 10 petabytes of archive storagearchive storage

Project Columbia SupercomputerProject Columbia Supercomputer

Technical Hardware DescriptionTechnical Hardware Description


Migration to Project ColumbiaMigration to Project Columbia

• NASA Advance Supercomputing (NAS) accounts requestedNASA Advance Supercomputing (NAS) accounts requested

• McIDAS software installationMcIDAS software installation

• Over 1000 programs and subroutinesOver 1000 programs and subroutines

• Testing the software componentsTesting the software components

• Port LaRC Icing code to Columbia platformPort LaRC Icing code to Columbia platform

• Data ingest system (satellite, model, input maps)Data ingest system (satellite, model, input maps)

• Processing script and aux program packageProcessing script and aux program package

• LaRC cloud algorithmsLaRC cloud algorithms

• Test and check for consistency of Icing products produced from Test and check for consistency of Icing products produced from LaRC and Columbia LaRC and Columbia

Possible security/firewall issues related to ingest and dissemination Possible security/firewall issues related to ingest and dissemination

of data and products via ADDE and FTP portsof data and products via ADDE and FTP ports


Parallel ProcessingParallel Processing

GOES-10 Phase GOES-10 Phase GOES-12 PhaseGOES-12 Phase

GOES-12 & GOES-10 areGOES-12 & GOES-10 areprocessed using single CPUprocessed using single CPU- at 8km takes 30 minsat 8km takes 30 minsat 4km takes 80 minsat 4km takes 80 mins

Time includes 5-8 mins forTime includes 5-8 mins forInput and outputInput and output

Two parallel processing approach:Two parallel processing approach:- run sub-domains using multiple CPUs (use same binary)run sub-domains using multiple CPUs (use same binary)- multi-threaded process for looping through each grid boxmulti-threaded process for looping through each grid box

Parallel processing will use 16 CPUs (8 for ea GOES)Parallel processing will use 16 CPUs (8 for ea GOES)- Columbia CPU are ~2.5x fasterColumbia CPU are ~2.5x faster


Parallel Processing (con’t)Parallel Processing (con’t)

Full Resolution 80 min process should be reduced to 9-12 minsFull Resolution 80 min process should be reduced to 9-12 mins


Summary and Future PlansSummary and Future Plans

• LaRC Satellite products available near real-time in several formats (McIDAS ADDE server, RUC Gridded netCDF, and LaRC Packed Binary)

• LaRC Icing code will be adapted to run in parallel processing mode

• Parallel processing will allow GOES Icing products to run at full resolution every 15 min

• Porting to Project Columbia Supercomputer to be completed by end of FY05 (ASAP project milestone)

• Work with the NASA Applied Sciences Program, the FAA and NOAA to integrate and transition the LaRC Icing products to the operational NWS Aviation Weather Center CIP product by end of FY06

This research was support by the NASA Aviation Safety Program. This research was support by the NASA Aviation Safety Program.

Documents

NASA Langley Research Center ASAP Science Meeting, Boulder, CO 13-14 Apr 2005 Louis Nguyen, Patrick Minnis, John Murray, and Rabindra Palikonda NASA Langley