SERVICE ORIENTED ATMOSPHERIC RADIANCES (SOAR): A SYSTEM OF SERVICES DELIVERING MULTI-SENSOR GRIDDED

DATA RECORDS AND APPLIED SCIENTIFIC ANALYSES

Overview

• Rationale for SOAR as SOS• Architecture• SOAR system walk through • Examples of climate analysis services• Summary

What is SOAR?Service Oriented Atmospheric Radiances

• A web-based service oriented framework to improve access and use of satellite acquired atmospheric IR radiance data

• Maps multi-sensor radiance data from orbital coordinates stored at distributed archives onto lat-lon grids of arbitrary resolution on-demand

• Serves up fundamental data records in image and/or structured data formats

• Provides observation-based analysis services for the study and intercomparison of long historical IR data records

• Currently serves up AIRS and MODIS data and selected periods of HIRS3, VTPR and AVHRR3

What makes SOAR a system of services?• SOAR includes a general ’Downloader’ for accessing multi-sensor

instrument data from distributed archives from multiple satellites

• SOAR contains a generalized ‘Gridder’ that incorporates the sensor’s spatial response function to accurately calibrate the emitted radiance from each observed footprints portion of a grid cell

• SOAR is based on an SOA architecture that employs a cloud computing paradigm for efficient data intensive processing

• SOAR provides a set of analysis tools application specific to the geometry of the respective instrument data sets

• Maps level 1B granules into gridded level 3 radiances

• Increases accessibility and usability by scientists• Most op’n’l forecast centers assimilate gridded radiances• Provides analysis services based directly on observations • Greatly reduces data volume (lossy compression)• A common framework for gridding multi instruments• No level 3 gridded radiance products available from AIRS

and MODIS or HIRS/AVHRR instrument teams

Why we need SOAR?

SOAR Web Service Architecture

Web Service

Lookup

Publish Services

WSDL

URLS

Web Service ProviderWeb Service Client

SOAP

UDDIDirectory

HTML

HTTP

SOARClient Server

User (Browser)

SOARWeb

ProcessServices

File Share (Binary

Data/Images)

Bluegrit Supercomputer

Cluster

GDAACMODAPSNCDCNCAR

SOAR Downloader

• A C++ based application for each sensor

• Downloading formats tailored to each sensors data files as a separate service on UDDI

• Sensor access invoked by soap message specified through client server pull down window

• Accesses on-line data sets over internet using protocols such as ftp, http, etc.

AIRS Visible Image (1 day)

Download of HIRS 3 Orbit

SOARs Generic GridderSOARs Generic Gridder• Philosophy: Philosophy: -- Common gridding algorithms for many instruments.-- Common gridding algorithms for many instruments.

– Spatial calibration with recursive ray casting algorithmSpatial calibration with recursive ray casting algorithm

– Spectral response functions for calibrating multi-sensorsSpectral response functions for calibrating multi-sensors

– Orbital drift calibrations with neural network algorithm.Orbital drift calibrations with neural network algorithm.

• Framework developed for gridding radiances from many Framework developed for gridding radiances from many

scanning instruments.scanning instruments.– Currently implemented for AIRS, MODIS, AIRS VisibleCurrently implemented for AIRS, MODIS, AIRS Visible

-- Extending to HIRS2/3, HIRS4 and VTPR-- Extending to HIRS2/3, HIRS4 and VTPR

– Incorporating artificial neural network algorithm into Gridder.Incorporating artificial neural network algorithm into Gridder.

Cloud ComputingCloud Computing

SOAR System of Services

SO

AR

We

b S

ystem

System Services• Query & Request Instrument Data• Transactions recorded in database• Analysis Routines• NCAR Graphics Visualization• Real-time Data Gridding• Subset by resolution (1°x 0.5° native)• Subset by geographic region• Remote Data Acquisition

Downloader Engine

Gridder Engine

Data Visualization Engine

Analysis Routines

Data Query Web Application

Transactions

Pre

-Grid

de

d d

ata

AIRSMODIS

HIRS

Staged Gridded Radiance Data• MODIS gridded data (Nov’04 – Oct’08)• AIRS gridded data (Sep’02 – Oct’08)• Placing new datasets into production: HIRS, VTPR, SIRS, AVHRR, SBUV, OMI

AVHRR VTPR

One Month 1/1/2005-1/31/2005 AIRS VIS Ch1

HIRS3 Ch 8 (12um) AIRS 528 ch 528 (12um)Jan. 1-14, 2005

SOAR Home Page

SOAR Request Page

SOAR Results Page

Some Analysis Routines•Monthly, Seasonal, Yearly Averages and Anomalies

•Tracking monthly shifts in Inter Tropical Convergence Zone

• Madden Julian Oscillations (EEOFs)

• EL Nino Southern Oscillation (ENSO)

• Outgoing Long Wave Radiation (OLR) and Latitudinal OLR

•AIRS-MODIS Intercomparisons by Grid cell or Region

•Quasi-Bienneial Oscillation

AIRS Monthly average 0.5ox1o at 12.18 µm

AIRS Monthly anomaly 0.5ox1o at 12.18 µm

• Year to year variances• Cold radiances Feb 05 (strong El Nino

year) convective cloud, Warm radiances Feb 07 cloud clear surface in Western Pacific

• Similar in Indian Ocean and West Pacific area

• Feb 05 warmer than other 2 year in East US (hurricanes)

• Variances in Intertropical Convergence Zone

Feb 2006 anomaly

Feb 2005 anomaly Feb 2007 anomaly

MJO- results

C)

Variances color code

BT color code

Variances color codeDec1506

Dec1706

Dec1906

Dec2106

Dec2306

Dec2506

Dec2706

Dec2906

Jan0107

Jan0307

Jan0507

Jan0707

Jan0907

Jan1107

Jan1307

Jan1507

Jan1707

lag1

lag2

lag3

lag4

lag5

The first EOF explaining about explaining about 14.3% variance14.3% variance

B)

2 day running mean of MODIS channel 2 day running mean of MODIS channel 3232 (Surface/Cloud Temperature) (Surface/Cloud Temperature) at 0.5at 0.5oox1 11.7 x1 11.7 µmµm -12.2 -12.2 µmµm 5S-5N 0-180E Brightness Temperature 5S-5N 0-180E Brightness Temperature descending orbit from Dec 15 2006 to descending orbit from Dec 15 2006 to Jan 17 2007.Jan 17 2007.

A)The extended EOF The extended EOF captures the captures the dynamics using a temporal lag dynamics using a temporal lag of 2 day running mean.of 2 day running mean.

AIRS/MODIS total OLR 0.5ox1o

• AIRS/MODIS total OLR• isentropic assumption • Compare with CERES/ERBE OLR Feb. 2005 vs April 1985

Summary• SOAR SOS provides transparent access, gridding and visualization on demand

of the following multi- sensor IR radiance data sets: AIRS, MODIS, AIRS VIS, HIRS3, VTPR;

• SOAR incorporates a variety of analysis tools that provide multi- year monthly and seasonal anomalies, MJO, ENSO, OLR and statistics within a grid

cell as well as regional and global;

• SOAR system provides multiple gridding options for arbitrary spatial/spectral resolutions for multi sensor intercomparisons;

• SOAR Scientific Findings: AIRS and MODIS IR spectral radiance measurements have not degraded in

over 6 years AIRS and MODIS gridded IR spectral radiances have potential to provide

long term (>10 year) Fundamental Decadal Data Record MJO can be tracked directly from raw observations with fewer uncertainties

Back Up Slides

SOAR Service Interaction Diagram

User Client Web Service Bluegrit

New Request

Submit New Request FormGet radiance data() :sessionKey

login()Submit login

Session Key

Get login page

Welcome Page/Recent Results

New Request Form

Request Status :requestID

Login Page (HTML)

Result List

Get user results() :sessionKey

Science Image File

Request Status Page :requestID

Get Results(requested) :sessionKey

Get raw dataRaw Data File Handle

Subset/Average Data

Condensed Data File Handle

Render Data as Image

Get Request Results

Request Results

File URL(s)

Set Status

Results Display

Image/Animation/Data URL(s)

What is SOAR?• A web based system with an interface for accessing and

invoking gridding and analysis services on level 1B Infra-red radiance data

• Employs SOA technologies to discover and select services for use of multi -sensor infra-red radiances

• Serves up pre-gridded (lat-lon) AIRS and MODIS IR spectral radiances on-demand and requested image and/or structured data formats

• Provides a platform for users to exploit IR data for climate analysis with traditional methodologies

Gridding MODIS/AIRS in SOAR system(SOAR- Service Oriented Atmospheric Radiances)

NASA GSFC archived Servers

Processor Server(Bluegrit)

10Gbps

Schedule jobs

Gridding routines

Bluegrit, IBM Blade CenterJS20 Blades & JS21 blades

http://bluegrit.cs.umbc.edu/soar/http://bluegrit.cs.umbc.edu/soar/

Web Server(Bluegrit)

Requests display

• Gridding routines• Subset, images• Simple statistic tools• Convolution routines• Climate applications

Sensor’s datasets

• requests• visualize • analysis• download

SOAR Technologies Used• Apache Tomcat – application server• Apache – web server• C/C++ – data processing utilities• Java – application programming• Apache AXIS – SOAP protocol library• PHP – web client programming• PostGRESQL – application database• Apache ANT – build and deployment scripting• Subversion – configuration management