Research To Operations (R2O, R2X and O2R) Sid-Ahmed Boukabara The JCSDA Summer Colloquium, August 7 th, 2015, Fort Collins, CO NOAA/NESDIS/STAR & JCSDA

Research To Operations(R2O, R2X and O2R)

Sid-Ahmed Boukabara

The JCSDA Summer Colloquium, August 7th, 2015, Fort Collins, CO

NOAA/NESDIS/STAR & JCSDANOAA/NESDIS/STAR & JCSDA

2

Contents

What Is R2O or R2X in General?1

What is an O2R and why is it important?2

What is needed for a successful R2O/O2R?3

Specific Example of the JIBB and S45

How do we measure Success in R2O?4

Conclusions and Lessons Learned6

What Is R2O or R2X in General?

R2O is the process by which a scientific project led by a researcher, proven to provide added value, is transitioned to the Operational environment

R2X refers to the transition of a research project outcome to operations, applications, and services (for example to decision makers). R2O is an example of R2X

3

Why is R2O Important for NOAA?(and other operational partners)

R2O is an efficient way to leverage research done in NOAA and in the outside research community, to improve NOAA operational systems

R2O allows NOAA to maximize the return on investment (ROI) of its outsourced research

R2O allows NOAA to remain competitive, and at the edge of science in its products.

4

R2O ?

R2O means often different things to different people

5

Researcher in Academia

NESDIS

NWS

STAR

OSPO

R2O

EMC

NCO

R2O

R2O

R2O

JCSDA R2O Concept

Goal: Accelerates the use of satellite data in NWP centers

Ensures resources are in place to achieve successful transition: supercomputer, a software integration team, etc.

Ensures consistency and demonstrates benefits with operational systems like: Hybrid GDAS system (global), HWRF (Hurricane forecast).

Scientific efforts in

satellite DA in academia,

CIs


satellite DA in NOAA/OAR


satellite DA in research community

Scientific efforts in satellite DA in NOAA/NESDIS

(funded by GOES-R PGs, JPSS PGs,

etc)

Products, techniques, improvements, with direct and immediate relevance

to NWP Operational Models

(both global and regional)

Made consistent with Operational Systems

JCSDA’s own DA

Activities

NWP Operational CentersOngoing Baseline

Improvement in NWP centers

Objective: Improvements in Forecast skills

Diverse R&D activities

Select projects for R2O

Transition

7

Contents







What is an O2R and why is it important?

O2R infrastructure brings the Operational environment to researchers

It allows scientists to perform their research in a system that is compatible with operational needs

When a successful research project is performed in an O2R environment, it increases its chances to transition to Operations (R2O)

Without an O2R environment, R2O is at best difficult, (often does not happen). 8

O2R Is the Bridge Between R and O

9

Research Operations

Valley of Death

O2R

How does O2R/R2O work?

10

11

Contents







Components of an O2R/R2OInfrastructure(1)

The infrastructure is not just the supercomputer:What Money can Buy….

Supercomputer (Hardware, basic software, and IT) (almost the easiest part!) Scientific Software Integration (this is the hardest part: keep in synch with

operational models) User Support & Documentation Management of resources to allow R2O and O2R to be sustained Documented Utilities to help scientists (CRTM, assessment, formatting tools) Constant engagement between Research and Operations Rigorous software configuration system to track changes Rigorous (and independent) testing mechanism to demonstrate added value

12

Components of an O2R/R2OInfrastructure (2)

What Money can NOT Buy…. And perhaps more importantly: the right culture.

• Transparency of R2O protocols and willingness to accept research performed by researchers & work collaboratively (on the operational partner)

• Willingness to work/start with operational system and follow protocols and accept constructive feedback (on the research partner)

• Sense of Mutual respect between Researchers and Operational partners: respect for innovation in research and pragmatism in the operational implementation

13

Accept that R2O is not for everyone: careful choice of research and operations partners is important

14

Contents







Metrics for Assessing O2R Success

When R2O transition has occurred: assess specific scientific metric(s) that was(ere) improved upon transition to operational environment. Example: Anomaly correlation, hurricane track

improvement, etc.

When R2O transition has not occurred yet, assess progress in the maturity of the project, as a result of the O2R infrastructure

15

Example of a project that led to an R2O transition (Improved QC)

16

Significant

Southern hemisphere 500-hPa height ACC for 10 September – 24 October 2012 for the control (black) and LNVD (red).

Bottom: Variation between LNVD (red) and the control (operational QC of MODIS winds: black) based on forecast hour.

All ACC differences outside of the bars are significant at the 95% confidence level.

The circle represents a statistically significant improvement for the Day-4 and Day-5 forecasts.

Plot courtesy of D. Santek, UW (S4 project)

How do we assess R2O-maturity?(Case of JCSDA/NESDIS)

17

NOAA Technical Readiness Level (TRL)TRL 1 Basic principles observed and reported

TRL 2 Technology concept and/or application formulated

TRL 3 Analytical and experimental critical function and/or characteristic proof-of-concept

TRL 4 Component/subsystem validation in laboratory environment

TRL 5 System/subsystem/component validation in relevant environment

TRL 6 System/subsystem model or prototyping demonstration in a relevant end-to-end environment

TRL 7 System prototyping demonstration in an operational environment

TRL 8 Actual system completed and “mission qualified” through test and demonstration in an operational environment

TRL 9 Actual system “mission proven” through successful mission operations

TRLs 1-2 are nominally considered Research, TRLs 3-5 are Development, TRLs 6-8 are Demonstration, and TRL 9 is Deployment, Implementation, or Operational Transition.

FF

O

Dir

ecte

d R

esea

rch

Vis

itin

g S

cie

nti

st

Pro

gra

m

JCSDA Modes of Operation

R2O

Pat

h

18

Contents







19

Supercomputer for Satellite Simulations and data assimilation Studies (S4)

• Overview:– NESDIS, in collaboration with UW

is making S4 available to scientists involved in satellite data assimilation and other activities related to it.

– One of the purposes is to consolidate data assimilation activities performed in the research community

– and to encourage the usage of NOAA operational models/tools/codes installed on it.

– Open policy for users (since hosted in academic institution).

– Only open source/non-restricted data allowed on S4.

– Status: Upgraded in 2014.

The S4 system, hosted by University of Wisconsin.

S4 technical Description

Brief Technical Description The S4 system is a Linux cluster (Dell/Intel hardware), 4700 CPU cores (60 Tflops) with 18TB of total RAM 1700TB in storage nodes Hosted in the UW/SSEC Data Center (with UPS) Able to run 4 GSI/GFS experiments simultaneously at T1534

System Administration and Resource Allocation SSEC provides system administration services and security

protocols for the system, NESDIS/STAR provides overall guidance on system

resource allocation and selection of users as well as scientific support (code management, software integration, etc)

20

S4 Intented ActivitiesThis system is intended to allow primarily (but not

exclusively) the following major activities: (1) Undertake satellite data assimilation experiments at global and/or regional

scales and the assessment of their impacts on forecast models skills, using currently flying satellite sensors and allowing scientists to test new science/methodology and

(2) In support of the activity above, undertake all necessary satellite data simulations, calibration, algorithms development/improvement, radiative transfer modeling and validation, quality control (QC) procedures, etc

(3) Perform Observing System Simulation Experiments (OSSEs) for new sensors (such as GOES-R and JPSS).

• This component is expected to address key questions that NESDIS constantly faces: (a) what is the impact and the added value of flying new sensors, (b) what is the optimal orbital configuration, (c) what are the best mitigation strategies possible, from an NWP perspective, given the budget and other constraints, (d) how to best coordinate with other national and international partners to achieve the most optimal Earth satellite observing system

21

S4 Governance & Management

22

• S4 is governed by a NESDIS/STAR policy memo identifying roles and responsibilities• IT support is handled by the host institution UW.

• Software Integration is handled by NESDIS (the same team handles the software integration on the JCSDA sister machine JIBB)

• To request an account, a simple email to STAR POC will suffice. Accounts usually set up the same day.

• The S4 committee includes NESDIS/STAR director, deputy, and three senior program scientists (JCSDA, JPSS and GOES-R), makes recommendations concerning 90% of the machine's resources. The remaining 10% is managed by UW.

Policies and Code management on S4

Code management is the hardest part• The software integration team is dedicated to making sure the NOAA codes installed on S4

are in synch with the operational models (including scripts, codes, datasets, libraries, etc)• SVN is extensively used when appropriate.

The following is a list of S4 Policies:– User account request– Managing resources– Account expiration– Tools and softwares guidance– User guide

Compiler & Software Availabilityo Intel Fortran / C++o Gfortran /gcc/g++o Netcdf 3 & 4o Matlabo IDLo NCO 4o GRADSo HDF 4 & 5o NCL

23

How much did Projects benefit from S4?

24

Project Title Project Leads Operational Partner Point of Contact at Oper. Partner

Expected Date (best guess) for

R2O

R2O Current Stage(TRL)

TRL before using S4

1. A New Quality Control Method for Satellite-derived Polar Winds in the NCEP GDAS/GFS

David Santek, and Brett Hoover

NCEP Andrew Collard, John Derber

Dec. 2015 8 6

2. Proactive Quality Control (PQC) based on Ensemble Forecast Sensitivity to Observations (EFSO)

Daisuke Hotta, Tse-Chun Chen and Eugenia Kalnay

NCEP Daryl Kleist, Mark Iredell

January 2016 7 3

3. Development of Community Satellite Data Thinning and Representation Optimization Tool (CSTROT)

Tong Zhu and Sid Boukabara

NCEP Andrew Collard June 2016 6 2

4. A near real time regional satellite data assimilation system for high impact weather research and forecasts

Jun Li, Jinlong Li, and Pei Wang

NHC and EMC NHC: Mark DeMaria, John L. BevenEMC: Vijay Tallapragada

October 2014 to NHCJune 2016 to EMC

3 1

5. Tropical cyclone data assimilation of MSG SEVIRI all-sky infrared satellite radiances

Milija Zupanski and Man Zhang

NCEP/EMC was Steve Lord begin in 2016 6 2

6. Studying tropical cyclone genesis and rapid intensification with the NCEP GSI data assimilation system

Zhaoxia Pu NCEP/EMC Vijay Tallapragada Dec. 2015 6 3

7. Assessment of Temperature and Humidity Prediction with Microwave and Hyperinfrared Satellite Data Assimilation over United States

Jianjun Xu, and Alfred Powell,Jr

STAR Mike Kalb &Sid Boukabara

June 2016 3 1

Maturity Assessment of S4 projects

25

Added value of S4 O2R Environment measured by the maturity-index (before and after using the S4)

Examples of S-NPP Projects Benefiting from O2R(Not directly funded by JCSDA, but supported through the O2R: HPC, user support, Porting of codes, etc)

# PI/Scientist Project Title Main Sensor used

1 Jinlong Li Near real-time assimilation system development for improving tropical cyclone forecast with NPP/JPSS sounder data

-CrIS, ATMS

2 Banglin Zhang -Improve Hurricane Structure Monitoring and Intensity Forecast Using NPP ATMS and GCOM-W AMSR2

-ATMS

3 Jianjun Xu Application of hyper-spectral satellite data in severe weather forecasts with GSI/WRF regional data assimilation system

-CrIS

4 Lin Lin -Evaluation of BG-Remapped ATMS on Hurricane Sandy Forecast & Test of Mie and DDA Scattering in the Simulation of ATMS

-ATMS

5 Marouane Temimi Assessment of Assimilating NPP/JPSS ATMS Ground Surface Observations -ATMS

6 Li Fang, et al. Dual Assimilation of Microwave and thermal-infrared observations of soil moisture into NLDAS for improved drought monitoring

- ATMS, VIIRS

7 Marouane Temimi Assessment of Assimilating NPP/JPSS ATMS Ground Surface Observations -ATMS

8 Hu Yang Scientific Support for JPSS Instrument Calibration -S-NPP Sensors (ATMS)

9 Daisuke Hotta Proactive-QC based on Ensemble Forecast Sensitivity to Observations (EFSO) -All S-NPP assimilated sensors (ATMS)

10 Xinjia Zhou BTs validation of CRTM simulations against AVHRR, MODIS and VIIRS BTs in SST bands

-VIIRS

11 Johnathan W. Smith

WRF-Chem data assimilation of NOAA-Unique CO retrievals: Impact study on tropospheric ozone enhancement observed during the 2010 AEROSE Campaign

-IASI (potential application to CrIS)

12 Agnes Lim Huei Ni Impact analysis of LEO Hyperspectral Sensor IFOV size on the next generation NWP model forecast performance

-CrIS

Resources installed for Global and Regional DA and OSSE Activities

The following models & tools are installed on S4: Global Data Assimilation System (GDAS) (including the new Hybrid

Ensemble system) Grid Statistical Interpolation (GSI) Weather Research & Forecast (WRF) (WRF-ARW and WRF-NMM) Hurricane Weather Research & Forecast (HWRF) Microwave Integrated Retrieval System (MIRS) Community Radiative Transfer Model (CRTM) Verification Statistics Data Base (VSDB) Ocean Model HYCOM Etc…

The following models & tools will be installed in the future: NOAA Environmental Modeling System (NEMS) Unified Post Processor (UPP) Land Information System (LIS) Etc..

Major projects on jibb

General data impact and OSSE experiments for Several Sensors

Next generation NOAA hybrid data assimilation system port

Hyperspectral IR methodology Hyperspectral IR cloudy radiances Satellite wind impact experiments Lightning assimilation methodology FFO projects of the JCSDA NASA Roses Projects of the JCSDA

28

JIBB and S4 Benchamarking

Efforts have been devoted to compare satellite data experiments results on S4 and JIBB, to those performed on the NOAA R&D

Benchmarking performed for currently operational GDAS (GSI+GFS)

The results show that the operational version of GDAS is running well on S4 and JIBB (both T382 and T574)

Efforts included the benchmark of the hybrid data assimilation system.

29

Anomaly correlation performed using GDAS system on NOAA R&D machine (Vapor), JCSDA JIBB machine and the S4 system. Benchmarking performed by J. Jung and the STAR/JCSDA SI team. Courtesy of J. Jung.

30

Contents







Lessons learned for an effective R2O

Importance of the O2R environmentImportance of availability of an accessible HPCImportance of a transparent R2O protocolImportance of code portability & synchonizationImportance of enforcing the exclusive usage (by researchers)

of the operational systems Importance of a rigorous traceability of changesImportance of the S4 and JIBB approaches

Importance of nurturing a culture of mutual respect between research and operations partners

31

Coordination & Synchronization: JCSDA Accelerated R2O coordination for SNPP/ATMS

32

• SNPP Was launched Nov 2011

• Dec 2011, the JCSDA coordinated the signature of the R2O Action Plan for ATMS R2O between NESDIS and NWS

• The main points were: Who, does what and when and where (between NESDIS and NWS)

• Strong backing from STAR and NCEP Directors (present at kick off meeting)

• 7 months after launch, ATMS was assimilated operationally in NOAA

• Important of ‘coordinate & synch’

Conclusions JCSDA and NESDIS are involved in R2O by making significant

resources available to scientists and collaborators (HPC, support, ..)

The two major activities targeted are: (1) Advanced Satellite data assimilation and (2) OSSE activities.

The O2R infrastructure is more than the supercomputer: components include software management, integration and user support, etc

The JIBB and S4 offer an O2R that is both operationally relevant as well as open and inclusive

S4 and JIBB access offers an O2R environment conducive of innovation, but with a clear R2O objective.

33

34

Questions?

Documents

Research To Operations (R2O, R2X and O2R) Sid-Ahmed Boukabara The JCSDA Summer Colloquium, August 7 th, 2015, Fort Collins, CO NOAA/NESDIS/STAR & JCSDA