Overview of Observing System Evaluation in GODAE OceanViewgodae-data/Symposium/GOV... · 2019-03-01 · Nov 07 Nov 08 June 09 Oct 10 June 11 Nov 11 Nov 12 Sept 14 (Paris) (Nice) (Toulouse)

GODAE OceanView Symposium, Hilton Baltimore, 4-6 November 2013

Overview of Observing System Evaluation in GODAE OceanView

Peter Oke1, Gilles Larnicol2

and all task team members

1CSIRO Marine and Atmospheric Research; 2CLS

November 2013


GOV OSEval-TT organisation

Co-Chairs:

Peter Oke (CSIRO)

Gilles Larnicol (CLS)

Associate members:

Mike Bell (UKMet)

Eric Dombrowsky (Mercator-Océan)

Fabrice Hernandez (Mercator-Océan)

Eric Lindstrom (NASA)

Andreas Schiller (CSIRO)

Hans Bonekamp (EUMETSAT)

Core Members:

Magdalena Balmaseda (ECMWF)

Anthony Weaver (CERFACS)

Laurent Bertino (NERSC)

Pavel Sakov (NERSC)

Gary Brassington (BoM)

Jim Cummings (NRL)

Pat Hogan (NRL)

Yosuke Fujii (JMA/MRI)

Villy Kourafalou (Univ. Miami)

Daniel Lea (UKMet)

Matthew Martin (UKMet)

Avichal Mehra (NOAA)

Elisabeth Remy (Mercator-océan)

Greg Smith (EC)

Clementa Tanajura (Remo)

Zhang Tianyu (NMEFC)


To provide consistent and scientifically justified

requirements and feedbacks to agencies in charge of

Global and Regional Ocean Observing Systems

GOV OSEval TT Objectives


Evaluation studies

Perform impact studies of GOOS and ROOS on forecast and reanalysis

GODAE systems

Methodologies: OSE, OSSE and alternative methods

Provision and dissemination of Observation

Impact Statements (OIS) based on OSEval

evidence

Capacity Building

GOV OSEval TT Objectives


Organisation of workshops to provide a forum for interactions:

• discuss best-practice methods and details of the technical and scientific aspects of impact studies;

• share experiences with new members; and

• coordinate activities and synthesize results in order to provide homogeneous feedback to observational agencies.

Relationships to broader communities:

• Coordination with OOPC and Clivar/GSOP on the climate and seasonal

forecasting applications progressively maintained

Capacity Building


Capacity Building

1st GODAE OSE/OSSE

workshop

Nov 07 Nov 08 June 09 Oct 10 June 11 Nov 11 Nov 12 Sept 14

(Paris) (Nice) (Toulouse) (Tokyo) (Santa Cruz) (Paris) (Rio) (Europe)

2nd GOV OSEval TT

workshop

OSEval-TT established, definition of a work plan

Consensus on the Routine Monitoring of Obs systems

Emergence of the idea of a TT dedicated to OSE

3rd wks: GOV &

CLIVAR GSOP

NRT OSE set up & demonstration

Concept of “Observation Impact Statement”

GOVST-II GOVST-I GOVST-III GOVST-IV

4th

workshop

To be organised - with E-AIMS European project

Nov 13

(Washington)


Provide feedback and requirements to the observation agencies

through evidence of impact of the observations in GODAE systems

Examples of responses to “observing system events”: - J1 new orbit (2009): recomm for an orbit that preserves mesoscale sampling

- J1 end of life (2010): recomm to maintain a good altimeter constellation

-ENVISAT loss (2012): collect feedbacks on the impact of the EN loss.

- Cryosat 2 ocean data (2012): recomm to deliver C2 ocean data

- XBT (2012): justification for continuance of the XBT line in Australia

- TAO (2013): GOV OSEval has been asked to provide feedback and requirements

7

Provision and dissemination of

Observation Impact Statements

(OIS)


Provide feedback and requirements to the observation agencies

through evidence of impact of the observations in GODAE systems

Examples of responses to “observing system events”: - J1 new orbit (2009): recomm for an orbit that preserves mesoscale sampling

- J1 end of life (2010): recomm to maintain a good altimeter constellation

-ENVISAT loss (2012): collect feedbacks on the impact of the EN loss.

- Cryosat 2 ocean data (2012): recomm to deliver C2 ocean data

- XBT (2012): justification for continuance of the XBT line in Australia

- TAO (2013): GOV OSEval has been asked to provide feedback and requirements

8



(OIS)

Lesson learnt:

Reactivity

Need to provide up-to-date results

Simple messages


• Definition of Observation Impact Statements

9



(OIS)

OISs are intended to be the primary

mechanism by which the OSEval-TT

will communicate findings of the

impact of observations on analysis

and forecast systems to observational

agencies

OISs are short reports that include a

standard set of graphics and tables

that can be easily understood by

stakeholders that are not experts in

data assimilation


Evaluation studies

Perform impact studies of GOOS and ROOS on forecast and

reanalysis GODAE systems

OSE, OSSE and alternative methods

Provision and dissemination of Observation

Impact Statements (OIS) based on OSEval

evidence

Synthesis of impacts studies

Promote observation impact studies as core operational

activities for all ocean forecast centers

feedbacks

Requirements

support

Get the most

out of the obs

to improve the

GOV syst.


Routine

Monitoring of the

GOOS

Delayed-mode

assessments of the

GOOS

Design and

evaluation of new

and future

observing system

components

• GOV OSEval-TT solution was to set up Near-Real-Time Observing System Experiments;

• It consists in running in parallel to the nominal operational forecast system and a second system where a single observation component of the GOOS will be with-held; and

• Implemented and tested by UKMet group in 2011#, under tested by Mercator-océan and Bluelink.

# Lea, D. J., M. J. Martin, P. R. Oke 2013: Demonstrating the complementarity of observations in an operational ocean forecasting system. Quarterly Journal of the Royal Meteorological Society, in press.

Synthesis of impacts studies


Feb 2011

Mar 2011

Apr 2011

May 2011

Jun 2011

July 2011

XBT

TAO

Jason-2

All altims

SST

Argo

• paper in press by Lea et al. (2013; QJRMS)

• Methodology needs to be refined to take into

account: time influence of a data type,

dependence to a system (model/assimilation)

• Difficulties: relatively costly to maintain two

operational systems in parallel

Routine monitoring of the GOOS:

NRT OSEs

Temperature differences

for July 2011

Impact of ARGO

30 m

100 m

Data are with-held for one month -2°C +2°C



Alternative methods

Forecast sensitivity method using the NRL HYCOM system: these diagnostics show the importance of all observation types

(blue positive impact … improves the forecast; red negative impact … degrades the forecast)

Argo Temperature Altimetry XBT

Total Data Impact

For Jul-Aug 2012



Alternative methods Degree of freedom of Signal (DFS) tested by several groups

(Bluelink, TOPAZ, Mercator-Océan, DUACS NRT System, …)

Background and analysis error estimates (e.g., Bluelink)






Variability Analysis error

Salinity error at 100 m

Impact of Argo array






Pre-Argo

(univariate) Salinity error at 100 m






Demonstrating the additional constrain on salinity of the Argo

Post-Argo

(univariate) Salinity error at 100 m


Routine

Monitoring of the

GOOS

Delayed-mode

assessments of the

GOOS

Design and

evaluation of new

and future

observing system

components

Synthesis of results


• OSEs using the Bluelink system with 0, 1, 2, and 3 altimeters;

• 1/10O-resolution - EnOI - Australian regional model

(showing 90th percentile of absolute differences between OSEs) (P. Oke)

The 1st altimeter has biggest impact – but the 2nd and 3rd altimeter provide

additional constraint 19

Impact of satellite altimetry June 2009

Independent XBT

3ALT

2ALT

1ALT

0ALT

NoDA


• OSEs using the REMO system

• 1/4o resolution - EnOI - Atlantic Basin

(showing RMSD with Argo profiles) (C. Tanajura)

Argo is the only data set to constrain salinity; and has the biggest impact on temperature

Impact of Argo, SST and altimetry


Skill increases with more Argo floats

More floats may further improve skill

Impact of Argo • OSEs using the JMA-MRI MOVE-G

system – 1/3o-1o resolution – 3dVar

(showing the change in anomaly correlation: OSEArgo-OSEAll)

(Y. Fujii)


Impact of TAO and Argo • OSEs using the JMA-MRI MOVE-G system – 1/3o-1o resolution – 3dVar

(showing the change in anomaly correlation: OSEArgo-OSEAll) (Y. Fujii)

Ch

an

ge

to

an

om

aly

co

rre

latio

n

Assimilation of Argo and TAO data improves forecasts on 1-

13 month times-scales


TAO data improves forecasts by ~0.4o in the Tropical Pacific

Impact of TAO, altimetry and Argo • OSEs using the ECMWF

system – ORSA4 (M. Balmaseda)

TAO data improves forecasts by ~0.4o in the Tropical Pacific


Impact of loss of all data

• OSEs using the Canadian Global Ice Ocean Prediction System (GIOPS) - 1/4o resolution global model – Mercator’s SEEK/EnOI system (G. Smith)

(showing global RMS difference for temperature)

The model “forgets” about assimilated data over

several months.


Impact of XBT • U Miami / NOAA (V. Kourafalou)

(showing impact on ocean heat content)

No airborne

profiles

1 deg XBT

sampling

0.5 deg

XBTs only

to 400 m

0.5 deg XBTs to

1000

m+altim+SST

High-res XBT data constrains features that

are not resolved by altimetry


Conclusions from OSEval studies

• Satellite altimetry: regarded as the most critical data type for short-range mesoscale ocean forecasts – multiple satellite are needed

• Argo: regarded as the most critical data type for seasonal prediction – is the only data set to constrain salinity – evidence that more floats may return additional benefit;

• TAO: Improves seasonal forecasts (1-13 months);

• Data outages: Models “forgets” about assimilated data over several months

• XBT: significant local impacts – constrain features that are not resolved by altimetry

• Several studies highlight that different data types are complimentary

• Very few OSSE – lower priority than OSEs

• No Impact studies for SMOS and Aquarius

(See poster session for detailed results)

26


Overall Summary

GOV OSEval-TT is an active community that is growing. Almost all the GODAE partners belongs to the TT;

New members give us the opportunity to revisit the impact studies(with new independent systems);

GOV OSEval-TT try to support observation agencies in demonstrating

the impact of observations on forecast and analysis products; Thanks to the different activities defined within the TT (NRT & DT

OSEs and alternative studies – DFS, …) and commitment from TT members we are able to cover large part of the demand from observational agencies;

Promote evaluation studies as a core activities of the GOV systems.


Future

Continue to advocate to support observing system agencies;

Disseminate results of OSEval studies to observational agencies

Workshop: next one in Sept-Oct 2014

Assess impacts (or potential impacts) of biogeochemical observations; SAR altimetry; SWOT, Deep Argo …

Improve and develop new diagnostics to assess the observation impact depending of the applications (temporal and spatial scale of interest)

Improve and develop approaches to assess the contribution of one component of the observing system in the context of the whole observing system (relative weight of each component)

OSSE – coordinated activities … or not?


Acknowledgements

Yosuke Fujii3, Emlyn M. Jones1, Vassiliki Kourafalou4, Ann Kristin Sperrevik5, Gregory C. Smith6, Daniel J. Lea7, Fiona Carse7, Baptiste Mourre8, Stephanie Guinehut2, Elisabeth Remy9, Clemente A. S. Tanajura10,11, Magdalena Alonso Balmaseda12, George R. Halliwell, Jr.13, Matthieu Le Hénaff4, Robert Atlas13, Andrew M. Moore11, Christopher A. Edwards11, Matthew J. Martin7, Alistair A. Stellar7, Alberto Alvarez8, Sandrine Mulet2, Isabelle Pujol2, Jim Cummings14

1CSIRO Marine and Atmospheric Research, Hobart, Tasmania, Australia 2CLS-Space Oceanography Division, Ramonville-Saint-Agne, France 3JMA/MRI, Tsukuba, Ibaraki, Japan 4University of Miami/RSMAS, Miami, FL USA 5Norwegian Meteorological Institute, Oslo, Norway 6Meteorological Research Division, Environment Canada, Quebec, Canada 7Met Office, Exeter, UK 8Centre for Maritime Research and Experimentation, La Spezia, Italy 9Mercator Ocean, Ramonville-Saint-Agne, France 10 Physics Institute, Federal University of Bahia (UFBA), Salvador, Brazil 11Department of Ocean Sciences, University of California, Santa Cruz, CA USA 12European Centre for Medium Range Weather Forecasts, Reading, UK. 13NOAA/AOML, 4301 Rickenbacker Causeway, Miami, FL, 33149, USA 14NRL, Monterey, CA, USA


Impact of Coastal biogeochemical moorings

• EnOI “footprints” using MODIS satellite data (E. Jones)

Weather-band

(10-30d)

Intra-monthly

(30-90d)

Intra-seasonal

(90-180d)


Impact of Coastal biogeochemical moorings

• EnOI “footprints” using MODIS satellite data (E. Jones)

A sparse array of moorings provides a

decent chance to monitor intra-sesonal

BGC variability

Weather-band

(10-30d)

Intra-monthly

(30-90d)

Intra-seasonal

(90-180d)

Isolated moorings over the GBR provide litte

constraint of short-time-scale BGC variability






Demonstrating the additional constrain on salinity of the Argo program

Pre-Argo

(multi-variate)


Post-Argo

(multi-variate)





Demonstrating the additional constrain on salinity of the Argo program

Documents

Overview of Observing System Evaluation in GODAE OceanViewgodae-data/Symposium/GOV... · 2019-03-01 · Nov 07 Nov 08 June 09 Oct 10 June 11 Nov 11 Nov 12 Sept 14 (Paris) (Nice) (Toulouse)