Overview Jeff Morisette [email protected], (301) 614-5498 EGU Breakout on Validation of Global Albedo products Vienna Austria 27 April 2005 LAND

Overview

Jeff [email protected], (301) 614-5498

EGU Breakout on Validation of Global Albedo productsVienna Austria27 April 2005

LAND PRODUCT

LPV overview for Albedo '05 2

LPV outlineLPV outline

• Review of subgroup goals and objectives

• Overview of LPV activities– CEOS core sites– Special issue on global land product validation– Accuracy statements and accuracy layers

• Validation protocol and international comparison– Leaf area index example– Albedo intercomparison



• Review of subgroup goals and objectives

• Overview of LPV activitiesOverview of LPV activities– CEOS core sitesCEOS core sites– Special issue on global land product validationSpecial issue on global land product validation– Accuracy statements and accuracy layersAccuracy statements and accuracy layers

• Validation protocol and international comparisonValidation protocol and international comparison– Leaf area index exampleLeaf area index example– Albedo intercomparisonAlbedo intercomparison


CEOS/WGCV/LPV Organizational StructureCEOS/WGCV/LPV Organizational Structure

CEOSCommittee on Earth Observing Satellites

WGCVWorking Group on Cal/Val(Current Chair Steve Ungar/Goddard)

LPVLand Product Validation Subgroup(current chair = Jeff MorisettePast chair = Jeff PrivetteFuture chair = Fred Baret)


http:landval.gsfc.nasa.gov/LPVShttp:landval.gsfc.nasa.gov/LPVS

Matches WGCV page layout and graphic

Quick links to:• Listserves• Announments• WGCV• CEOS and• CEOS calendar

Pull-down menu for main topical areas:•Land cover•Biophysical•Fire/Burn•Surface Radiation

Each pull-down lists:•Background•Producers *•Meetings•Case studies•Intercomparisons

* input needed

web curator: Jaime Nickeson


Validation:the process of assessing by independent

means the quality of the data products derived from the system outputs

LPV operates under this definition, but with the understanding that validation activities should consider user accuracy needs and feedback to algorithm improvements.

CEOS DefinitionCEOS Definition


Mission Statement & GoalsMission Statement & Goals

• to foster quantitative validation of higher level global land products derived from remote sensing data and relay results so they are relevant to users

• to increase the quality and economy of global satellite product validation via developing and promoting international standards and protocols for field sampling, scaling, error budgeting, data exchange for global land product validation

• to advocate mission-long validation and intercomparison programs for current and future earth observing satellites.


Big PictureBig Picture

LPV provides a validation service to the Integrated Global Observation Strategy’s:

Global Terrestrial Observation System and Global Carbon Observing System

and, potentially, the GEOSS (Global Earth Observation System of Systems)

Implications:– Focus Products of GTOS – Working in conjunction with GOFC/GOLD’s regional networks– Consider validation implications for GEOSS focus areas


INTEGRATION WITH GLOBAL OBSERVING SYSTEM FOR CLIMATEIMPLEMENTATION PLANINTEGRATION WITH GLOBAL OBSERVING SYSTEM FOR CLIMATEIMPLEMENTATION PLAN

Essential Climate Variable for land:

LPV will watch if & how these are adopted by GEOSS

WMO, 2004. Implementation Plan for the Global Observing System for Climate in Support of the UNFCCC, October, GCOS - 92, WMO/TD No. 1219, United Nations Environment Programme International Council.

River DischargeLake Level/AreaGround WaterWater UseSnow CoverGlaciers and Ice Caps

Permafrost & seasonally frozen groundAlbedoLand CoverfAPAR and LAIBiomassFire Disturbance


GEOSS focus areasGEOSS focus areas

• natural and human-induced disasters• environmental factors affecting human health• management of energy resources• climate variability and change• water resource management• weather information, forecasting, and warning• terrestrial, coastal, and marine ecosystems• sustainable agriculture and combating

desertification• biodiversity



• Review of subgroup goals and objectivesReview of subgroup goals and objectives

• Overview of LPV activities– CEOS core sites– Special issue on global land product validationSpecial issue on global land product validation– Accuracy statements and accuracy layersAccuracy statements and accuracy layers



CEOS Core SitesCEOS Core Sites

“CEOS Core Sites”: WGISS Test FacilityJoint project between CEOS Working Group on Cal/Val and

Working Group on Information Systems and Services

The WGCV 22nd plenary (June 2004) established, for the long term sustainability of this effort, it must be a distributed system where each space agency stages data and derived-products from their sensors.



ETM+ MODIS ASTER SRTM NED Lidar AVHRRSPOT VEG MERIS CHRIS

Barton Bendish X X O O X

Boreas North X X X O O X

Harvard X X X X O O X

Mongu X X X O O X

Puget Sound X X X Link O O

Railroad Valley X X X X O O

Southwest Amazon O O

Uardry X X O O X

X = data at USGS/EROSO = “distributed data”



http://edcsgs16.cr.usgs.gov/wgiss/




• Overview of LPV activities– CEOS core sitesCEOS core sites– Special issue on global land product validation– Accuracy statements and accuracy layersAccuracy statements and accuracy layers



LPV “Special Issue” – ongoingLPV “Special Issue” – ongoing• Special Issue: describing the state of the art research on both protocol

and results for validation and accuracy assessment of global land products (Liang, Baret and Morisette, eds.)

• Over 20 papers have been submitted, covering land cover, burned area, biosphysical (VI, LAI, fAPAR, GPP), and BRDF.

• Several members from the user community have agreed to write a note for each section on the implication for the uncertainty/validation of a given product.

MFJDNSAJJMAMFJDNOSAJJMAM

2004 2005 2006

Announcement

Validation papers submissions reviews revisions review final/profs

User perspective papers submissions reviews revisions final/profs

Publication date March 2006 ->

Currently on schedule!




• Overview of LPV activities– CEOS core sitesCEOS core sites– Special issue on global land product validationSpecial issue on global land product validation– Accuracy statements and accuracy layers



Accuracy/Confidence LayersAccuracy/Confidence Layers

MODIS Land cover product (MOD12Q1)

MODIS land cover Associated Confidence layer(lighter = more confidence)

Note lower confidence in deforestation areas


Accuracy statementsAccuracy statements

• Should be “user-oriented” and supported with peer-review literature

• Standardize/summarize information for each product

• MODIS land team had planned to update CEOS information for MODIS land products


CEOS/WMO pageCEOS/WMO page


Link to accuracy statement for each product … supporting materials

• Overall accuracy statement• Link to QA information• List of support material

•Title, author, abstract•Figures/captions•Tables/captions

CEOS/WMO database, potential frameworkCEOS/WMO database, potential framework

Producer maintained validation page

CEOS/WMO data base WGCV subgroup page


Foster’s Kreb’s ruleFoster’s Kreb’s rule

“every datum should come with an explicit estimate of its uncertainty.”*

LPV is working to apply this rule to satellite derived global land products.

* Krebs's Ecological Methodology (1989) (from Personal communication with Foster Brown)


“Typical” Validation“Typical” Validation

60 70 80 90 100 110 120

b1

20

40

60

80

100

120

b3

“Reference data” (such as a field observation)

Sat

ellit

e pr

oduc

t va

lue

Reference DataClass A Class B … Class N

Satellite Class A X e … eProduct Class B e X … e

… … … … …Class N e e e X

Regression analysisorError Matrix


Validation to assess confidence layerValidation to assess confidence layer

40 50 60 70 80 90 100 110

b2

20

40

60

80

100

120

b3

Confidence Layer values

Agr

eem

ent

betw

een

prod

uct

and

obse

rved




• Overview of LPV activitiesOverview of LPV activities– CEOS core sitesCEOS core sites– Special issue on global land product validationSpecial issue on global land product validation– Accuracy statements and accuracy layersAccuracy statements and accuracy layers

• Validation protocol and international comparison– Leaf area index example– Albedo intercomparison


Strategy for developing protocolsStrategy for developing protocols

LPV is working toward protocols with three steps:

– Workshops (kick off, strategy/work plan, results)Bringing together producers, users, and validation experts to initiate discussion, establish the “state of the art”, and consider core sites or regions for validation activities

– Case studies (previously) - Inter-comparisons (currently)Posted on the LPV web siteFirst step in developing a more formal protocol

– Publication(s) (special issue)Peer review document with details pertaining to the validation of a given global land product.


“Intercomparison” General Timeline“Intercomparison” General Timeline

LAI Albedo Fire Land coverTopical meeting

to establish data requirements

Decide on Sites

Develop data sharing infrastructure

Field Campaigns & individual product analysis

Synthesis of results

Boston UPrivette et al. 1998

Frascati, ItalyPrivette et al. 2001

MontanaAugust 2004

Current, on-goingresearch

Boston UPrivette et al. 2002

EGU, Vienna2005

Lisbon - fireMorisette et al. 2001Darmstadt(geostationary)2004

Toulouse2001Percent cover:2005

Boston U2004


Background on LAI intercomparisonBackground on LAI intercomparison

• The initial research efforts to validate global Leaf Area Index products included the European “VALERI” program, the NASA-funded “BigFoot” program, BU’s interest in validating its own product and Canada’s validation of their country LAI product (Privette et al. 1998)

• CEOS LPV brought these and additional efforts together for its initial “intercomparison” (Privette et al., 2001)

• Currently, nine groups are participating and sharing LAI-field data and high-resolution LAI images covering 56 sites(Morisette et al., 2004)

• Collaboration involves sharing data, software, and ideas.


LAI workshop results: Global product validation framework LAI workshop results: Global product validation framework

a) Organizing entity: CEOS LPV

b) Participants- interest in using and/or validating global LAI products

-willing to share data-existing resources/funding-ability to meet occasionally

Site-specific procedures and results from each participant(exists for ~50 sites)

c) Data sharing mechanismfor site-specific field data and high resolution LAI maps from each site(Mercury system at ORNL)

d) Synthesis of resultstoward global accuracyassessment(research needed)


High spatial resolutionimage (SPOT/ETM/ASTER …)

Individual measurements

Value at the ESU level

Value(s) at the site level

Global validation

Averaging

Transferfunction

Correlationanalysis

50-100 sites

10-100 measurements

(ElementarySampling

Unit)

20-100 ESUs/site

Local sampling schemes for an ESU12 local measurements (LAI2000/Hemi-phot) required

Local sampling schemes for an ESU12 local measurements (LAI2000/Hemi-phot) required

D

D

D DDD D

D

D

D

D D

D

D

DD

D

D

DDD

D

D

D

D D

D

DD

D

D

DD

D

D

DD

D

D

DD

D

D DDDD

D

D

D

DD

D

D

DDD

D

D

DD

D

D

DD DD D DD

DDD

DD D

DDD DD D

D

D

D

D

DD

D

DD

DD

D

FFF

F

FF

F

FF

F

FFFF

F F

F

FFFF

FFF FFF

F FFFFFF

FFF

F

F

F

FF

FF

FF

FF

F

F

F

FFFFFF

FF

FFF

F

FFFFFFFFF

FFFFF

F

F

F

FFF

FFFFFF FF

FFF

F FFFFFF

F

FFF

F

FF

FFF F

F

FFF

F

F

FF

FFF

FFFFFF

FF

FFFFF

F FF F

FF

FFF FFFFFF

FF

F

FF FF F

F F

FF

F FFFFF

AA

A

A

A

A

A

A

AA

AA

AAA

AA

A

A

A

A

A

A

A

AA

A

A

A

A

A

AA

A A

A

AA

CC C

CC C

C

C

C C CC

C

C C

CC

CCCC

CC

CC C C

C

C

CC C

C

C

CCCC

C

C

CC

C CC C

CC

C C

CCC

CC C

C CCC

C

C C

C

C

C

C

CC

C

C

C

C

C

C

C

-180 -150 -120 -90 -60 -30 0 30 60 90 120 150

90

60

30

0

-30

-60

Medium resolution products to be validated

General global product validation protocolGeneral global product validation protocol


Global map of LAI sites

Bare Water DBF ENF EBF Crops Grass

Validation of global moderate resolution LAI Products: a framework proposed within the CEOS Land Product Validation subgroup, Morisette, et al, in press, TGARS special issue

LAI Intercomparison sites:9 groups and 56 sitesLAI Intercomparison sites:9 groups and 56 sites


Representative nature of site networks Representative nature of site networks

Evaluation of the representativeness of networks of sites for the validation and inter-comparison of global land biophysical products. Proposition of the CEOS-BELMANIP, Baret, Morisette, et al, in press, TGARS special issue

• VALERI (27)• CCRS (27)• BIGFOOT (9)• Others (35)• TOTAL=100

0 5 10 15 20 25 30 35

-60

-40

-20

0

20

40

60

component fraction (%)

latit

ude

(°)

GrassCropsEvergreen broadleafConifersDeciduous BroadleafBare surface

0 5 10 15 20 25 30 35

-60

-40

-20

0

20

40

60

component fraction (%)

latit

ud

e (

°)

GrassCropsEvergreen broadleafConifersDeciduous BroadleafBare surface

Global distributionGlobal distribution Field site distributionField site distribution


Albedo/BRDF IntercomparisonAlbedo/BRDF Intercomparison

2005 Avignon, France Co-chairs: Fred Baret, Crystal Schaaf, Jeff Privette, Jeff Morisette

This workshop will build on the discussions held at the first LPV workshop on Albedo Products held in Boston, USA in October 2002.

The goals of this day and a half workshop are to design and initiate a validation and comparison exercise for the various satellite-derived land surface albedo products which are now available from a number of instruments.

It is hoped that participants from a number of long term field sites will be willing to work with satellite albedo producers for one or more designated periods to provide an assessment of the operational products. While these would primarily be "virtual" campaigns - utilizing existing field and satellite data resources - the LPV will be able to provide limited support for posting data and comparisons and facilitating communication on the web.


Morisette, J. , C. Justice, J. Pereira, J.M. Grégoire, and P. Frost, 2001, “Report from the GOFC – Fire: Satellite Product Validation Workshop”, Earth Observer, September/October, v. 13, n. 5, p. 15-18..( available on-line at http://eospso.gsfc.nasa.gov/eos_observ/9_10_01/Sept_Oct01.pdf

Morisette, Privette, Strahler, Mayaux, Justice, “Validation of Global Land-Cover Products by the committee on Earth Observing Satellites”, Geospatial Data Accuracy Assessment, Lunetta and Lyon eds., 2004.

Morisette, Jeffrey L. Privette, Jaime Nickeson, Frèdéric Baret, Ranga B. Myneni, and Nikolay Shabanov, Summary of the Third International Workshop on LAI Product Validation, Earth Observer, Sept./Oct. 2004, v.16, n.5, p.28-31 (available on-line at http://eospso.gsfc.nasa.gov/eos_observ/pdf/Sept-Oct04.pdf).

Privette, J., R. Myneni, J. Morisette and C. Justice, 1998. Global validation of EOS LAI and FPAR products, EOS Earth Observer, 10(6) 39-42.

Privette, J.L, J.T. Morisette, F. Baret, S.T. Gower and R.B. Myneni, 2001. Summary of the international workshop on LAI product validation, EOS Earth Observer, 13(3) 18-22.

Privette, J.L., C.B. Schaaf, A. Strahler, R. Pinker, M. Barnsley, and J. Morisette, 2002. Summary of the international workshop on albedo product validation. EOS Earth Observer, 14 (6) p.17-18.

WMO, 2004. Implementation Plan for the Global Observing System for Climate in Support of the UNFCCC, October, GCOS - 92, WMO/TD No. 1219, United Nations Environment Programme Internation Council.

referencesreferences

Documents

Overview Jeff Morisette [email protected], (301) 614-5498 EGU Breakout on Validation of Global Albedo products Vienna Austria 27 April 2005 LAND