Upload
grant-carson
View
218
Download
0
Tags:
Embed Size (px)
Citation preview
evaluation
aerosol CCI retrievals
Reading 2012
the participants / the task
• AATSR F v142• AATSR O v202/v2q2• AATSR S v040/v031• MERIS A v21• MERIS B v11• MERIS E 802• PARASOL v30
• MODIS_aqua c5.1
• MODIS_terrac5.1
• SEAWIFS• MISR v31
• evaluate daily global
‘CCI-aerosol’ retrievals
– for the entire year 2008• AOD 550nm• Angstrom (if offered)
• versus AERONET / MAN
• performance in context
of established retrievals
ECMWF AOD
FBOV (single assim.)FMNG (dual assim.)
less aerosol
ECMWF Angstrom
FBOV (single assim.)FMNG (dual assim.)
larger sizes
data content AOD year 2008
MISR
MODIS TERRA
PARASOL
MERIS-ESA
MERIS-BAER
MERIS-ALAMO
ATSR-SU
ATSR ORAC
ATSR ADV
data volume AOD 1/1/2008
MODIS AQUA
MISR
PARASOL
MERIS-ESA
MERIS-BAER
MERIS-ALAMO
ATSR-SU
ATSR ORAC
ATSR ADV
MODIS TERRA
the scoring concept
• establish 1x1 gridded AERONET daily data
– default: use +/- 30min of overpass of satellite
– option: use daily averages
• identify local data pairs to level 3 daily 1x1
satellite retrievals (test data & reference data)
• require at least 10 events to perform a
statistical analysis (e.g. correlation, bias)
the scoring concept
• establish 1x1 gridded AERONET daily data
– default: use +/- 30min of overpass of satellite
– option: use daily averages
• identify local data pairs to level 3 daily 1x1
satellite retrievals (test data & reference data)
• require at least 10 events to perform a
statistical analysis (e.g. correlation, bias)
scoring steps
• establish regional scores (1: best, 0: poor)
– Step1 regional spatial score for each day• (rank) correlation and (rank) bias
– Step2 regional temporal score for each 1x1 • (rank) correlation and (rank) biast
– Step3 combined regional score• total score = bias * spat.corr * temp.corr
– Step4 combine regional to one global score• ‘single score’ from regional total scores
• investigated property: AOD550
MODIS TERRA AOD performance
• total error
– bias
– spatial
– temporal
• bias
– tendency
– albsolute
AOD bias versus AERONET
smaller larger
MODIS TERRA
MISR
PARASOL
MERIS-ESA
MERIS-BAER
MERIS-ALAMO
ATSR-SU
ATSR-ADV
ATSR-ORAC
AOD difference to MODIS
smallerthan MODIS
largerthan MODIS
PARASOL
ATSR-SU
ATSR-ORAC
ATSR-ADV
AOD total error comparison
Increasingly larger ERROR
MODIS TERRA
MISR
PARASOL
MERIS-ESA
MERIS-BAER
MERIS-ALAMO
ATSR-SU
ATSR-ORAC
ATSR-ADV
overall AOD scores
global land ocean• MODIS A/T .63 .59 .66• AATSR S v40 -.60 .63 -.61• AATSR O v202 -.56 -.56 .58• MISR .53 .58 .53• AATSR F v142 -.52 -.56 -.53• AATSR S v31 -.48 -.56 -.47• MERIS B v11 .42 .35 .46• SEAWIFS -.41 -.37• PARASOL -.23 -.25• MERIS ESA 802 -.20 -.22• MERIS A 21 -.20 -.21
based on a10+ sample statistics
absolutelarger scoreis better
+/- sign for overall bias
let us go regional (1)
• boreal– MODIS (.47), MERIS-BAER(.22)
• N.America temp– ATSR-S40 (-.62), ATSR-O202 (.58), ATSR-F142 (-.57),
MISR (.56), MODIS (.44)• S.America trop
– Modis (-.67)• S.America temp
– MODIS (.44), Seawifs (.34), MERIS-BEAR (.31)• N.Africa
– ARSR-S31 (-.72), ATSR-S40(.70), MOD (.67), BEAR (.37)• S.Africa
– MODIS (.63)
10 + events
let us go regional (2)
• EU-Asia boreal– MODIS (-.58)
• EU-Asia temp– MODIS (.63), MISR (-.62), ATSR-S40 (-.58), ATSR-F142
(-.54), ATSR-O202 (-.53), ATSR-S31 (-.52), BAER (-.41)• Asia trop
– Modis (-.62), MERIS-BEAR (.47)• Europe
– MISR (-.63), ATSR-S40 (-.61), ATSR-F142 (-.59), ATSR-O202 (.59), MODIS (.58), ATSR-S31 (-.53), BEAR (.31)
• N.Pacific temp– MODIS (.61), ARSR-F142 (-.54), MISR (.53), BEAR (.47)
• N.Atlantic temperate– MODIS (.61), SEAWIFS (.47), BEAR (.45)
10 + events
what regional scores tell us
• ATSR data coverage is poor compared to MODIS•
• comparisons in regions with sufficient data statistics indicated that all three ATSR AOD data-sets in quality are at/near the level of MODIS/MISR
• current overall ATSR AOD ranking: – 1. SU4.0– 2. O202– 3. F142
• MERIS potentially offers data similar to MODIS
• MERIS BEAR 1.1 demonstrates data-volume processing … but data-quality is relatively poor
overall Angstrom scores
global land ocean• AATSR S v40 .60 .56 .65• AATSR S v31 .52 .47 .57• AATSR F v142 .47 .51 .47• AATSR O v202 .41 .38 .43• MISR .38 .43 .37• MERIS ESA 802 . 27 .28• PARASOL .26 .27• MERIS A 21 .21 .22
all Angstrom data are biased HIGH!land scores are poorer than ocean scores
based on a10+ sample statistics
absolutelarger scoreis better
+/- sign for overall bias
final thoughts
• some of the current CCI retrievals for AOD have
reached the maturity of US products
• while ATSR has become competitive, MERIS
products lag behind
• ocean reference data are needed (MAN data are
too sparse) to statistically evaluate retrieval
quality over oceans, also in order to get reliable
scores for PARASOL and MERIS data
extra slides
further thoughts
• the combination of ‘sufficient’ coverage and of
‘sufficient’ accuracy matters
• temporal correlation (seasonality) is generally
poorer than spatial correlation (patterns)
issues
more solid (statistical) evaluations require …
• better spatial coverage
–how to get references for non AERONET loc ?
• better temporal coverage
–4 months are not enough for limited swath
• better accuracy
–more confidence in AERONET reference
–fewer sites, less variability, several per hour
2ass … minus … 1 ass
improvement deteriation
2 ass … minus … 1 ass
improvement deteriation
O202 … minus … SU40
improvement deteriation
O202 … minus … SU40
improvement deteriation
O202 … minus … SU40
improvement deteriation
SU40 … minus … SU31
improvement deteriation
F142 … minus … SU40
improvement deteriation
extrasthis
this
Thnis
this
Thnis
Thnis
Thnis
MODIS AQUA AOD performance
• total error
– bias
– spatial
– temporal
• bias
– tendency
– absolute
MISR AOD performance
• total error
– bias
– spatial
– temporal
• bias
– tendency
– absolute
POLDER AOD performance
• total error
– bias
– spatial
– temporal
• bias
– tendency
– absolute
MERIS ESA AOD performance
• total error
– bias
– spatial
– temporal
• bias
– tendency
– absolute
MERIS BEAR AOD performance
• total error
– bias
– spatial
– temporal
• bias
– tendency
– absolute
MERIS ALAMO AOD performance
• total error
– bias
– spatial
– temporal
• bias
– tendency
– absolute
AATSR SU40 AOD performance
• total error
– bias
– spatial
– temporal
• bias
– tendency
– absolute
AATSR SU31 AOD performance
• total error
– bias
– spatial
– temporal
• bias
– tendency
– absolute
AATSR O202 AOD performance
• total error
– bias
– spatial
– temporal
• bias
– tendency
– absolute
AATSR O202iq AOD performance
• total error
– bias
– spatial
– temporal
• bias
– tendency
– absolute
effectively ‘identical to O202
AATSR ADV142 AOD performance
• total error
– bias
– spatial
– temporal
• bias
– tendency
– absolute
AATSR SU40 AOD performance
• total error
– bias
– spatial
– temporal
• bias
– tendency
– absolute
MODIS terra
AATSR O202 AOD performance
• total error
– bias
– spatial
– temporal
• bias
– tendency
– absolute
MODIS terra
AATSR ADV142 AOD performance
• total error
– bias
– spatial
– temporal
• bias
– tendency
– absolute
MODIS terra
POLDER AOD performance
• total error
– bias
– spatial
– temporal
• bias
– tendency
– absolute
MODIS terra