Evaluation of precipitation from reanalyses and satellite products in Australia and East Asia

Evaluation of precipitation from reanalyses and satellite products in Australia and East AsiaJorge L. Peña Arancibia, Albert I.J.M. van Dijk, Luigi J. Renzullo and Tim Raupach, all at CSIROMark Mulligan KCL24 April 2012CSIRO LAND AND WATER/WATER FOR A HEALTHY COUNTRY

Gauge + Satellite

Gauge only

12/12/2009

http://eos.csiro.au/

Data and methods NCEP-DOE, JRA-25, ERA-Interim, TRMM 3B42 V6, CMORPH, PERSIANN FD mapped ensemble precipitation SILO (Jeffrey et al., 2001, ENVSOFT),

APHRODITE SEA (Yatagai et al., 2009, SOLA) Detection and agreement metrics for 2003–

2007 (rainfall >1mm d-1).

Global satellite and reanalyses precipitation

EGU 2012 | Evaluation of reanalyses and satellite precipitation data

Six member statistics

FD mapped ensemble

EGU 2012 | Evaluation of reanalyses and satellite precipitation data

iensPprobprob ,* *probPP obsadj

prob* is the probability of the value Pens on day i in the ensemble precipitation time series (Pens), Padj is the empirically adjusted precipitation time series, Pobs is the observed precipitation time series, Pens is the ensemble time series

Detection: occurrence of precipitation

EGU 2012 | Evaluation of reanalyses and satellite precipitation data

How well did the forecast "yes" events correspond to the observed "yes" events(accounting for hits due to chance)?

random

random

hitssfalsealarmmisseshits

hitshitsETS

total

sfalsealarmhitsmisseshitshitsrandom

ETS

Detection: occurrence of precipitation

EGU 2012 | Evaluation of reanalyses and satellite precipitation data

How well did the forecast "yes" events correspond to the observed "yes" events(accounting for hits due to chance)?

ETSETS

Colour codes means best product: red for reanalysis and blue for satellite, ensemble black

Detection: occurrence of precipitation

EGU 2012 | Evaluation of reanalyses and satellite precipitation data

ETS

Range: -1/3 to 1, 0 indicates no skill.   Perfect score: 1.

Agreement: quantity/timing of precipitation

EGU 2012 | Evaluation of reanalyses and satellite precipitation data

Pmonth: difference in the ratio of monthly precipitation amount to total days with precipitation

Correlation RMSD Pmonth

Pmonth

RMSD

Correlation

Agreement: quantity/timing of precipitation

EGU 2012 | Evaluation of reanalyses and satellite precipitation data

Pmonth: difference in the ratio of monthly precipitation amount to total days with precipitation

Correlation RMSD Pmonth

Agreement: quantity/timing of precipitation

EGU 2012 | Evaluation of reanalyses and satellite precipitation data

Pmonth: difference in the ratio of monthly precipitation amount to total days with precipitation

Pmonth

RMSD

Correlation

TS averaged over the entire domain

EGU 2012 | Evaluation of reanalyses and satellite precipitation data

Correlation

ETS

RMSD

Pmonth

Summary

EGU 2012 | Evaluation of reanalyses and satellite precipitation data

              Diff

Rank ETS POD FAR BIAS Correlation RMSD Pmonth (%)

1 0.27 (±0.05) 0.63 (±0.09) 0.43 (±0.07) 0.94 (±0.11) 0.42 (±0.09) 6.94 (±1.5) 1.83 (±11.8)

2 0.27 (±0.05) 0.62 (±0.08) 0.46 (±0.06) 1.06 (±0.23) 0.40 (±0.08) 7.11 (±1.6) -3.7 (±11.3)

3 0.26(±0.06) 0.55 (±0.07) 0.46 (±0.09) 0.93 (±0.21) 0.39 (±0.08) 7.16 (±1.6) -10. (±11.3)

4 0.23 (±0.04) 0.57 (±0.09) 0.47 (±0.07) 1.13 (±0.14) 0.35 (±0.09) 7.72 (±1.7) -11. (±7.21)

5 0.20 (±0.05) 0.44 (±0.07) 0.48 (±0.07) 0.82 (±0.11) 0.31 (±0.05) 8.10 (±1.6) -11. (±9.71)

6 0.20 (±0.06) 0.42 (±0.07) 0.48 (±0.10) 1.21 (±0.25) 0.31 (±0.09) 8.18 (±1.8) 28.0 (±7.99)

7 0.11 (±0.04) 0.34 (±0.07) 0.59 (±0.09) 1.29 (±0.14) 0.15 (±0.05) 10.9 (±2.5) 33.2 (±12.2)

NCEP-DOE JRA-25 ERA-InterimTRMM 3B42 V6 CMORPH PERSIANN Ensemble

POD: probability of detection (0-1) perfect score 1FAR: false alarm ratio (0-1) perfect score 0BIAS: bias in over (>1) or underdetecting (<1) perfect score 1

Summary

EGU 2012 | Evaluation of reanalyses and satellite precipitation data

              Diff

Rank ETS POD FAR BIAS Correlation RMSD Pmonth (%)

1 0.27 (±0.05) 0.63 (±0.09) 0.43 (±0.07) 0.94 (±0.11) 0.42 (±0.09) 6.94 (±1.5) 1.83 (±11.8)

2 0.27 (±0.05) 0.62 (±0.08) 0.46 (±0.06) 1.06 (±0.23) 0.40 (±0.08) 7.11 (±1.6) -3.7 (±11.3)

3 0.26(±0.06) 0.55 (±0.07) 0.46 (±0.09) 0.93 (±0.21) 0.39 (±0.08) 7.16 (±1.6) -10. (±11.3)

4 0.23 (±0.04) 0.57 (±0.09) 0.47 (±0.07) 1.13 (±0.14) 0.35 (±0.09) 7.72 (±1.7) -11. (±7.21)

5 0.20 (±0.05) 0.44 (±0.07) 0.48 (±0.07) 0.82 (±0.11) 0.31 (±0.05) 8.10 (±1.6) -11. (±9.71)

6 0.20 (±0.06) 0.42 (±0.07) 0.48 (±0.10) 1.21 (±0.25) 0.31 (±0.09) 8.18 (±1.8) 28.0 (±7.99)

7 0.11 (±0.04) 0.34 (±0.07) 0.59 (±0.09) 1.29 (±0.14) 0.15 (±0.05) 10.9 (±2.5) 33.2 (±12.2)

NCEP-DOE JRA-25 ERA-InterimTRMM 3B42 V6 CMORPH PERSIANN Ensemble

and the winner is…ensemble!!!

Summary

EGU 2012 | Evaluation of reanalyses and satellite precipitation data

              Diff

Rank ETS POD FAR BIAS Correlation RMSD Pmonth (%)

1 0.27 (±0.05) 0.63 (±0.09) 0.43 (±0.07) 0.94 (±0.11) 0.42 (±0.09) 6.94 (±1.5) 1.83 (±11.8)

2 0.27 (±0.05) 0.62 (±0.08) 0.46 (±0.06) 1.06 (±0.23) 0.40 (±0.08) 7.11 (±1.6) -3.7 (±11.3)

3 0.26(±0.06) 0.55 (±0.07) 0.46 (±0.09) 0.93 (±0.21) 0.39 (±0.08) 7.16 (±1.6) -10. (±11.3)

4 0.23 (±0.04) 0.57 (±0.09) 0.47 (±0.07) 1.13 (±0.14) 0.35 (±0.09) 7.72 (±1.7) -11. (±7.21)

5 0.20 (±0.05) 0.44 (±0.07) 0.48 (±0.07) 0.82 (±0.11) 0.31 (±0.05) 8.10 (±1.6) -11. (±9.71)

6 0.20 (±0.06) 0.42 (±0.07) 0.48 (±0.10) 1.21 (±0.25) 0.31 (±0.09) 8.18 (±1.8) 28.0 (±7.99)

7 0.11 (±0.04) 0.34 (±0.07) 0.59 (±0.09) 1.29 (±0.14) 0.15 (±0.05) 10.9 (±2.5) 33.2 (±12.2)

NCEP-DOE JRA-25 ERA-InterimTRMM 3B42 V6 CMORPH PERSIANN Ensemble

and the winner is…ensemble!!! But not a fair comparison!!!

Similar findings to other studies in different geographical domains (e.g. Ebert et al., 2007 BAMS)

Other sources of error Enhancement of metrics when

producing ensembles Importance of ground monitoring

networks, particularly were satellite/reanalysis algorithms need refinement and evaluation

More confidence in areas were ground monitoring is scarce

HS1 A276 poster Tuesday 24

Conclusion

EGU 2012 | Evaluation of reanalyses and satellite precipitation data

EGU 2012 | Evaluation of reanalyses and satellite precipitation data

http://eos.csiro.au/

HS1 A276 poster

Thank youCSIRO Land and WaterJorge Luis Peña Arancibia

t +61 2 6246 5711e jorge.penaarancibia@csiro.auw www.csiro.au

CSIRO LAND AND WATER