Upload
sheba
View
32
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Using FLUXNET data to evaluate land surface models. Ray Leuning and Gab Abramowitz 4 – 6 June 2008. Land surface model evaluation framework. Reto Stockli’s ‘Model farm’. Schematic diagram of model components from a systems perspective. system boundary, B inputs, u initial states, x 0 - PowerPoint PPT Presentation
Citation preview
Using FLUXNET data to evaluate land surface models
Ray Leuning and Gab Abramowitz
4 – 6 June 2008
CSIRO. Using FLUXNET data to evaluate land surface models
Land surface model evaluation framework
Reto Stockli’s ‘Model farm’
CSIRO. Using FLUXNET data to evaluate land surface models
Schematic diagram of model components from asystems perspective
Liu, Y. Q. and Gupta, H. V. (2007). Uncertainty in Hydrologic Modeling: Toward an Integrated Data Assimilation Framework. Water Resources Research 43, W07401, doi:10.1029/2006/WR005756.
1. system boundary, B2. inputs, u3. initial states, x04. parameters, θ5. model structure, M6. model states, x7. outputs, y
Errors in each component affects model performance
CSIRO. Using FLUXNET data to evaluate land surface models
Parameter estimation Multiple objective functions possible
CSIRO. Using FLUXNET data to evaluate land surface models
Parameter estimation Multiple criteria possible, e.g. λE, NEE
The dark line between the two criteria’s minima, α and β, represents the Pareto set
CSIRO. Using FLUXNET data to evaluate land surface models
Comparing RMSE of models of varying complexity across sites after parameter optimization
Hogue, T. S., Bastidas, L. A., Gupta, H. V., and Sorooshian, S. (2006). Evaluating Model Performance and Parameter Behavior for Varying Levels of Land Surface Model Complexity. Water Resources Research 42, W08430, doi:10.1029/2005WR004440.
Models
Sites
Ideal result (0,0)
λE
H
CSIRO. Using FLUXNET data to evaluate land surface models
SOLO neural network - cluster analysis
Abramowitz, G., Gupta, H., Pitman, A., Wang, Y.P., Leuning, R. and Cleugh, H.A. (2006). Neural Error Regression Diagnosis (NERD): A tool for model bias identification and prognostic data assimilation. Journal of Hydrometeorology, 7:160-177.
CSIRO. Using FLUXNET data to evaluate land surface models
Poor model performance not just due to poor parameter estimation
CABLE with 4 different parameter sets
SOLO – cluster analysis
observedcablesolo
CSIRO. Using FLUXNET data to evaluate land surface models
No model or single performance measure is best for all fluxes
NEE of CO2 (µmol/m2/s) Latent heat flux (W/m2) Sensible heat flux (W/m2)
mean (obs)
rmse grad int rsq mean (obs)
rmse grad
int rsq mean (obs)
rmse grad
int rsq
CAB -2.3 (-0.1) 5.48 0.55 -2.2 0.52 49 (52) 55.9 0.71 12.0 0.61 38 (54) 58.3 0.79 -5.0 0.82
ORC -0.5 (-0.1) 5.16 0.47 -0.4 0.50 46 (52) 58.1 0.73 8.1 0.59 14 (54) 118.1 0.16 6.0 0.78
CLM -1.2 (-0.1) 6.13 0.24 -1.6 0.35 26 (52) 72.9 0.38 6.5 0.40 56 (54) 84.5 0.83 10.9 0.64
MLR -1.5 (-0.1) 4.40 0.55 -1.4 0.70 45 (52) 47.6 0.59 14.5 0.74 43 (54) 54.9 0.69 5.2 0.87
T u m b a r u
ANN -1.3 (-0.1) 4.65 0.52 -1.2 0.64 42 (52) 49.9 0.54 14.1 0.74 41 (54) 51.4 0.81 -3.0 0.86
CAB -1.7 (-0.9) 6.13 0.61 -1.2 0.49 43 (47) 64.5 0.89 1.8 0.55 -0.1 (26) 71.8 0.62 -16.1 0.32
ORC -0.4 (-0.9) 9.07 0.68 0.2 0.29 30 (47) 48.0 0.84 -9.4 0.70 3.2 (26) 52.2 0.69 -14.5 0.56
CLM -1.2 (-0.9) 7.92 0.12 -1.1 0.08 33 (47) 50.5 0.63 3.0 0.62 30 (26) 57.9 1.01 4.3 0.59
MLR -0.7 (-0.9) 7.38 0.21 -0.5 0.20 38 (47) 45.8 0.53 13.6 0.73 26 (26) 47.7 1.08 -2.0 0.71
B o n d v i l
ANN 0.0 (-0.9) 7.69 0.20 0.2 0.15 39 (47) 45.7 0.53 14.4 0.72 27 (26) 49.0 1.14 -2.2 0.73
CABLE, ORCHIDEE, CLM,
MLR multiple linear regression,
ANN artificial neural network
CSIRO. Using FLUXNET data to evaluate land surface models
Model comparisons - average seasonal cycle
NEE
λE
H
Global default parameters for each PFT used
CSIRO. Using FLUXNET data to evaluate land surface models
Model comparisons - average daily cycle
NEE
λE
H
Global default parameters for each PFT used
CSIRO. Using FLUXNET data to evaluate land surface models
PDF’s for NEE, λE & H across 6 sites
CSIRO. Using FLUXNET data to evaluate land surface models
NEE Perturbed-parameter ensemble simulations
Monthly averages Average diurnal cycle
CSIRO. Using FLUXNET data to evaluate land surface models
λE Perturbed-parameter ensemble simulations
Monthly averages Average diurnal cycle
CSIRO. Using FLUXNET data to evaluate land surface models
H Perturbed-parameter ensemble simulations
Monthly averages Average diurnal cycle
CSIRO. Using FLUXNET data to evaluate land surface models
Partitioning climate space into 9 SOM nodes
S↓ Tair qair
night
night
night
S↓ Tair qairS↓ Tair qair
CSIRO. Using FLUXNET data to evaluate land surface models
NEE PDFs at nodes 7 -9 at Tumbarumba
night
S↓ Tair qair S↓ Tair qair S↓ Tair qair
7 8 9
CSIRO. Using FLUXNET data to evaluate land surface models
Suggested set of discussion topics
• Primary objectives• Establish a framework that provides standardised data sets
and an agreed set of analytical tools for LSM evaluation• Analytical tools should provide a wide range of diagnostic
information about LSM performance
• Datasets specifically formatted for LSM execution and evaluation
• Specific objectives• To detect and eliminate systematic biases in several LSMs in
current use• To obtain optimal parameter values for LSMs after biases have
been diminished or eliminated• To evaluate the correlation between key model parameters and
bioclimatic space
CSIRO. Using FLUXNET data to evaluate land surface models
Tasks for meeting 1
• Discuss what form the LSM evaluation framework should take• PILPS style?• What will be asked of data providers?• What will be asked of LS modellers?
• Agree on a minimal set of LSM flux performance measures (model vs observations vs benchmark):
• Average diurnal cycle?• Average annual cycle (monthly means)?• Some type of frequency analysis (wavelet, power spectrum etc)?• Conditional analysis (SOM node analysis):
• Overlap of pdfs
• Multiple criteria cost function set (mean, rmse, rsq, regression gradient and intercept)
• Discuss other LSM outputs and datasets useful for process evaluation
• Discuss ways to include parameter uncertainty in LSM evaluation (c.f. Abramowitz et al., 2008)
CSIRO. Using FLUXNET data to evaluate land surface models
Tasks for meeting 2
• Discuss options for the most effective way to provide these services
• Will individual groups do benchmarking, evaluation of model states?
• Preference for an automated web-based interface and data server• Automatic processing through a website?• Abramowitz suggests automation of basic LSM performance measure
plots, including benchmarking (as in Abramowitz, 2005). • Uploaded output from LSM runs in ALMA format netcdf could return
standard plots to the user and/or post on website.
• Model detective work and improvement to be done by individual groups
CSIRO. Using FLUXNET data to evaluate land surface models
Data analysis will use:
• Several current LSMs• Quality controlled Fluxnet datasets• SOFM (Self-organizing feature maps) analysis
• to classify bioclimatic data into n2 nodes
• to evaluate model biases for each node to help the ‘detective work’ of identifying areas of model weaknesses
• to identify upper-boundary surfaces for stocks of C and N and P in global ecosystems as a function of the n2 climate nodes
• Benchmarking • to compare model predictions at each climate node against
multiple linear regression (MLR) estimates
CSIRO. Using FLUXNET data to evaluate land surface models
Tools currently available from Abramowitz
• SOLO (SOFM + MLR) software (Fortran)• LSMs ‘Model Farm’ of Reto Stöckli plus CABLE• CSV to ALMA netcdf conversion routine (Fortran)• Plotting routines in R• Fluxnet database in CSV and netcdf formats