Upload
tamira
View
26
Download
0
Embed Size (px)
DESCRIPTION
Sensitivity and Importance Analysis. Charles Yoe [email protected]. Sensitivity Analysis Defined. - PowerPoint PPT Presentation
Citation preview
“ Building Strong “
Delivering Integrated, Sustainable, Water Resources Solutions
Sensitivity and Importance Analysis
Charles [email protected]
“ Building Strong “
Delivering Integrated, Sustainable, Water Resources Solutions
Sensitivity Analysis Defined
• Study of how the variation in the output of a model can be apportioned, qualitatively or quantitatively, to different “sources of variation” in the inputs for the purpose of increasing confidence in the analysis– Include assumptions– Input uncertainty– Scenario/model uncertainty
“ Building Strong “
Delivering Integrated, Sustainable, Water Resources Solutions
The Point
• Complex analysis may have dozens of input and output variables that are linked by a system of equations
• Analysts and decision makers must understand the relative importance of the components of an analysis
• Some outcomes and decisions are sensitive to minor changes in assumptions and input values
“ Building Strong “
Delivering Integrated, Sustainable, Water Resources Solutions
Sensitivity Analysis
• If it is not obvious which assumptions and uncertainties most affect outputs, conclusions and decisions the purpose of sensitivity analysis is to systematically find this out
“ Building Strong “
Delivering Integrated, Sustainable, Water Resources Solutions
Systematic Investigation of…
• Future scenarios
• Model parameters
• Model inputs
• Assumptions
• Model functional form
“ Building Strong “
Delivering Integrated, Sustainable, Water Resources Solutions
Assumptions Sensitivity
• List the key assumptions (scenarios) of your analysis
• Explore what happens as you change/drop each one individually– Do your answers change?
• Challenging assumptions can be effective sensitivity analysis
“ Building Strong “
Delivering Integrated, Sustainable, Water Resources Solutions
Input Sensitivity
• Parameter-how sensitive is our output to forecast error or other changes in inputs? Unexpected change or error
• Decision variables (Inputs we control)-might changes in our decisions/actions improve our outputs
“ Building Strong “
Delivering Integrated, Sustainable, Water Resources Solutions
Sensitivity Analysis Methods
• Deterministic one-at-a-time analysis of each factor
• Deterministic joint analysis• Scenario analysis• Subjective estimates• Parametric analysis--range of values• Probabilistic analysis can be used for
importance analysis
“ Building Strong “
Delivering Integrated, Sustainable, Water Resources Solutions
One-At-A-Time Analysis
• Hold each parameter constant– Expected value– Representative value
• Let one input vary– Assumption– Input– Parameter
• Common, useful, dangerous
“ Building Strong “
Delivering Integrated, Sustainable, Water Resources Solutions
One-At-A-Time Analysis
• Do not equate magnitude with influence• A=U(107,108), B=U(2,6)• C = A + B; A dominates• C = AB; B dominates
“ Building Strong “
Delivering Integrated, Sustainable, Water Resources Solutions
One-At-A-Time Analysis
• Dependence and branching in model creates flaws with this logic
If A<50 then
C = B + 1
Else
C = B100
What value do we set A equal to?
“ Building Strong “
Delivering Integrated, Sustainable, Water Resources Solutions
Joint Analysis
• Change combinations of variables at same time
• Enables analysts to take dependencies explicitly into account
• Can have same limitations as OAAT analysis
“ Building Strong “
Delivering Integrated, Sustainable, Water Resources Solutions
Subjective Estimates
• Subjective estimates of uncertain values can be used to identify threshold values of importance to the risk assessment
“ Building Strong “
Delivering Integrated, Sustainable, Water Resources Solutions
Range of Values
• A specific (not subjective) range of values is used– E.g., 10th, 50th, 90th percentiles
• Ceteris paribus approach• All possible combinations approach
– All 10th percentiles, 10th with 90th and so on
“ Building Strong “
Delivering Integrated, Sustainable, Water Resources Solutions
Importance Analysis
• How much does each model input contribute to the variation in the output?
• Typically a few key inputs account for most output variation– These are your important inputs.
• Not particularly good at identifying nonlinear or multivariate relationships
“ Building Strong “
Delivering Integrated, Sustainable, Water Resources Solutions
Habitat Units CreatedP
rob
ab
ility
HUs
0.000
0.005
0.010
0.015
0.020
0.025
0.030
0.035
0.040
30 60 90 120
“ Building Strong “
Delivering Integrated, Sustainable, Water Resources Solutions
Regression Sensitivity for Grand Total HUs/D14
Std b Coefficients
B=>15 Degrees C / Use this.../L10 .071
V5: water temperature / Me.../H9 .073
V6: Dissolved oxygen / Mea.../N22-.075
V15: Pool class / Use this.../O26-.082
V6: Dissolved oxygen / Mea.../H22-.105
B=>15 Degrees C / Use this.../O10-.106
V15: Pool class / Use this.../H26-.117
V5: water temperature / Me.../N9 .147
V6: Dissolved oxygen / Mea.../B22-.157
V5: water temperature / Me.../E9 .16
V5: water temperature / Me.../K9 .162
A=resident rainbow trout /.../X6-.163
B=>15 Degrees C / Use this.../U10-.194
B=>15 Degrees C / Use this.../E10 .22
A=resident rainbow trout /.../L6-.287
A=resident rainbow trout /.../R6-.45
-1 -0.75 -0.5 -0.25 0 0.25 0.5 0.75 1
“ Building Strong “
Delivering Integrated, Sustainable, Water Resources Solutions
“ Building Strong “
Delivering Integrated, Sustainable, Water Resources Solutions
Advanced Statistical Methods
• Apportion variation in output to inputs via– Regression analysis– Analysis of variance– Response surface methods– Fourier amplitude sensitivity test (FAST)– Mutual information index (MII)– Classification and regression trees (CART)
“ Building Strong “
Delivering Integrated, Sustainable, Water Resources Solutions
So What?
• When decision is sensitive to changes or uncertainties within realm of possibility then more precision and additional information may be required– More data (research)– Better models– Conservative risk management
“ Building Strong “
Delivering Integrated, Sustainable, Water Resources Solutions
Take Away Points
• “What if” analysis is essential to good risk assessment
• Systematic investigations of model parameters, model inputs, assumptions, model functional form
• Essential to good risk management
“ Building Strong “
Delivering Integrated, Sustainable, Water Resources Solutions
Charles Yoe, [email protected]
Questions?