A general first-order global sensitivity analysis method Chonggang Xu, George Z. Gertner* Department of Natural Resources and Environmental Sciences, University

A general first-order global sensitivity analysis method

Chonggang Xu, George Z. Gertner*

Department of Natural Resources and Environmental Sciences,University of Illinois at Urbana-ChampaignUSA

Uncertainty & Sensitivity analysis Techniques

Design of experiments method Sampling-based method Fourier Amplitude Sensitivity Test (FAST) Sobol’s method ANOVA method Moment independent approaches

FAST Search function:

Fourier transformation:

Variance decomposition:

-1 1 1( arcsin(sin( s))) s

2i i ix F

1 2 n( , ,..., ) ( )Y f x x x f s

01

{ cos( ) sin( )}k kk

Y A A ks B ks

0 -

-

-

1( )d

21

( )cos( )d

1( )sin( )d

k

k

A f s s

A f s ks s

B f s ks s

2 2k k k

1( )

2A B p

piiV

i characteristic frequency parameter i

Sum spectrum

FAST advantages & limitations

Computationally efficient global sensitivity analysis method;

Suitable for nonlinear and non-monotonic models;

Aliasing effects for small sample sizes(frequency interference) ;

Suitability for only models with independent parameters;

Real applications

Dependence among parameters;

Complex model with many parameters, which needs much computation times and large sample sizes in traditional FAST;

FAST improvement Reorder

independent parameters limitation;

Random Balance Design (Tarantola et al. 2006) commom frequency for all parameters

and permuting

Overcome aliasing effect limitation;

Tarantola, S, Gatelli, D, Mara, TA. Random balance designs for the estimation of first order global sensitivity indices. Reliability Engineering and System Safety, 2006; 91(6): 717-727.

1i

1 1arcsin(sin s s

2

i(x = F ( ( ))) )

1 2{ , ,..., ,..., } j NS s s s s

for each parameter.

↑

Details of synthesized FAST

Xu, C. and G.Z. Gertner 2007. A general first-order global sensitivity analysis method. Reliability Engineering and System Safety (In press).

Reorder for correlation: Model Y = x1+ x2, correlation=0.7, characteristic frequency is 5 and 23

respectively.

Independent sample order Correlated sample order of X1

0

0.5

1

1.5

2

0 100 200 300 400 500

y

00.10.20.30.40.50.60.70.80.9

1

1 101 201 301 401

x1

x2

00.10.20.30.40.50.60.70.80.9

1

1 81 161 241 321 401

x1

x2

Response Variable Y

Reorder for correlation: Model Y = x1+ x2,

correlation=0.7,common characteristic frequency is 5 for both x1 and x2

Independent sample order Correlated sample order of X1

00.10.20.30.40.50.60.70.80.9

1

1 101 201 301 401

x1

x2

00.10.20.30.40.50.60.70.80.9

1

1 81 161 241 321 401

x1

x2

0

0.5

1

1.5

2

0 100 200 300 400 500

yResponse Variable Y

00.10.20.30.40.50.60.70.80.9

1

1 81 161 241 321 401

x1

x2

correlation=0.0

00.10.20.30.40.50.60.70.80.9

1

1 81 161 241 321 401

x1

x2

correlation=0.2

00.10.20.30.40.50.60.70.80.9

1

1 81 161 241 321 401

x1

x2

correlation=0.5

00.10.20.30.40.50.60.70.80.9

1

1 81 161 241 321 401

x1

x2

correlation=0.9

FAST sample with common characteristic

frequency

Reordered sample

Model outputs

based on reordered

sample

Sample for FAST

analysis of x1

Sample for FAST

analysis of x2

Sample for the common variable s

1)

2)

3)

4)

5)

5)

6)

6)

x1 x2 order

0.72 0.72 1

0.83 0.83 2

0.39 0.39 3

0.06 0.06 4

0.50 0.50 5

0.94 0.94 6

0.61 0.61 7

0.17 0.17 8

0.28 0.28 9x1 x2 order1 order2

0.39 0.61 3 70.72 0.28 1 90.50 0.94 5 60.17 0.17 8 80.61 0.72 7 10.28 0.50 9 50.06 0.06 4 40.94 0.83 6 20.83 0.39 2 3

x1 x2 order1 order2 y0.39 0.61 3 7 1.000.72 0.28 1 9 1.000.50 0.94 5 6 1.440.17 0.17 8 8 0.330.61 0.72 7 1 1.330.28 0.50 9 5 0.780.06 0.06 4 4 0.110.94 0.83 6 2 1.780.83 0.39 2 3 1.22

x1 x2 order1 order2 y0.61 0.72 7 1 1.330.94 0.83 6 2 1.780.83 0.39 2 3 1.220.06 0.06 4 4 0.110.28 0.50 9 5 0.780.50 0.94 5 6 1.440.39 0.61 3 7 1.000.17 0.17 8 8 0.330.72 0.28 1 9 1.00

x1 x2 order1 order2 y0.72 0.28 1 9 1.000.83 0.39 2 3 1.220.39 0.61 3 7 1.000.06 0.06 4 4 0.110.50 0.94 5 6 1.440.94 0.83 6 2 1.780.61 0.72 7 1 1.330.17 0.17 8 8 0.330.28 0.50 9 5 0.78

s

-2.79

-2.09

-1.40

-0.70

0.00

0.70

1.40

2.09

2.79

Maximum Harmonic order selection

Frequency

0 50 100 150 200 250

Am

plit

ude

0.00

0.05

0.10

0.15

0.20

0.25

Frequency

0 50 100 150 200 250

Am

plit

ude

0.000

0.005

0.010

0.015

0.020

pp

iiV Scaled characteristic spectrum

1. Random balance design may introduce random error.2. Assume that the low characteristic amplitudes at high harmonic order are more susceptible to the random error than relatively high characteristic amplitudes at a low harmonic order .

Simulated annealing refinement for correlated samples

Sample size

0 200 400 600 800 1000 1200

Dev

ianc

e

0.00

0.02

0.04

0.06

0.08

0.10

0.12

0.14

BoxplotAverage0.02 reference

Temperature PAR Precipitation CO2

Temperature 1.00 0.70 0.24 0.41

PAR 0.70 1.00 0.27 0.07

Precipitation 0.24 0.27 1.00 0.05

CO2 0.41 0.07 0.05 1.00

0

0.01

0.02

0.03

0.04

0.05

0.06

0.07

0.08

1 11 21 31 41 51 61 71 81 91 101

Iterations

Dev

ianc

e

PAR is photosynthetic active radiation

TEST CASES

Synthesized FAST specification for test cases

Model Characteristic Frequency

Sample Size Maximum harmonic order

Test case one 23 921 14

Test case two 23 4614 for x1-x4

2 for others

Test case three 23 4614 for x1-x3

2 for others

Test case four 23 4614 for x2-x5

2 for others

Test case one:

Parameter

x1 x2

Sen

siti

vity

0.0

0.2

0.4

0.6

0.8

1.0

Analytical sensitivity index of x1

Analytical sensitivity index of x2

SFAST 10 replicationsSFAST 50 replicationsSFAST 100 replications

Y=2x1+3x2 , where x1 and x2 are standard normally distributed with a Pearson correlation coefficient of 0.7

SFAST is synthesized FASTCircles are analytical

Test case two: (Lu and Mohanty, 2001)

Parameter

x1 x2 x3 x4 x5 x6 x7 x8 x9 x10

Sen

siti

vity

0.00

0.05

0.10

0.15

0.20

0.25

0.30

0.35


103 2

1

y ( ) 50( ) ,i i i 1 2 3i

a x x x x x

1 2 3 4 5

6 7 8 9 10

a = 100, a = 80, a = 60, a = 40, a = 20,

a = 0.1, a = 0.08, a = 0.06, a = 0.02, a = 0.01

1 0 0 0 0 0 0 0 0 0

0 1 0.3 0.7 0 0 0 0 0 0

0 0.3 1 0.4 0 0 0 0 0 0

0 0.7 0.4 1 0 0 0 0 0 0

0 0 0 0 1 0 0 0 0 0

0 0 0 0 0 1 0 0 0 0

0 0 0 0 0 0 1 0.4 0 0

0 0 0 0 0 0 0.4 1 0 0

0 0 0 0 0 0 0 0 1 0

0 0 0 0 0 0 0 0 0 1

Circles are based on correlation ratio method by Saltelli(2001) based McKay’s one-way ANOVA. Nonparametric method suitable for nonlinear and monotonic models. 50,000 model runs (=100 replications x 500 samples)

Rank correlation

Test case three:

Parameter

x1 x2 x3 x4 x5 x6 x7 x8

Sen

siti

vity

0.0

0.2

0.4

0.6

0.8


1

( )n

i ii

f g x

4 2

( ) for 0 1 and 01i i

i i i ii

x ag x x a

a

1 0 0.6 0 0 0 0 0

0 1 0 0 0.7 0 0 0

0.6 0 1 0 0 0 0 0

0 0 0 1 0 0 0 0

0 0.7 0 0 1 0 0 0

0 0 0 0 0 1 0.4 0

0 0 0 0 0 0.4 1 0

0 0 0 0 0 0 0 1

(G-function of Sobol’. Non-monotonic test model)

Rank correlation

Circles are based on correlation ratio method . 50,000 model runs (=100 replications x 500 samples)

Test case four: World 3 Model(Meadows et al., 1992)

IndustrialCapital

fraction of industrial outputallocated to investment

industrial output

initial industrial capital

average life of industrial capital

industrial capitaldepreciation

industrial capitalinvestment

average life of industrial capital 1

average life of industrial capital 2

<POLICY YEAR>

<industrial capital output ratiomultiplier from resource

conservation technology>

<fraction of industrial outputallocated to agriculture>

fraction of industrial outputallocated to consumption

<fraction of industrial outputallocated to services>

<capacity utilization fraction><fraction of industrial

capital allocated toobtaining resources>

industrial capital output ratio

industrial capital output ratio 1

industrial capital output ratio 2

<Time>

<industrial capital outputratio multiplier from pollution

technology>

<industrial capital outputratio multiplier from land yield

technology>

fraction of industrialoutput allocated to

consumption constant

fraction of industrialoutput allocated to

consumption variable

industrialequilibrium time

<Time>fraction of industrial output allocated

to consumption constant 1

fraction of industrial output allocatedto consumption constant 2

industrialoutput per

capita desired

industrial outputper capita

fraction ofindustrial output

allocated toconsumptionvariable table

<population>

<POLICY YEAR>

<POLICY YEAR>

<Time>

Parameter uncertainty specification

Parameter Label Lower bound Upper bound

x1 industrial output per capita desired 315 385

x2 industrial capital output ratio before 1995 2.7 3.3

x3fraction of industrial output allocated to

consumption before 19950.387 0.473

X4fraction of industrial output allocated to

consumption after 19950.387 0.473

X5average life of industrial capital before

199512.6 15.4

X6average life of industrial capital after

199516.2 19.8

x7 initial industrial capital 1.89(10+11) 2.31(10+11)

Rank correlation of .6 between x3 and x4 and .4 between x5 and x6

Year

1900 1950 2000 2050 2100

Sen

siti

vity

0.00

0.02

0.04

0.06

0.08

0.10

0.12

0.14

0.16

0.18

SFASTCRM

Year

1900 1950 2000 2050 2100

Sen

siti

vity

0.00

0.02

0.04

0.06

0.08

0.10

0.12

0.14

0.16

0.18

0.20

0.22

SFASTCRM

Year

1900 1950 2000 2050 2100

Sen

siti

vity

0.1

0.2

0.3

0.4

0.5

0.6

SFASTCRM

Year

1900 1950 2000 2050 2100

Sen

siti

vity

0.0

0.1

0.2

0.3

0.4

0.5

SFASTCRM

x2 x3

x4 x5

Year

Sen

sitiv

ity

(Correlation ratio method (CRM). 50,000 model runs (=100 replications x 500 samples)

Year

1900 1950 2000 2050 2100

Sens

ivity

0.0

0.1

0.2

0.3

0.4

0.5

0.6

x2 x3 x4x5

Assume Independence

Year

1900 1950 2000 2050 2100

Sen

siti

vity

0.1

0.2

0.3

0.4

0.5

0.6

SFASTCRM

Year

1900 1950 2000 2050 2100

Sen

siti

vity

0.00

0.02

0.04

0.06

0.08

0.10

0.12

0.14

0.16

0.18

0.20

0.22

SFASTCRM

Assume correlation

3 x

4 x

Application: Uncertainty in forest landscape response to global warming

PnET-IITemperature

Precipitation

ANPP

SEP

LANDIS-IILandscape

composition

CO2

PnET-II is a forest ecosystem process model (LINKAGES) is a forest GAP modelLANDIS is a spatially dynamic forest landscape model)

Month

1 2 3 4 5 6 7 8 9 10 11 12

Tem

pera

ture

(o C

)

-20

-10

0

10

20

30

1984-1993 2090-2099

Year

1960 1980 2000 2020 2040 2060 2080 2100 2120

Pre

cipi

tati

on (

cm)

50

60

70

80

90

100

110

Year

1960 1980 2000 2020 2040 2060 2080 2100 2120

CO

2 C

once

ntra

tion

(pp

m)

300

400

500

600

700

800

Year

1960 1980 2000 2020 2040 2060 2080 2100 2120

Tem

pera

ture

(o C

)

0

2

4

6

8

10

(c) (d)

(b)(a)

Example of data after 1994 is based on prediction by Canadian Climate Center (CCC) in the Phase-II Vegetation-Ecosystem Modeling and Analysis Project (VEMAP). Data before 1994 is historical data. We assume the climate stabilizes after year 2099.

Parameter uncertainty Based on 27 predicted climate data structure from the

Intergovernmental Panel on Climate Change (IPCC) Third and Fourth Assessment Report , and the Phase-II Vegetation-Ecosystem Modeling and

Analysis Project (VEMAP) Variable N Mean Std Dev Median Minimum Maximum

Temperature 27 7.43576 2.36065 7.45714 3.97337 12.58936

PAR 27 571.26298 32.19750 566.47531 522.93955 633.86171

Precipitation 27 92.53673 6.93255 92.73039 79.19724 105.41716

CO2 27 689.34074 111.10015 699.40000 546.80000 923.25000


Rank correlation structure

Spearman Correlation Coefficients, N = 27 Prob > |r| under H0: Rho=0

Temperature PAR Precipitation CO2

Temperature 1.00000 0.69780<.0001

0.238710.2305

0.409340.0340

PAR 0.69780<.0001

1.00000 0.268620.1755

0.069630.7300

Precipitation 0.238710.2305

0.268620.1755

1.00000 0.054950.7854

CO2 0.409340.0340

0.069630.7300

0.054950.7854

1.00000


Uncertainty

0

0.01

0.02

0.03

0.04

0.05

0.06

0.07

0.08

0.09

0.1

0 100 200 300 400

0

0.1

0.2

0.3

0.4

0.5

0.6

0 100 200 300 400

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 100 200 300 4000

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 100 200 300 400

Spruce-firPine

Aspen-birch Maple-ash

Simulation year

Unc

erta

inty

Sensitivity

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 100 200 300 400

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 100 200 300 400

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 100 200 300 400

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 100 200 300 400

Spruce-firPine

Aspen-birch Maple-ash

Simulation year

Sen

sitiv

ity

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 200 400

Temperature

PAR

Precipitation

CO2

Conclusion Proposes a general first-order global sensitivity approach

for linear/nonlinear models with as many correlated or uncorrelated parameters as the user specifies;

FAST is computationally efficient and would be a good choice for uncertainty and sensitivity analysis for models with correlated parameters;

Conclusion Proposes a general first-order global sensitivity approach

for linear/nonlinear models with as many correlated or uncorrelated parameters as the user specifies;

FAST is computationally efficient and would be a good choice for uncertainty and sensitivity analysis for models with correlated parameters;

Thank You!

Documents

A general first-order global sensitivity analysis method Chonggang Xu, George Z. Gertner* Department of Natural Resources and Environmental Sciences, University