The Global Institute for Water Security

Saman Razavi & Amin Haghnegahdar

𝑒𝑒 = �𝒚𝒚 − 𝒚𝒚

𝒙𝒙 𝒚𝒚

Parametersp1 p2 p3 p4

(a) Simulated Processes

(c) Discretization Schemes

(b) ConceptualizationSchemesM

odel

Str

uctu

re s1

s2

s3

�𝒚𝒚 = 𝑓𝑓 𝒑𝒑, 𝒔𝒔,𝒙𝒙,𝑔𝑔(… )Model:

Anthropogenic FactorsWater/Land Management

Example: Nottawasaga River BasinMESH Model with 40 Parameters

Forcing Variables0

20

40Precipitation (mm)

0

25

50

0

50

100

0

500

1000

0

25

50

0

15

30

0

0.2

0.4

Liquid water stored in soil layer 1 (mm)

Liquid water stored in soil layer 2 (mm)

Liquid water stored in soil layer 3 (mm)

Frozen water stored in soil layer 1 (mm)

Frozen water stored in soil layer 2 (mm)

Frozen water stored in soil layer 3 (mm)

Sta

te V

aria

bles

0

3

60

3

6Evapotranspiration (mm)

Total Runoff (cms)

Sep 1, 2004 Sep 30, 2007Res

pons

eV

aria

bles

10.8

0.60.4

0.200

0.2

0.4

0.6

0.8

0.74

0.73

0.72

0.71

0.7

0.69

0.68

0.75

1N

S-lo

g1

0.80.6

0.40.2

00

0.2

0.4

0.6

0.8

0.68

0.7

0.72

0.74

0.76

1

NS

10.8

0.60.4

0.200

0.2

0.4

0.6

0.8

0.64

0.66

0.68

0.7

0.72

1

NS

(4/16)

1

0.8

0.6

0.4

0.2

00

0.2

0.4

0.6

0.8

10

8

6

4

2

0

1

BIAS

10.8

0.60.4

0.200

0.2

0.4

0.6

0.8

0.74

0.73

0.72

0.71

0.7

0.69

0.68

0.75

1N

S-lo

g1

0.80.6

0.40.2

00

0.2

0.4

0.6

0.8

0.68

0.7

0.72

0.74

0.76

1

NS

10.8

0.60.4

0.200

0.2

0.4

0.6

0.8

0.64

0.66

0.68

0.7

0.72

1

NS

(4/16)

1

0.8

0.6

0.4

0.2

00

0.2

0.4

0.6

0.8

10

8

6

4

2

0

1

BIAS

0.8

0.78

0.76

0.74

0.72

0.70.2

0.22

0.24

0.26

0.28

0.7205

0.72

0.7195

0.719

0.721

0.3

(2/16)

1) Ambiguous Characterization of Sensitivity?! Non-unique, conflicting, incomprehensive, etc.

2) Computational Demand…Large numbers of samples (i.e., model runs) required.

Figure from Song et al. (2015) JoH

Derivate-based Approache.g., Morris and its variations

Variance-based Appraoche.g., Sobol’ and FAST

Different approaches to global sensitivity analysisin hydrologic modelling

-0.25

0

0.25

0.5

0.75

1

1.25

-1.25 -1 -0.75 -0.5 -0.25 0 0.25 0.5 0.75 1 1.25

y

f1(x) = 1.11x2

x

y

Response Surfacesf2(x) = x2 - 0.2 cos(7πx)

x1x2

y

-0.25

0

0.25

0.5

0.75

1

1.25

-1.25 -1 -0.75 -0.5 -0.25 0 0.25 0.5 0.75 1 1.25

y

f1(x) = 1.11x2

x

y

Response Surfaces

-8

-6

-4

-2

0

2

4

6

8

-1.25 -1 -0.75 -0.5 -0.25 0 0.25 0.5 0.75 1 1.25

Derivative Functions

x

dy/d

x

f2(x) = x2 - 0.2 cos(7πx)

Derivative-based SA (Morris Approach)

Variance-based SA (Sobol Approach)

-0.25

0

0.25

0.5

0.75

1

1.25

0 1 2 3 4 5 6

y

Probability density

Probability DistributionFunctions

(3/16)

x1x2

y

Variogram Function:

Covariogram Function:

h1h2

where

Response Function:

Distance:where

VARSImportant Notation

1) DirectionDirectional Variograms:

2) Scale

, ,

xAxB

Sample two points and

A ‘Pair’

yB

yA

(7/16)

0

0.1

0.2

0.3

0.4

0.5

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2

γ(h)

hh

𝛾𝛾(h)

Directional Variograms

Higher 𝛾𝛾(hi) for any given hi, indicates a higher ‘rate of variability’ in thedirection of the ith factor, at the scale represented by that hi.

The rate of variability at a particular scale in the problem domain is arepresentation of the ‘scale-dependent sensitivity’ of the response surface.

-0.25

0

0.25

0.5

0.75

1

1.25

-1.25 -1 -0.75 -0.5 -0.25 0 0.25 0.5 0.75 1 1.25

y

x

f1(x) = 1.11x2

x

y

Response Surfacef2(x) = x2 - 0.2 cos(7πx)

Meaningful within Half of Factor Range

(8/16)

0

0.1

0.2

0.3

0.4

0.5

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2

γ(h)

hh

𝛾𝛾(h)

Directional Variograms

0.00001

0.0001

0.001

0.01

0.1

1

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2

H

Γ(H

)

Integrated Variograms

There may not exist a single particular scale that provides an accurateassessment of sensitivity.

This warrants the development of SA metrics that encompasssensitivity information over a range of scales.

Γ(0.2)

(9/16)

Data(Samples from Response Surface)

VARS

VARS generates a ‘spectrum’ of information on sensitivity, while as limiting cases, it reduces to Morris and Sobol.

IVAR

S M

etric

s

Morris(Derivative-based)

Sobol(Variance-based)

Integrated VariogramsAcross a Range of Scales

(10/16)

x1x2

x3

x1x2

x3

Computational Cost =

# Stars# Factors

VARS Resolution

o Directional Variograms, Integrated Variograms (IVARS), and Directional Covariograms

o Derivative-based Sensitivity Measures (Morris)

o Variance-based Total-Order Effect (Sobol)

o Confidence Intervals on Sensitivity Metrics and Reliability Estimates on Sensitivity Rankings

VARS-STAR Products:

h* Parameter ranges scaled between zero and one

γ(h)

WF_R2: WATFLOOD river roughness factor. Incorporates, channel shape, width to depth ratio, and Manning's n.

SDEP: Depth to Bedrock [m] for Crop GRU type

DDEN: Drainage Density. Total length of streams per unit area [km/km2]

C denotes the Crop GRU type

What parameters control peak flows?

SDEP_C8% DDEN_C

7%

WF_R279%

What parameters control peak flows?

WF_R2: WATFLOOD river roughness factor. Incorporates, channel shape, width to depth ratio, and Manning's n.

SDEP: Depth to Bedrock [m] for Crop GRU type

DDEN: Drainage Density. Total length of streams per unit area [km/km2]

C denotes the Crop GRU type

According to IVARS50:

ROOT_C38%

SDEP_C32%

DDEN_C13%

ROOT_G2%

SDEP_G2%

CLAY_Sa3%

RATIO_Sa2%

What parameters control runoff volume (bias)?According to IVARS50:

ROOT: Annual maximum rooting depth of vegetation category (m)

SDEP: Depth to Bedrock [m]

DDEN: Drainage Density. Total length of streams per unit area [km/km2]

C denotes the Crop GRU type

SDE…

DDEN_C12%

WF_R274%

What parameters control Nash-Sutcliffe?According to IVARS50:

WF_R2: WATFLOOD river roughness factor. Incorporates, channel shape, width to depth ratio, and Manning's n.

DDEN: Drainage Density. Total length of streams per unit area [km/km2]

SDEP: Depth to Bedrock [m] for Crop GRU type

C denotes the Crop GRU type

Factor 1Fa

ctor

2

For example:1

23

4

56

VARS efficiency is partly because it is based on the information contained in pairs of points, rather than in individual points.

VARS is efficient and statistically robust, for high-dimensional response surfaces. Several case studies have shown VARS to be more than 1-2 orders of magnitude more efficient than existing SA approaches.

o 6 points form 15 pairs.

o 5 points form 10 pairs.

o A set of k points sampled across a response surface results in pairs (combinations of 2 out of k points).

Number of pairs grows as , where n is rate of increase of points. For k=1,000 points Pairs = 499,500Doubling (n=2) to k=2,000 points Pairs = 1,999,000 (4-fold (n2=4) increase).

(15/16)

Existing SA approaches (e.g. Sobol & Morris) are limited in consistency and utility.

VARS (Variogram Analysis of Response Surfaces) based on Star-based Samplingprovides a Comprehensive framework for sensitivity analysis.

VARS provides spectrum of information about sensitivities variance-based (Sobol) and derivative-based (Morris) are limiting cases (theoretical relations exist).

VARS is --

VARS provides sensitivity information spanning a range of scales, from small-scale features such as roughness and noise to large-scale features such as multi-modality.

Computationally Efficient & Robust

(16/16)

10.8

0.60.4

0.200

0.2

0.4

0.6

0.8

0.83

0.82

0.77

0.81

0.8

0.79

0.78

1

10.8

0.60.4

0.200

0.2

0.4

0.6

0.8

0.74

0.73

0.72

0.71

0.7

0.69

0.68

0.75

1

10.8

0.60.4

0.200

0.2

0.4

0.6

0.8

2

1.5

1

0.5

0

1

10.8

0.6

0.4

0.2

00

0.2

0.4

0.6

0.8

0.6

0.2

0.4

0

-0.2

0.8

1

10.8

0.60.4

0.200

0.2

0.4

0.6

0.8

0.64

0.66

0.68

0.7

0.72

1

0.8

0.78

0.76

0.74

0.72

0.70.2

0.22

0.24

0.26

0.28

0.7205

0.72

0.7195

0.719

0.721

0.3

10.8

0.60.4

0.200

0.2

0.4

0.6

0.8

10

4

2

0

6

8

1

10.8

0.60.4

0.200

0.2

0.4

0.6

0.8

0.7

0.6

0.5

0.3

0.4

1

10.8

0.60.4

0.200

0.2

0.4

0.6

0.8

0.68

0.7

0.72

0.74

0.76

1

10.8

0.60.4

0.200

0.2

0.4

0.6

0.8

0.78

0.76

0.74

0.72

1

10.8

0.60.4

0.200

0.2

0.4

0.6

0.8

1.82

1.84

1.86

1.88

1.9

1.92

1.94

1.96

1

10.8

0.60.4

0.200

0.2

0.4

0.6

0.8

0

8

10

12

14

4

2

6

1

NS-

log

NS-

log

NS

VBIA

S(%

)

NS-

log

NS

VBIA

S(%

)

NS

NS

NS-

log

VBIA

S(%

)

VBIA

S(%

)

(a) (b) (c) (d)

(e) (f) (g) (h)

(i) (j) (k) (l)

Local Sensitivities (i.e., first order derivatives)

Global Distribution of Local Sensitivities (characterized, for example, by mean and variance)

Global Distribution of Model Responses (characterized, for example, by variance)

Structural Organization of the Response Surface (including multi-modality and degree of non-smoothness/roughness)

(from Razavi and Gupta, 2015)

(6/16)