Steps in Uncertainty Quantification

Challenge:• How do we do uncertainty quantification for computationally expensive models?

• Example:

– We have a computational budget of 5000 model evaluations.

– Bayesian inference and uncertainty propagation require 120,000 evaluations.

Uncertainty Quantification ChallengesExample: MFC model – Fourth-order PDE

Capacitor probe

MFC Patch

Bayesian Inference: Took 6 days!Macro-Fiber Composite

Solution: Highly efficient surrogate models

Problem:

M = -c

E

I

@2w

@x

2 - c

D

I

@3w

@x

2@t

- [k1e(E ,�0)E + k2"irr

(E ,�0)]�MFC

(x)

⇢@2

w

@t

2 -@2

M

@x

2 = f

cE I1.2 ⇥ 10

5

PDE solutions

138

Surrogate Models: MotivationExample: Consider the heat equation

Notes:• Requires approximation of PDE in 3-D

• What would be a simple surrogate?

t

1 x , y , z

with the response

Boundary Conditions

Initial Conditions

y(q) =

Z1

0

Z 1

0

Z 1

0

Z 1

0u(t , x , y , z)dxdydzdt

@u

@t

=@2

u

@x

2 +@2

u

@y

2 +@2

u

@z

2 + f (q)

139

Surrogate Models: MotivationExample: Consider the heat equation

with the response

Boundary Conditions

Initial Conditions

y(q) =

Z1

0

Z 1

0

Z 1

0

Z 1

0u(t , x , y , z)dxdydzdt

@u

@t

=@2

u

@x

2 +@2

u

@y

2 +@2

u

@z

2 + f (q)

Question: How do you construct a polynomial surrogate?• Regression

• Interpolation

t

1 x , y , z

Surrogate: Quadraticys(q) = (q - 0.25)2 + 0.5

Surrogate ModelsRecall: Consider the model

Question: How do you construct a polynomial surrogate?• Interpolation

• RegressionM=7k=2

with the response

Boundary Conditions

Initial Conditions

y(q) =

Z1

0

Z 1

0

Z 1

0

Z 1

0u(t , x , y , z)dxdydzdt

@u

@t

=@2

u

@x

2 +@2

u

@y

2 +@2

u

@z

2 + f (q)

t

1 x , y , z

Surrogate: Quadraticys(q) = (q - 0.25)2 + 0.5

Surrogate Models

Question: How do we keep from fitting noise?• Akaike Information Criterion (AIC)

• Bayesian Information Criterion (AIC)

Likelihood:

M=7k=2

Minimize

Maximize

SSq =MX

m=1

[ym - ys(qm)]2

AIC = 2k - 2 log[⇡(y |q)]

BIC = k log(M)- 2 log[⇡(y |q)]

⇡(y |q) =1

(2⇡�2)M/2 e-SSq/2�2

142

Surrogate Models

Question: How do we keep from fitting noise?• Akaike Information Criterion (AIC)

• Bayesian Information Criterion (AIC)

Likelihood:

M=7k=2

Maximize

SSq =MX

m=1

[ym - ys(qm)]2

AIC = 2k - 2 log[⇡(y |q)]

BIC = k log(M)- 2 log[⇡(y |q)]

⇡(y |q) =1

(2⇡�2)M/2 e-SSq/2�2

Example: Exercise:• Construct a polynomial surrogate using the code

response_surface.m.

• What order seems appropriate?

y = exp(0.7q1

+ 0.3q2

)

!

q 2

q1

Minimize

Data-Fit ModelsNotes:• Often termed response surface models, emulators, meta-models.

• Constructed via interpolation or regression.

• Data can consist of high-fidelity simulations or experiments.

y = f(q)

Example: Steady-state Euler-Bernouilli beam model with PZT patch

Data: Displacement observations

Parameter:

YI

d

4w

dx

4 (x) = k

p

V�pzt

(x)

YI

144

Data-Fit ModelsExample: Steady-state Euler-Bernouilli beam model with PZT patch

Data: Displacement observationsParameter: YI Training points: 5000

Bayesian Inference

YI x

disp

lace

men

t

Polynomial surrogate: 6th order

YI

d

4w

dx

4 (x) = k

p

V�pzt

(x)

Data-Fit ModelsNotes:• Often termed response surface models, surrogates, emulators, meta-models.

• Rely on interpolation or regression.

• Data can consist of high-fidelity simulations or experiments.

• Common techniques: polynomial models, kriging (Gaussian process regression), orthogonal polynomials.

Statistical Model:

y = f(q)

Strategy: Consider high fidelity model

with M model evaluations

Surrogate:

Note: k(Q) orthogonal with respect to

inner product associated with pdf

e.g., Q ⇠ N(0, 1): Hermite polynomials

Q ⇠ U(-1, 1): Legendre polynomials

yK (Q) = fs(Q) =KX

k=0

↵k k(Q)

y = f (q)

ym = f (qm) , m = 1, ... , M

fs(q): Surrogate for f (q)

ym = fs(qm) + "m , m = 1, ... , M 146

Orthogonal Polynomial RepresentationsRepresentation:

yK (Q) =KX

k=0

↵k k(Q)

Note: 0(Q) = 1 implies that

E[ 0(Q)] = 1

E[ i(Q) j(Q)] =

Z

� i(q) j(q)⇢(q)dq

= �ij�i

where �i = E[ 2i (Q)]

Properties:

(i) E[yK (Q)] = ↵0

(ii) var[yK (Q)] =KX

k=1

↵2k�k

Note: Can be used for:• Uncertainty propagation

• Sobol-based global sensitivity analysis

Issue:• Stochastic Galerkin techniques (Polynomial Chaos Expansion – PCE)

• Nonintrusive PCE (Discrete projection)

• Stochastic collocation

• Regression-based methods with sparsity control (Lasso)

How does one compute ↵k , k = 0, ... , K ?

Note: Methods nonintrusive and treat code as blackbox. 147

Orthogonal Polynomial RepresentationsNonintrusive PCE:

to obtain

Take weighted inner product of y(q) =P1

k=0

↵k k(q)

↵k =1�k

Z

�y(q) k(q)⇢(q)dq

Note: Sample points

↵k ⇡ 1�k

RX

r=1

y(qr ) k(qr )wr

Note:(i) Low-dimensional: Tensored 1-D quadrature rules – e.g., Gaussian

(ii) Moderate-dimensional: Sparse grid (Smolyak) techniques

(iii) High-dimensional: Monte Carlo or quasi-Monte Carlo (QMC) techniques

Regression-Based Methods with Sparsity Control (Lasso): Solve

Quadrature:

d = [y(q1) , ... , y(qm)]

{qm}Mm=1e.g., SPGL1

• MATLAB Solver for large-scale sparse reconstruction

⇤ 2 RM⇥(K+1) where ⇤jk = k(qj)

min

↵2RK+1

k⇤↵- dk2

subject to

KX

k=0

|↵k | 6 ⌧

148

Stochastic CollocationStrategy: Consider high fidelity model

with M model evaluations

Collocation Surrogate:

Note: Result:

ym

Lm(q) =MY

j=0j 6=m

q - q j

q m - q j =(q - q 1) · · · (q - q m-1)(q - q m+1) · · · (q - q M)

(q m - q 1) · · · (q m - q m-1)(q m - q m+1) · · · (q m - q M)

Lm(q j) = �jm =

�0 , j 6= m1 , j = m

y = f (q)

ym = f (qm) , m = 1, ... , M

where Lm(q) is a Lagrange polynomial, which in 1-D, is represented by

Y M(q) =MX

m=1

ymLm(q)

Y M(qm) = ym

149

Orthogonal Polynomial Methods for PDE

Evolution Model: e.g., thermal-hydraulic equations

@u

@t

= N(u, Q) + F (Q) , x 2 D, t 2 [0,1)

B(u, Q) = G(Q) , x 2 @D, t 2 [0,1)

u(0, x , Q) = I(Q) , x 2 D

Weak Formulation: For all v 2 VZ

D

@u

@t

vdx +

Z

D

N(u, Q)S(v)dx =

Z

D

F (Q)vdx

Response: y(t , x) =

Z

�u(t , x , q)⇢(q)dq

Example: q = ↵

@u

@t

= ↵@2

u

@x

2

y(t , 0) = u(t , L) = 0

u(0, x) = u0(x)

For all v 2 H1

0

(0, L)Z

L

0

@u

@t

vdx + ↵

ZL

0

@u

@x

dv

dx

= 0

Representation:

u

K (t , x , Q) =KX

k=0

u

k

(t , x) k

(Q)

=KX

k=0

JX

j=1

u

jk

(t)�j

(x) k

(Q)

u

k

(t , x) ⇡ 1�

k

RX

r=1

u(t , x , q

r ) k

(qr )wr

Discrete Projection:

e.g., Finite elements150

Surrogate Models – Grid Choice

Example:

−1 −0.5 0 0.5 1−0.2

0

0.2

0.4

0.6

0.8

1

1.2

q

f(q)

Consider the Runge function f (q) = 1

1+25q2

with points

qj = -1 + (j - 1)2M

, j = 1, ... , M

151

Surrogate Models – Grid Choice

Example:

−1 −0.5 0 0.5 1−0.2

0

0.2

0.4

0.6

0.8

1

1.2

q

f(q)

−1 −0.5 0 0.5 1−80

−60

−40

−20

0

20

q

f(q)

Consider the Runge function f (q) = 1

1+25q2

with points

qj = -1 + (j - 1)2M

, j = 1, ... , M

152

Surrogate Models – Grid Choice

Example:

−1 −0.5 0 0.5 1−0.2

0

0.2

0.4

0.6

0.8

1

1.2

q

f(q)

−1 −0.5 0 0.5 10

0.2

0.4

0.6

0.8

1

q

f(q)

−1 −0.5 0 0.5 1−80

−60

−40

−20

0

20

q

f(q)

Consider the Runge function f (q) = 1

1+25q2

with points

qj = -1 + (j - 1)2M

, j = 1, ... , M qj = - cos

⇡(j - 1)

M - 1

, j = 1, ... , M

153

Surrogate Models – Grid Choice

Example:

−1 −0.5 0 0.5 1−0.2

0

0.2

0.4

0.6

0.8

1

1.2

q

f(q)

−1 −0.5 0 0.5 10

0.2

0.4

0.6

0.8

1

q

f(q)

−1 −0.5 0 0.5 1−80

−60

−40

−20

0

20

q

f(q)

−1 −0.5 0 0.5 10

0.2

0.4

0.6

0.8

1

q

f(q)

Consider the Runge function f (q) = 1

1+25q2

with points

qj = -1 + (j - 1)2M

, j = 1, ... , M qj = - cos

⇡(j - 1)

M - 1

, j = 1, ... , M

154

Sparse Grid TechniquesTensored Grids: Exponential growth Sparse Grids: Same accuracy

p R` Sparse Grid R Tensored Grid R = (R`)p

2 9 29 81

5 9 241 59,049

10 9 1581 > 3⇥ 109

50 9 171,901 > 5⇥ 1047

100 9 1,353,801 > 2⇥ 1095

155

Surrogate Construction: Nuclear Power Plant Design

Subchannel Code (COBRA-TF): 33 VUQ parameters reduced to 5 using SA

Surrogate: Total pressure drop

• Kriging (GP) emulator constructed using 50 COBRA-TF runs perturbing 5 active inputs.

• Use remaining computational budget to evaluate quality of surrogate using post-processed Dakota outputs.

●●

●

●

●

●

●

●

●●

●

●

●

●●

●●

●

●

●

1.10 1.15 1.20 1.25

1.10

1.15

1.20

1.25

Calculated Total Pressure Drop

Pred

icte

d To

tal P

ress

ure

Dro

p

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

1.10 1.15 1.20 1.25

−10

12

Calculated Total Pressure Drop

Stan

dard

ized

Res

idua

l Tot

al P

ress

ure

Dro

p

Pred

icte

d To

tal P

ress

ure

Dro

p

Calculated Total Pressure Drop Theoretical Quantities

Sam

ple

Qua

ntiti

es

Out-of-Sample Validation

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●●

●

●

●

−2 −1 0 1 2

−2−1

01

Leave−one−out Cross Validation

Theoretical Quantiles

Sam

ple

Qua

ntile

s

●

●

●

● ●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

−2 −1 0 1 2

−10

12

Out−of−sample Validation

Theoretical Quantiles

Sam

ple

Qua

ntile

s

156

Example: SIR Cholera ModelModel:

dSdt

= bN - �LSBL

L + BL- �HS

BH

H + BH- bS

dIdt

= �LSBL

L + BL+ �HS

BH

H + BH- (�+ b)I

dRdt

= �I - bR

dBH

dt= ⇠I - �BH

dBL

dt= �BH - �BL

Model Parameter Symbol Units ValuesRate of drinking BL cholera �L

1week 1.5

Rate of drinking BH cholera �H1

week 7.5 (⇤)

BL cholera carrying capacity L# bacteria

m` 106

BH cholera carrying capacity H# bacteria

m`L700

Human birth and death rate b 1week

11560

Rate of decay from BH to BL � 1week

1685

Rate at which infectious individuals ⇠ # bacteria# individuals·m`·week 70

spread BH bacteria to waterDeath rate of BL cholera � 1

week7

30Rate of recovery from cholera � 1

week75

δ

BH

BL

βH

βL

I RSγ

ξ

χ

157

�L �HL b � ⇠ �L �

0

0.1

0.2

0.3

0.4

0.5

parameter

generalizedSobolindex

MC (Nmc = 10

4)

Alg 1 (Nquad = 609)

Alg 1 (Nquad = 2177)

Alg 2 (Nmc = 150/Nkl = 2)

Alg 2 (Nmc = 150/Nkl = 5)

Alg 2 (Nmc = 150/Nkl = 15)

Example: SIR Cholera ModelStrategy: Employed collocation and discrete projection with sparse grids to compute time-dependent global sensitivity indices.

Conclusion: Sensitive indices�: Recovery rate

�H : Rate of drinking BH cholera

L: BL carrying capacity; Note H = L/700

⇠: Rate at which BH bacteria spread

10

�110

010

110

20

0.2

0.4

0.6

0.8

1

time [weeks]

totalindices

�L

�HL

b�⇠�L�

10

�110

010

110

20

0.2

0.4

0.6

0.8

1

time [weeks]

generalizedtotalindices

�L

�HL

b�⇠�L�

δ

BH

BL

βH

βL

I RSγ

ξ

χ

Example: SIR Cholera ModelModel:

dSdt

= bN - �LSBL

L + BL- �HS

BH

H + BH- bS

dIdt

= �LSBL

L + BL+ �HS

BH

H + BH- (�+ b)I

dRdt

= �I - bR

dBH

dt= ⇠I - �BH

dBL

dt= �BH - �BL

Model Parameter Symbol Units ValuesRate of drinking BL cholera �L

1week 1.5

Rate of drinking BH cholera �H1

week 7.5 (⇤)

BL cholera carrying capacity L# bacteria

m` 106

BH cholera carrying capacity H# bacteria

m`L700

Human birth and death rate b 1week

11560

Rate of decay from BH to BL � 1week

1685

Rate at which infectious individuals ⇠ # bacteria# individuals·m`·week 70

spread BH bacteria to waterDeath rate of BL cholera � 1

week7

30Rate of recovery from cholera � 1

week75

159

Steps in Uncertainty Quantification

160

6. Quantification of Model Discrepancy – Thin Beam“Essentially all models are wrong, but some are useful” George E.P. Box

with

Example: Thin beam driven by PZT patches

Euler-Bernoulli Model:

Statistical Model:

Note: 7 parameters, 32 states

For all � 2 V

ZL

0

⇢(x)

@2w

@t

2 + �@w

dt

��dx +

ZL

0

YI(x)

@2w

@x

2 + cI(x)@3

w

@x

2@t

�� 00

dx

= k

p

V (t)

Zx2

x1

� 00dx

⇢(x) = ⇢hb + ⇢p

h

p

b

p

�p

(x) , YI(x) = YI + Y

p

I

p

�p

(x)

cI(x) = cI + c

p

I

p

�p

(x)y(t

i

, q) = w(ti

, x , q)

ti

YiYi = y(ti , q) + "i 161

Quantification of Model Discrepancy – Thin BeamExample: Good model fit

Note: Observation errors not iid

Model Fit to Data

0 20 40 60 80 100 1200

0.2

0.4

0.6

0.8

1

x 10−7

Frequency

Fre

quency

Conte

nt

Mathematical Model y(ti;q)

Data

Reference: Additive observation errors

Yi = y(ti , q) + "i

Yi = y(ti , q) + �(ti , eq) + "i

• M.C. Kennedy and A. O’Hagan, Journal of the Royal Statistical Society, Series B, 2001.

Quantification of Model Discrepancy – Thin BeamExample: Good model fit

Problem: Observation errors not iid

Result: Prediction intervals wrong

Prediction Intervals and Data

Model Fit to Data

Approaches: • GP Model: Inaccurate for extrapolation• Control-based approaches: difficult to extrapolate.• Problem: correct physics or biology required for extrapolation!

Quantification of Model Discrepancy – Thin BeamProblem: Measurement errors not iid

Result: Prediction intervals wrong

Prediction Intervals and Data

Model Fit to Data

One Approach: • Determine components of model you trust (e.g., conservation laws) and don’t trust (e.g., closure relations). Embed uncertainty into latter.• T. Oliver, G. Terejanu, C.S. Simmons, R.D. Moser, Comput Meth Appl Mech Eng, 2015.

2018-19 SAMSI Program: Model Uncertainty: Mathematical and Statistical (MUMS)

Quantification of Model Discrepancy – Thin Beam

Our Solution: “Optimize” calibration interval

• Use damping/frequency domain results to guide.

0.75 0.8 0.85 0.9 0.95 1−60

−40

−20

0

20

40

60

Time (s)

Dis

pla

cem

en

t (µ

m)

2 2.1 2.2 2.3 2.4 2.5−50

0

50

Time (s)

Dis

pla

cem

en

t (µ

m)

3 3.1 3.2 3.3 3.4 3.5−40

−30

−20

−10

0

10

20

30

40

Time (s)

Dis

pla

cem

en

t (µ

m)

0 1 2 3−30

−20

−10

0

10

20

30

Time (s)

Dis

pla

cem

ent (µ

m)

0 1 2 3−40

−30

−20

−10

0

10

20

30

Time (s)

Dis

pla

cem

en

t (µ

m)

Note: We have substantially extended calibration regime.Calibrate on [0,1] Calibrate on [0.25,1.25]

Concluding RemarksNotes:• UQ requires a synergy between engineering, statistics, and applied mathematics.

• Model calibration, model selection, uncertainty propagation and experimental design are natural in a Bayesian framework.

• Goal is to predict model responses with quantified and reduced uncertainties.

• Parameter selection is critical to isolate identifiable and influential parameters.

• Surrogate models critical for computationally intensive simulation codes.

• Codes and packages: Sandia Dakota, R, MATLAB, Python, nanoHUB.

• Prediction is very difficult, especially if it’s about the future, Niels Bohr.

2 2.1 2.2 2.3 2.4 2.5−50

0

50

Time (s)

Dis

pla

cem

en

t (µ

m)

References

Books:

• R.C. Smith, Uncertainty Quantification: Theory, Implementation, and Applications, SIAM, Philadelphia, 2014.

Incomplete References:

Orthogonal Polynomial Methods and High-Dimensional Approximation Theory:• M. Babuška, F. Nobile and R. Tempone, Numer. Math., 2005

• A. Cohen, R. De Vore and C. Schwab, Analysis and Applications, 2011

• M. Gunzburger, C.G. Webster and G. Zhang, Acta Numerica, 2014.

• O.P. Le Maître and O.M. Knio, Spectral Methods for Uncertainty Quantification, 2010

• S., Uncertainty Quantification: Theory, Implementation, and Applications, 2014.

• A.L. Teckentrup, P. Jantsch, M. Gunzburger and C.G. Webster, SIAM/ASA J. Uncert. Quant., 2015

• D. Xiu, Numerical Methods for Stochastic Computations: A Spectral Method Approach, 2010.

167

Incomplete References:

Sparse Grids:• H-J. Bungartz and M. Griebel, Acta Numerica, 2004

• M. Gunzburger et al., Water Resources Research, 2013.

• F. Nobile, R. Tempone and C.G. Webster, SIAM J. Num. Anal., 2008

Compressed Sensing:• A. Chkifra, N. Dexter, H. Tran and C.G. Webster, Mathematics of Computation, Submitted

References

Infinite-Dimensional Bayesian Inference: • A.M. Stuart, Acta Numerica 2010

• T. Bui-Thanh et al., SIAM J. Sci. Comput., 2013

• S.J. Vollmer, SIAM/ASA J. Uncertainty Quantification, 2015.

• T. Bui-Thanh and Q. Nguyen, Inverse Problems and Imaging, 2016, 168