Quantification of Uncertainty in Extreme Scale ... · Uncertainty is represented using probability theory Parameter Estimation, Model Calibration Experimental measurements Regression,

Introduction SW Alg Closure

Quantification of Uncertaintyin Extreme Scale Computations

www.quest-scidac.org

Habib N. Najm

Sandia National Laboratories,Livermore, CA

2014 SciDAC-3 PI MeetingJuly 30 – August 1, 2014

Washington, DC

SNL Najm QUEST 1 / 24


Acknowledgement

QUEST Team:

SNL M. Eldred, B. Debusschere, J. Jakeman,K. Chowdhary, C. Safta, K. Sargsyan

USC R. Ghanem

Duke O. Knio, O. Le Maı̂tre, J. Winokur

UT O. Ghattas, R. Moser, C. Simmons, A. AlexanderianT. Bui-Thanh, N. Petra, G. Stadler

LANL D. Higdon, J. Gattiker

MIT Y. Marzouk, P. Conrad, T. Cui, A. Gorodetsky

This work was supported by:

US Department of Energy (DOE), Office of Advanced Scientific ComputingResearch (ASCR), Scientific Discovery through Advanced Computing (SciDAC)

Sandia National Laboratories is a multiprogram laboratory operated by Sandia Corporation, a Lockheed MartinCompany, for the United States Department of Energy under contract DE-AC04-94-AL85000.



Outline

1 Introduction

2 Progress Highlights – SoftwareDAKOTAQUESO

3 Progress Highlights – AlgorithmsSparsityRandom FieldsAdaptive Sparse QuadratureAsympotoically Exact MCMC

4 Closure



Why UQ? Why in SciDAC?

Why UQ?Assessment of confidence in computational predictionsValidation and comparison of scientific/engineering modelsDesign optimizationUse of computational predictions for decision-supportAssimilation of observational data and model construction

Why UQ in SciDAC?Explore model response over range of parameter variationEnhanced understanding extracted from computationsParticularly important given cost of SciDAC computations



Uncertainty Quantification and Computational Science

!"#$%&'(")'*+,"-.*+

y = f(x)

/)$%&0+

1%&$%&+

23.-45(")0+

2'3'#.&.30+

!"#$%&'(")'*+,"-.*+

y = f(x) x+ y+

Forward problem




!"#$%&'(")'*+,"-.*+

y = f(x)

/)$%&0+

1%&$%&+

23.-45(")0+

2'3'#.&.30+

,.'0%3.#.)&+,"-.*+

z = g(x)

+

,.'0%3.#.)&+,"-.*+

z = g(x)

!"#$%&'(")'*+,"-.*+

y = f(x) x+ y+

6'&'+

Inverse & Forward problems




!"#$%&'(")'*+,"-.*+

y = f(x)

/)$%&0+

1%&$%&+

23.-45(")0+

2'3'#.&.30+

,.'0%3.#.)&+,"-.*+

z = g(x)

+

,.'0%3.#.)&+,"-.*+

z = g(x)

!"#$%&'(")'*+,"-.*+

y = f(x) x+ y+

6'&'+ zd+

Inverse & Forward UQ




!"#$%&'(")'*+,"-.*+

y = f(x)

/)$%&0+

1%&$%&+

23.-45(")0+

2'3'#.&.30+

,.'0%3.#.)&+,"-.*+

z = g(x)

+

,.'0%3.#.)&+,"-.*+

z = g(x)

!"#$%&'(")'*+,"-.*+

y = f(x) x+ y+

6'&'+ zd+

yd+

y ={f1(x), f2(x)7+87+fM(x)}+

Inverse & Forward UQModel validation & comparison, Hypothesis testing



Key Elements of our UQ strategy

Probabilistic frameworkUncertainty is represented using probability theory

Parameter Estimation, Model CalibrationExperimental measurementsRegression, Bayesian Inference

– Markov Chain Monte Carlo (MCMC) methods

Forward propagation of uncertaintyPolynomial Chaos (PC) Stochastic Galerkin methods

– Intrusive/non-intrusiveStochastic Collocation methods

Model comparison, selection, and validationExperimental design and uncertainty management



QUEST UQ Software tools

DAKOTA: Optimization and calibration; non-intrusive UQ;global sensitivity analysis; ∼10K registered downloads.

QUESO: Bayesian inference; multichain MCMC; modelcalibration and validation; decision under uncertainty.

GPMSA: Bayesian inference; Gaussian processemulation; model calibration; model discrepancy analysis.

UQTk: Intrusive and non-intrusive forward PC UQ; customsparse PCE; random fields.

MUQ: Adaptive forward PC UQ; advanced MCMC andvariational methods for inference; efficient surrogates.


Introduction SW Alg Closure DAKOTA QUESO

DAKOTA – Recent Developments – dakota.sandia.gov

High-dimensionality:Sparse representations

Memory conserving approaches to high-dimensionalcompressed sensing and variance-based decompositionMultifidelity compressed sensing

PCE regression with high dimensional basis adaptation.Model Complexity:

Orthogonal least interpolationTail probability estimation – adaptive importance samplingImproved response QoI scalability

Software Integration:Bayesian calibration with QUESO/GPMSA/DREAM

Architecture:Dynamic multi-level job schedulers (MPI & hybrid)


Introduction SW Alg Closure DAKOTA QUESO

QUESO – Recent Developments UT Austin

Migration to Github; expanded user base significantlyhttps://github.com/libqueso/queso

Software quality and usability improvementsFull user documentation and a large number of examplesDeveloper documentation in developmentQUESO-Dakota interface

Ongoing effort to add Gaussian process (GP) basedemulation capabilities to QUESOUsing GPMSA as a referenceEnabling Dakota to access such new capabilities in QUESO

Inference of random fieldsInitial support for fault tolerant samplingInitial support for heterogenous architectures


Introduction SW Alg Closure Sparsity RF ASQ MCMC

Sparsity and Compressive Sensing

Many physical models have a large # of uncertain inputsUQ in this high-dimensional setting is a majorcomputational challenge

too many samples and/or large # PC modesYet physical models typically exhibit sparsity

A small number of inputs are importantSeek sparse PC representation on input space

Small number of dominant terms

Compressed sensing (CS) is useful for discovering sparsityin high dimensional modelsIdentify terms that contribute most to model outputvariationIdeal for when data is limited



Sparse Representations – developments SNL

CS algorithms have been developed for under-determinedsolutions of the coefficients of PC expansions (PCEs)

basis pursuit, basis pursuit denoisingorthogonal matching pursuitleast angle regressionleast absolute shrinkage and selection operator (LASSO).

Orthogonal least interpolation (OLI)determines the lowest order PCE that can interpolate agiven (unstructured) data set.

New capabilities include:support for gradient (adjoint) enhancementfault tolerancecross-validation of algorithm parameterseither structured (sub-sampled tensor product) orunstructured (Latin hypercube) grids



Adaptive Basis Selection

Set 1 Set 2 Set 3

Cardinality of total degree basis grows factorially with thenumber of uncertain inputs.Even for lower dimensional problems redundant basisterms can degrade accuracyTo reduce redundancy and improve accuracy the PCEtruncation can be chosen adaptively.



Random Fields – Relevance

Many applications involve uncertain inputs/outputs thathave spatial or time dependenceSuch an uncertain function, represented probabilistically, isa random field/process.

It is a random variable at each space/time locationGenerally with some correlation structure in space/timeAn infinite-dimensional object

The Karhunen Loeve expansion (KLE) provides an optimalrepresentation of random fields, employing a (small)number of eigenmodes of its covariance function



Random Fields – sparse data SNL

Developed a Bayesian procedure for KLE constructiongiven sparse data

Bayesian Principal Component Analysis (BPCA)Address challenges arising due to

approximate knowledge of the covariance matrixlack of positive definiteness of sample covariance matrix

BPCA framework explores the space of orthonormalvectors, seeking those that best explain the data

Likelihood density p(Φ) is peaked at

Φ∗ = arg minΦ∈Vk(Rd)

n∑i=1

‖xi − PΦxi‖2

where Vk(Rd) is the space of k orthonormal d-dimensional vectors

Resulting KLE incorporates uncertainty due to smallnumber of samples



BPCA Example – Data from a 3D MVN

15 10 5 0 5 10 15

105

05

10

15

10

5

0

5

10

15

Samples of random variables from a3D Multivariate Normal (MVN) distribution

1.00.5

0.00.5

1.0

0.50.00.5

0.5

0.0

0.5

Samples from p(Φ) using 100 samples, xi

1.00.5

0.00.5

1.00.5

0.00.5

0.5

0.0

0.5

First two principal components.Black is the vector with maximum variance

1.00.5

0.00.5

1.0

0.50.00.5

0.5

0.0

0.5

Samples from p(Φ) using 300 samples, xi



BPCA Example – Brownian motion – 25 samples

500-dimensional Brownianmotion stochastic process.Using only 25 samples, wecompute samples from p(Φ) andplot the first three principalcomponents.The dark solid lines represent theprincipal components and theshaded region represents errorbars based on samples using theBayesian PCA approach.



BPCA Example – Brownian motion – 250 samples

Using 250 samplesModes are evaluated withimproved accuracyLower uncertainty



Random Fields – large scale NOAA data – SVD

KLE for uncertain Sea Surface Temperature (SST)1/4-degree spatial resolution data

106-dimensional random field encompassing spatial andtemporal uncertainty in SST dataSVD using Trilinos / parallelized block Krylov Schur solverHopper / NERSC implementation

Mean SST 1st KL modeSNL Najm QUEST 18 / 24


Adaptive Sparse Quadrature (ASQ) for UQ Duke/MIT

Non-Intrusive Pseudospectral projection using sparsetensorization of 1-D quadrature formulas:

prevent internalaliasingimprove accuracyreduce number ofsimulations

Accuracy Requirement Comparison

Final projection exactness requirements are significantly reduced!

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 200123456789

1011121314151617181920

j1 orderj 2 o

rder

Polynomial IndexRequired Quadrature Exactness

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 200123456789

1011121314151617181920

j1 order

j 2 ord

er

Polynomial IndexRequired Quadrature Exactness

In this schematic example, we set the polynomial tensor K and prescribedL. In practice we set L which prescribes K

• A PSP with L has the same realization stencil as a quadrature builtwith L but admits more polynomials than a direct projection

• Especially high-order monomial termsJustin Winokur Hierarchical Sparse Adaptive Sampling 8 / 34

Adaptivity:progressive construction by introducing new tensorizationswith cost controlrobust error indicator to guide the adaptation processnested hierarchical approximation (local dimension-wiseerror control)



UQ with ASQ – Ocean Dynamics Simulation

Example of application: uncertainty in subgrid mixing and windcoupling parameterization (4-dimensions) in hurricane Ivansimulations (. 400 realizations)Variance(analysis(

! Based(on:("  mean(SST(in(“skill”(region;(and("  heat(flux((energy(uptake)(beneath(the(hurricane(

−95 −90 −85 −80

20

22

24

26

28

30

Longitude

Latit

ude

At t = 150, ⟨SST⟩ = 28.04

25

26

27

28

29

30

Longitude

Latit

ude

At t = 150, ⟨Q⟩ = -362.70

150150

250

250

250 350

350

350

450

450450

450

550

550

550

550

550

650

650

650650

650

650 750

750

750750

750

750

850

850

850

850

850

850

950

950

950

950

950

950

−95 −90 −85 −80

20

22

24

26

28

30

−800

−600

−400

−200

0



Asymptotically Exact MCMC – I MIT

Forward UQ yields useful surrogates for BayesianinferenceYet surrogates should be most accurate in regions of highposterior probabilityWe have developed a new approach for incrementallyconstructing local approximations during MCMC

earlier times later timesSNL Najm QUEST 21 / 24


Asymptotically Exact MCMC – II

Algorithm applies approximate MCMC transition kernels,but is provably ergodic with respect to the exact posteriorProbability of evaluating the full forward model during agiven MCMC iteration approaches zeroSpeedups of several orders of magnitude over directMCMC sampling

Applied to large-scaleinference problem with ablack-box forward model :MITgcm for ice-oceandynamics in the WestAntarctic Ice Sheet (with P.Heimbach, MIT)

Code available in the latestrelease of MUQ

102oW 30’ 101oW 30’ 100oW 30’

24’

12’

75oS

48’

36’

0

100

200

300

400

500

600

700

800

900

1000

Satellite image and sample locations



Asymptotically Exact MCMC – III

Elliptic PDE inverse problem: ∇ · (κ(x)∇u(x)) = −fInfer permeability field κ(x) from limited/noisy observationsof pressure u

104 105

MCMC step

10-3

10-2

10-1

100

101

Rela

tive c

ovari

ance

err

or

True model

Linear

Quadratic

GP

Accuracy of chains

104 105

MCMC step

102

103

104

105

Tota

l num

ber

of

evalu

ati

ons True model

Linear

Quadratic

GP

Cost of chains

Only 300 model evaluations needed for 105 MCMC samples!



Closure

Highlights of recent progressSoftwareAlgorithms

Refining and robustifying QUEST algorithms and softwareto address UQ challenges in large-scale problems

high dimensionalitylarge range of scalescomplex models and high computational cost

Addressing UQ needs of SciDAC application partnershipsEight funded active partnerships

Read more at: quest-scidac.org


Documents

Quantification of Uncertainty in Extreme Scale ... · Uncertainty is represented using probability theory Parameter Estimation, Model Calibration Experimental measurements Regression,