A Framework for Quantifying Uncertainty in InfoSymbiotic ... · International Journal of Numerical Methods in Engineering, Vol. 113, Issue 3, pp. 512–533, 2018. A Framework for

A Framework for Quantifying Uncertainty inInfoSymbiotic Systems

Arising in Atmospheric EnvironmentsAFOSR DDDAS FA9550–17–1–0015Program manager: Dr. Erik Blasch

Adrian Sandu1

1Computational Science Laboratory (CSL)“Compute the Future!”

Department of Computer ScienceVirginia Tech

September 20, 2018

InfoSymbiotic Systems for Atmosphere. Title. [1/17]DDDAS PI meeting, Sep. 20 2018. Computational Science Lab (http://csl.cs.vt.edu)

Tropical storm Florence: a reminder of the importanceof DDDAS in atmospheric environments

(a) Satellite view (b) DDDAS predicted path

(c) USS Nitze departing Norfolk (d) Return after evacuation

InfoSymbiotic Systems for Atmosphere. Importance to Air Force. [2/17]DDDAS PI meeting, Sep. 20 2018. Computational Science Lab (http://csl.cs.vt.edu)

DDDAS setting: fuse information from {prior, model,observations} to learn about system and sensors

Observations

SAMSI Workshop on Statistical Inverse Problems

Computer Model

Physical reality

Sensor Network Configuration

Data Assimilation

Modeling &Discretization

Measure-ments

DDDAS inference fuses information from prior, model, and observations, to learn about system, sensors

Inference

InfoSymbiotic Systems for Atmosphere. Introduction. [3/17]DDDAS PI meeting, Sep. 20 2018. Computational Science Lab (http://csl.cs.vt.edu)

Summary of FA9550-17-1-0015 activities in Yrs 1 & 2:space/environment awareness program theme


Summary of FA9550-17-1-0015 activities in Yrs 1 & 2:space/environment awareness program theme


Ensemble Kalman filters (EnKF) take a Monte-Carloapproach to propagate uncertainty

InfoSymbiotic Systems for Atmosphere. Covariance localization. [6/17]DDDAS PI meeting, Sep. 20 2018. Computational Science Lab (http://csl.cs.vt.edu)

Goal A: accurate ensemble-based estimation of errorcovariances. Small ensembles = big challenges.Error correlations

http://www.virginiaplaces.org

Error correlations decrease withphysical distance between variables

EnKF sampling error lead to spurious correlations between distant locations

Pb ← ρ ◦ Pf ⇔Pb

i,j ← ρi,j Pbi,j ≈ cov(xi , xj);

ρi,j = `(dist(xi , xj)/r

),

`(u) = exp(−u2) (or other).

EnKF performance dependscritically on the choice of r


Bayesian approach to adaptive localization: estimateinfluence radii from ensemble and observation data

π(x , r | y) ∝ π(y | x , r)π(x | r ,y)π(r)π(y) (Bayes)

r ML = arg min(− logπ(x , r | y)

).

ψ

0 0.2 0.4 0.6 0.8 1

x

0

0.2

0.4

0.6

0.8

1

y

-60

-40

-20

0

20

40

60

80

q

0 0.2 0.4 0.6 0.8 1

x

0

0.2

0.4

0.6

0.8

1

y

-1.5

-1

-0.5

0

0.5

1

1.5

2

×105

(e) Stream function ψ of QG-1.5{Sakov and Oke, 2008}

1 1.05 1.1 1.15

Inflation

0.2

0.4

0.6

0.8

1

1.2

RMSE

Constant RadiusAdaptive Loc.

(f) EnKF: Bayes-adaptive vs. Sakov’s hand-tuned fixed radius. {Under submission}


Machine Learning approach to adaptive covariancelocalization: learn “ensemble to radius” mappingUse multiple analyses with different radii to learn:{

φ : F(ensemble features)→ ropt,

C(r) = w1 ‖H(xa)− y‖R−1 + w2 DKL(rank histogram | uniform

).

(g) EnKF performance, testing phase (h) Evolution of the localization radii

Figure: QG-1.5 {https://arxiv.org/pdf/1801.00548.pdf}


Goal B: data assimilation for highly non-Gaussian errorpdfs. Approach: sample directly the posterior PDF• Why? EnKF updates Gaussian. MLEF/hybrid, Iterated EnKF, PF.

1. Forecast step: prior represented by GMM (AIC/BIC, EM):

Pb(x) =NC∑i=1

τi N (x;µi , Σi) where τi > 0,NC∑i=1

τi = 1 .

2. Analysis step: posterior is sampled via Hamiltonian MC:

Bayes:

{Pa(x) ∝ exp (−J (x)) ,J (x) = − log P(y|x)− log Pb(x),

Symplecticintegration:

{H(p, x) = 1

2 pT M−1p + J (x),x′ = M−1p , p′ = −∇xJ (x),

Sample fromcanonical pdf:

{P(p, x) ∝ exp

(− 1

2 pT M−1p)Pa(x).

3. Multi-chain version (MC-C`HMC): separate chain per component.

InfoSymbiotic Systems for Atmosphere. DA via posterior sampling. [10/17]DDDAS PI meeting, Sep. 20 2018. Computational Science Lab (http://csl.cs.vt.edu)

C`HMC results with a quasi-geostrophic model

0 20 40 60 80 100Time (assimilation cycles)

0123456789

RM

SE

ForecastHMCClHMCMC-ClHMC

(a) Root mean square errors

0 5 10 15 20 25Rank (truth among ensemble members)

0.00

0.05

0.10

0.15

0.20

Rela

tiv

e f

req

uen

cy

(b) MC-C`HMC rank histogram

Figure: Data assimilation with the QG model and a nonlinear observation operator.All standard EnKF methods diverge for this problem. {Atmosphere, 9(6):213, 2018;International Journal of Numerical Methods in Fluids, 83(1):90-112, 2017.}

InfoSymbiotic Systems for Atmosphere. DA via posterior sampling. [11/17]DDDAS PI meeting, Sep. 20 2018. Computational Science Lab (http://csl.cs.vt.edu)

Goal C: guide physical package configuration incomplex models to aid forecast accuracy

(a) NCEP: 6AM, 5/1/2017 (b) NCEP: 12PM, 5/1/2017 (c) NCEP: 6PM, 5/1/2017

(d) WRF: 6PM, 5/1/2017 (e) Physics {Dudhia, 2015}

InfoSymbiotic Systems for Atmosphere. ML for UQ in complex models. [12/17]DDDAS PI meeting, Sep. 20 2018. Computational Science Lab (http://csl.cs.vt.edu)

Problem one: estimate in advance aspects of interestof the systematic model error

∆̂t+1 = φerror(Θ,F

(∆t+1−`, . . . ,∆t ,ot+1−`, . . . ,ot

)),

δ̂t+1 = x̂t+1 − xt+1 ≈ cov(xt ,ot) (cov(ot ,ot) + Rt)−1︸︷︷︸

Kalman gain

∆̂t+1.

(f) WRF vs. NCEP (g) (WRF+δ̂t+1) vs. NCEP

Figure: Systematic error estimation. Discrepancy in precipitation predictions overVirginia at 6PM on 5/1/2017. {arxiv 1802.08055}


Problem two: identify physical packages thatcontribute most to the forecast uncertainty

train: φphysics (∆t) ≈ Θ

apply: φphysics(∆{i}t,j

)≈ Θ

{i}j , i = 1, 2, . . .

monitor: Θ{i1}j 6= Θ

{i2}j .

Figure: Frequency of change in the physics from ∆testt=12PM to ∆test

t=12PM/2.{Metrologia:55(1), 2018; https://arxiv.org/abs/1802.08055}


Tech transfer: DATeS – Python-based Extensible DataAssimilation Testing Suite

DATeS: Python-based Extensible DataAssimilation Testing Suite

DATeS is aimed to be a unified testing suite for data assimilation (DA) applications whereresearchers can collaborate, so that it would be much easier to understand and comparedifferent methodologies in different settings. The core of DATeS is implemented in Python toenable for Object-Oriented capabilities. The main functionalities, such as the models, the dataassimilation algorithms, the linear algebra solvers, and the time discretization routines areindependent of each other, such as to offer maximum flexibility to configure data assimilationapplications. DATeS can interface easily with large third party models written in Fortran or C,and with various external solvers.

License

This program is subject to the terms of the Virginia Tech Non-Commercial/Commercial License.Using the software constitutes an implicit agreement with the terms of the license. You shouldreceive a copy of the Virginia Tech Non-Commercial License with this program; if not, pleasecontact the computational Science Laboratory to obtain it.

Contributors

1- Ahmed Attia [email protected] Adrian Sandu [email protected] Mahesh Narayanamurthi [email protected] Steven Ross Glandon [email protected] Paul Tranquilli [email protected] Arash Sarshar [email protected]

http://people.cs.vt.edu/∼attia/DATeS/About.html (arxiv 1704.05594)InfoSymbiotic Systems for Atmosphere. Summary. [15/17]DDDAS PI meeting, Sep. 20 2018. Computational Science Lab (http://csl.cs.vt.edu)

Contributors

Andrey Popov Azam Moosavi(Tesla)

Mahesh Narayanamurthi

Elias D. Nino(U. del Norte, Colombia)

Vishwas Rao(Argonne N.L.)

Ahmed Attia(Argonne N.L.)

InfoSymbiotic Systems for Atmosphere. Summary. [16/17]DDDAS PI meeting, Sep. 20 2018. Computational Science Lab (http://csl.cs.vt.edu)

Journal publications to date1. E.D. Niño-Ruiz*, A. Sandu, and X. Deng: “An Ensemble Kalman Filter Implementation Based on Modified Cholesky

Decomposition for Inverse Covariance Matrix Estimation.” SIAM Journal on Scientific Computing, Vol. 40(2), pp.A867–A886, 2018.

2. E.D. Niño-Ruiz*, A. Sandu, and X. Deng: “A Parallel Implementation of the Ensemble Kalman Filter Based on ModifiedCholesky Decomposition.” Journal on Computational Science, DOI:10.1016/j.jocs.2017.04.005, 2018.

3. E.D. Niño-Ruiz*, A. Sandu, and X. Deng: “Efficient Parallel Implementation of DDDAS Inference Using an EnsembleKalman Filter with Shrinkage Covariance Matrix Estimation.” Cluster Computing, DOI:10.1007/s10586-017-1407-1, 2017.

4. V. Rao*, A. Sandu, M. Ng, and E.D. Niño-Ruiz*: “Robust Data Assimilation Using L1 and Huber Norms.” SIAM Journal onScientific Computing, Vol. 39, Issue 3, pp. B548–B570, 2017.

5. M. Narayanamurthi*, U. Römer**, and A. Sandu: “Solving Parameter Estimation Problems with Discrete AdjointExponential Integrators.” Optimization Methods and Software, Vol. 33, Issue 4-6, pp. 750–770, 2018.

6. A. Attia*, A.S.Z. Moosavi*, and A. Sandu: “Cluster Sampling Filters for Non-Gaussian Data Assimilation.” Atmosphere,special issue on “Efficient Formulation and Implementation of Data Assimilation Methods”, Vol. 9(6), paper 213, 2018.

7. A. Attia*, V. Rao*, and A. Sandu: “A Hybrid Monte-Carlo Sampling Smoother for Four Dimensional Data Assimilation.”International Journal of Numerical Methods in Fluids, Vol. 83, Issue 1, pp. 90–112, 2017.

8. A. Attia*, R. Stefanescu*, and A. Sandu: “The Reduced-Order Hybrid Monte-Carlo Sampling Smoother.” InternationalJournal of Numerical Methods in Fluids, Vol. 83, Issue 1, pp. 28–51, 2017.

9. A. Attia* and A. Sandu: “DATES: A Highly-Extensible Data Assimilation Testing Suite.” (https://arxiv.org/abs/1704.05594)10. A.S.Z. Moosavi*, A. Attia*, and A. Sandu: “A Machine Learning Approach to Adaptive Covariance Localization.”

(https://arxiv.org/pdf/1801.00548.pdf)11. A.S.Z. Moosavi*, V. Rao*, and A. Sandu: “A Learning Based Approach for Uncertainty Analysis in Numerical Weather

Prediction Models.” (https://arxiv.org/abs/1802.08055)12. A.S.Z. Moosavi*, R. Stefanescu**, and A. Sandu: “Parametric Domain Decomposition for Accurate Reduced Order

Models: Applications of MP-LROM Methodology.” Journal of Computational and Applied Mathematics, Volume 340, pp.629–644, 2018.

13. A.S.Z. Moosavi*, R. Stefanescu**, and A. Sandu: “Efficient Construction of Local Parametric Reduced Order ModelsUsing Machine Learning Techniques.” International Journal of Numerical Methods in Engineering, Vol. 113, Issue 3, pp.512–533, 2018.

14. A.S.Z. Moosavi* and A. Sandu: “A State-Space Approach to Analyze Structural Uncertainty in Physical Models.”Metrologia, MET–100885, Vol. 55, No. 1, 2018.

InfoSymbiotic Systems for Atmosphere. Summary. [17/17]DDDAS PI meeting, Sep. 20 2018. Computational Science Lab (http://csl.cs.vt.edu)

Documents

A Framework for Quantifying Uncertainty in InfoSymbiotic ... · International Journal of Numerical Methods in Engineering, Vol. 113, Issue 3, pp. 512–533, 2018. A Framework for