Discrete optimality in nonlinear model reduction: analysis ...ktcarlb/presentations/kc_15_04_27... · Discrete optimality in nonlinear model reduction: analysis and application to

Discrete optimality in nonlinear model reduction:analysis and application to CFD

Kevin Carlberg1, Matthew Barone1, Harbir Antil2

Sandia National Laboratories1

George Mason University2

1st Pan-American Congress on Computational MechanicsBuenos Aires, Argentina

April 27, 2015

Discrete optimality Carlberg, Barone, Antil 1 / 37

Model reduction at Sandia

CFD model

100 million cells200,000 time steps

High simulation costs

6 weeks, 5000 cores6 runs maxes out Cielo

Barrier

Design engineers requirefaster simulations

Uncertainty quantification

Objective: break barrier


Discrete optimality outperforms Galerkin, but why?

Discrete-optimal ROM outperforms Galerkin on large-scalecompressible-flow problems [Carlberg, 2011, Carlberg et al., 2013]

K.$Carlberg /25GNAT$nonlinear$model$reduction

Ahmed$body:$MinimumRresidual$ROM$is$accurate

10

0 0.02 0.04 0.06 0.08 0.10.345

0.35

0.355

time$(s)

drag

coefficient FOM$(82,074$dof)

MinimumRres.$ROM$

(127$POD$vectors)

Galerkin$ROM

$(127$POD$vectors)

discrete(optimal.ROM.(127.POD.vectors)

Galerkin.ROM.(127.POD.vectors)

full(order.model.(82,000.dof)

time.(s)

drag

.coe

fficien

t

Strong performance attributed to discrete optimality

Limited comparative analysis of the two approaches

Goal: Deeper understanding of Galerkin v.discrete-optimal projection for general time integrators


Outline

Time-continuous and time-discrete representations

Equivalence conditions

Discrete-time error bounds


Outline





Continuous and discrete representations

Full-order modelODE

time discretization

Full-order modelO∆E


Full-order model

ODE (initial value problem)

dx

dt= f(x, t), x(0) = x0,

O∆E, linear multistep schemes: rn (wn) = 0

rn (w) := α0w −∆tβ0f(w, tn) +k∑

j=1

αjxn−j −∆t

k∑j=1

βj f(

xn−j , tn−j)

xn = wn (explicit state update)

O∆E, Runge–Kutta: rni (wn1, . . . ,w

ns ) = 0 , i = 1, . . . , s

rni (w1, . . . ,ws) := wi − f(xn−1 + ∆ts∑

j=1

aijwj , tn−1 + ci∆t)

xn = xn−1 + ∆ts∑

i=1

biwni (explicit state update)



Full-order modelODE

Galerkinprojection

Continuous-optimal ROMODE

time discretization



Galerkin ROM: continuous representation

ODE: Galerkin projection on FOM ODE

1 x(t) ≈ x(t) = Φx(t)⇡ =

2 ΦT (f(x, t)− d xdt ) = 0

⇡ =

( (=

d x

dt= ΦT f(Φx, t), x(0) = ΦTx0.

Theorem

The Galerkin ROM velocity minimizes the error in the FOM velocity fover range (Φ):

d x

dt(Φx, t) = arg min

v∈range(Φ)‖v − f(Φx, t)‖2

2.



Full-order modelODE

Galerkinprojection


time discretization

Continuous-optimal ROMO∆E

time discretization



Galerkin ROM: discrete representation

O∆E, linear multistep schemes: rn (wn) = 0

rn (w) := α0w − ∆tβ0ΦT f(Φw, tn) +k∑

j=1

αj xn−j − ∆t

k∑j=1

βjΦT f(

Φxn−j , tn−j)

xn = wn (explicit state update)

O∆E, Runge–Kutta: rni (wn1, . . . , w

ns ) = 0 , i = 1, . . . , s.

rni (w1, . . . , ws) := wi − ΦT f(Φxn−1 + ∆ts∑

j=1

aijΦwj , tn−1 + ci∆t)

xn = xn−1 + ∆ts∑

i=1

bi wni (explicit state update)


Galerkin ROM: Commutativity

Theorem

Projection and time discretization are commutative for Galerkin ROMs:

rn (w) = ΦT rn (Φw)

rni (w1, . . . , ws) = ΦT rni (Φw1, . . . ,Φws) , i = 1, . . . , s,

Full-order modelODE

Galerkinprojection


time discretizationtime discretization


Galerkinprojection

Continuous-optimal ROMO∆E



Full-ordermodelODE

Galerkinprojection

Continuous-optimal ROM

ODE


Full-ordermodelO∆E

Galerkinprojection


O∆E

Residualminimization

Discrete-optimal ROM

O∆E


Discrete-optimal ROM: discrete representation

O∆E, linear multistep schemes:

wn = arg minz∈Rp‖Arn (Φz) ‖2

2.

m

Ψn(wn)T rn(Φwn) = 0, Ψn(w) := ATA

∂rn

∂w(Φw)

A = I: Least-squares Petrov–Galerkin [LeGresley, 2006, Carlberg et al., 2011]

A = (PΦr )+ P: GNAT [Carlberg et al., 2013]

Alternative norm: `1[Abgrall and Amsallem, 2015]

O∆E, Runge–Kutta:(wn

1, . . . , wns

)= arg min

(z1,...,zs )∈Rp×s

s∑i=1

‖Ai rni (Φz1, . . . ,Φzs) ‖2

2

ms∑

j=1

Ψnij(wn

1, . . . , wns )T rnj

(Φwn

1, . . . ,Φwns

)= 0, i = 1, . . . , s

Ψnij(w1, . . . , ws) := AT

i Ai∂rni∂wj

(w1, . . . , ws)



Full-ordermodelODE

?Galerkin

projection


ODE



Galerkinprojection


O∆E



O∆E



The discrete-optimal ROM sometimes has atime-continuous representation.

Full-ordermodelODE

Petrov–Galerkinprojection


ODE

time discretization

Galerkinprojection


ODE



Galerkinprojection


O∆E



O∆E


Discrete-optimal ROM: continuous representation

Theorem (Linear multistep schemes)

The discrete-optimal ROM is equivalent to applying a Petrov–Galerkinprojection to the ODE with test basis

Ψ(x, t) = ATA

(α0I−∆tβ0

∂f

∂x(x0 + Φx, t)

)Φ

and subsequently applying time integration with time step ∆t if

1 βj = 0, j ≥ 1 (e.g., a single-step method),

2 the velocity f is linear in the state, or

3 β0 = 0 (i.e., explicit schemes).


Discrete-optimal ROM: continuous representation

Theorem (Runge–Kutta schemes)

The discrete-optimal ROM is equivalent to applying a Petrov–Galerkinprojection to the ODE with test basis

Ψ(x, t) = ATA

(I−∆ta11

∂f

∂x(x0 + Φx, t)

)Φ

and subsequently applying time integration if either

1 aij = 0 ∀i 6= j and aii = ajj ∀i , j , or

2 the scheme is explicit, i.e., aij = 0, ∀j ≥ i .


Outline





Equivalence

Ψn(w) := ATA∂rn

∂w(Φw) = ATA

(α0I−∆tβ0

∂f

∂x(Φw, tn)

)Φ


Galerkin projection is discrete-optimal (Ψn(w) = Φ)

1 in the limit of ∆t → 0 with A = 1/√α0I,

2 if the scheme is explicit (β0 = 0) with A = 1/√α0I, or

3 if ∂rn

∂w is positive definite with A the Cholesky factor of [∂rn

∂w ]−1


Outline





Discrete-time error bound


If the following conditions hold:

1 f(·, t) is Lipschitz continuous with Lipschitz constant κ, and

2 ∆t is such that 0 < h := |α0| −|β0|κ∆t,

then∥∥δxnG∥∥ ≤ ∆t

h

k∑`=0

|β`|∥∥∥∥(I− V) f

(x0 + Φxn−`

G

)∥∥∥∥+1

h

k∑`=1

(|β`|κ∆t +|α`|

)∥∥∥δxn−`G

∥∥∥∥∥δxnD

∥∥ ≤ ∆t

h

k∑`=0

|β`|∥∥∥∥(I− Pn) f

(x0 + Φxn−`

D

)∥∥∥∥+1

h

k∑`=1

(|β`|κ∆t +|α`|

)∥∥∥δxn−`D

∥∥∥ ,with

δxnG := xn? −ΦxnG .

δxnD := xn? −ΦxnD

V := ΦΦT

Pn := Φ(

(Ψn)TΦ)−1

(Ψn)T


Discrete-time error bound

Theorem (Backward Euler)

If conditions (1) and (2) hold, then∥∥δxnG∥∥ ≤ ∆t

n−1∑j=0

1

(h)j+1

∥∥∥∥(I− V) f(

x0 + Φxn−jG

)∥∥∥∥︸︷︷︸εn−jG∥∥δxnD

∥∥ ≤ ∆t

n−1∑j=0

1

(h)j+1

∥∥∥∥(I− Pn−j)

f(

x0 + Φxn−jD

)∥∥∥∥︸︷︷︸εn−jD

εkG =

∥∥∥∥ΦxkG −∆tf(

x0 + ΦxkG

)−Φxk−1

G

∥∥∥∥εkD =

∥∥∥∥ΦxkD −∆tf(

x0 + ΦxkD

)−Φxk−1

D

∥∥∥∥ = miny

∥∥∥Φy −∆tf (x0 + Φy)−Φxk−1D

∥∥∥Corollary (Discrete-optimal smaller error bound)

If xk−1D = xk−1

G , then εkD ≤ εkG .


Discrete-optimal time-step dependence

Corollary

Define xj as the full-space solution centered at the discrete-optimalsolution:

xj = ∆tf(

x0 + xj)

+ Φxj−1D , j = 1, . . . , n.

Then, the discrete-optimal error can be bounded as∥∥δxnD∥∥ ≤ ∆t(1 + κ∆t)n−1∑j=0

µn−j

(h)j+1‖f(xn−j)‖

with∆xj := xj −Φxj−1

D

∆xjD := xjD − xj−1D

µj :=∥∥∥Φ∆xjD −∆xj

∥∥∥ /∆xj

Effect of decreasing ∆t:

+ The terms ∆t(1 + κ∆t) and 1/(h)j+1 decrease- The number of total time instances n increases? The term µn−j may increase or decrease, depending on the

spectral content of the basis ΦDiscrete optimality Carlberg, Barone, Antil 24 / 37

Example: Cavity-flow problem

Unsteady Navier–Stokes

DES turbulence model

1.2 million degrees offreedom

Linear multistep: BDF2

∆t? = 1.5× 10−3 sec bytime-step verification study

Re = 6.3× 106

M∞ = 0.6


FOM responses

Figure : vorticity field

Figure : pressure fieldDiscrete optimality Carlberg, Barone, Antil 26 / 37

Galerkin and discrete-optimal responses for basis dimension p = 204

0 1 2 3 4 5 61.4

1.6

1.8

2

2.2

2.4

2.6

2.8

t ime

pre

ssure

FOM, ∆t = 0.0015∆t=9.375e-05∆t=0.0001875∆t=0.000375∆t=0.00075∆t=0.0015∆t=0.003∆t=0.006∆t=0.012∆t=0.015∆t=0.024

0 2 4 6 8 10 12 141.6

1.8

2

2.2

2.4

2.6

2.8

3

t ime

pre

ssure

FOM, ∆t = 0.0015∆t=0.00015∆t=0.0001875∆t=0.000375∆t=0.00075∆t=0.0015∆t=0.003∆t=0.006∆t=0.012∆t=0.015∆t=0.024

(a) Galerkin

0 2 4 6 8 10 12 141.6

1.8

2

2.2

2.4

2.6

2.8

3

t ime

pre

ssure

FOM, ∆t = 0.0015∆t=0.00015∆t=0.0001875∆t=0.000375∆t=0.00075∆t=0.0015∆t=0.003∆t=0.006∆t=0.012∆t=0.015∆t=0.024

(b) Discrete optimal

- Galerkin ROMs unstable for all time steps. Consistent withprevious results [Carlberg et al., 2013, Carlberg et al., 2011, Carlberg, 2011]

+ Discrete-optimal ROMs accurate and stable, with a cleardependence on the time step ∆t.


Superior performance (p = 204)

10−5

10−4

10−3

10−2

10−1

∆t

errorin

time-averagedpressure

1

GalerkinMinimum residual

(c) 0 ≤ t ≤ 0.55

10−5

10−4

10−3

10−2

10−1

∆t

errorin

time-averagedpressure

1


(d) 0 ≤ t ≤ 1.1

10−4

10−2

10−4

10−3

10−2

10−1

∆t

errorin

tim

e-averagedpressure

1


(e) 0 ≤ t ≤ 1.54

X When Galerkin is stable, the discrete-optimal ROM yields asmaller error for all time intervals and time steps.


Limiting equivalence

tim

e-av

erag

edpre

ssure

diff

eren

ce

∆t10−4 10−3 10−2 10−1

10−3

10−2

(f) p = 204

tim

e-av

erag

edpre

ssure

diff

eren

ce∆t

10−4 10−3 10−2 10−110−3

10−2

tim

e-av

erag

edpre

ssure

diff

eren

ce∆t

10−4 10−3 10−2 10−110−4

10−2

(g) p = 368

tim

e-av

erag

edpre

ssure

diff

eren

ce

∆t10−4 10−3 10−2 10−1

10−3

10−2

tim

e-av

erag

edpre

ssure

diff

eren

ce

∆t10−4 10−3 10−2 10−1

10−4

10−2

tim

e-av

erag

edpre

ssure

diff

eren

ce

∆t10−4 10−3 10−2 10−1

10−4

10−2

(h) p = 564

Figure : Galerkin/discrete-optimal difference in the stable Galerkininterval 0 ≤ t ≤ 1.1.

X The discrete-optimal ROM converges to Galerkin as ∆t → 0.


POD basis spectral analysis

(a) mode 1 (b) mode 21 (c) mode 101

Mode number0 100 200 300 400 500 600

τ 95

10-2

10-1

100

Higher modes numbers associate with smaller spatial andtemporal scales.


Backward Euler error bound

∥∥δxnD∥∥ ≤ ∆t(1 + κ∆t)n−1∑j=0

µn−j

(h)j+1‖f(xn−j)‖

Approximate µk :=∥∥∥Φ∆xkD −∆xk

∥∥∥ /∆xk with relative projection error.

maxim

um

relativeprojectionerror

∆t

p = 564p = 368p = 204

0 0.04 0.08 0.12 0.16 0.20

0.1

0.2

0.3

0.4

0.5

0.6

0.7

errorbound

∆t

p = 564

p = 368

p = 204

0 0.04 0.08 0.12 0.16 0.210

3

104

105

106

1 Adding basis vectors for larger time steps yields little improvement

2 Approximated error bound: intermediate time step → lowest boundDiscrete optimality Carlberg, Barone, Antil 31 / 37

Discrete-optimal performance in 0 ≤ t ≤ 2.5

∆t

p = 564p = 368p = 204

errorin

time-averaged

pressure

10−5 10−4 10−3 10−2 10−1 10010−4

10−3

10−2

10−1

100

p = 564p = 367p = 204

simulation

time(seconds)

∆t10−4 10−3 10−2 10−1

103

104

105

106

107

1 Adding basis vectors for larger time steps yields little improvement

2 Intermediate time step → lowest error

p = 564 case:

∆t = 1.875× 10−4 sec: relative error = 1.40%, time = 289 hrs

∆t = 1.5× 10−3 sec: relative error = 0.095%, time = 35.8 hrsDiscrete optimality Carlberg, Barone, Antil 32 / 37

GNAT model

wn = arg minz∈Rp‖ (PΦr )+ Prn (Φz) ‖2

2

Sample mesh [Carlberg et al., 2013]: 4.1% nodes, 3.0% original cells

+ Allows GNAT to run on 2 cores instead of 48 cores


GNAT performance

errorin

time-averaged

pressure

∆t10−3 10−2 10−1

10−2

10−1

100

computationalsavings

∆t

10−3 10−2 10−1101

102

103

1.5× 10−3 sec: relative error = 3.32%, cpu savings = 14.9

6.0× 10−3 sec: relative error = 2.25%, cpu savings = 55.7


Conclusions


Galerkin: projection and time-discretization are commutativeDiscrete-optimal: a continuous representation sometimes exists


1 Limit of ∆t → 02 Explicit schemes3 Positive definite residual Jacobians


Discrete-optimal ROM yields smaller error bound than GalerkinAmbiguous role of time step

Numerical experiments

Discrete-optimal ROM always yields a smaller error thanGalerkinEquivalent as ∆t → 0Approximated error bound and actual error minimized forintermediate ∆t


Acknowledgments

Charbel Farhat: permitting the open use of AERO-F

Julien Cortial, David Amsallam, Charbel Bou-Mosleh:contributing to implementation of model reduction in AERO-F

Stephen Pope: insightful conversations that inspired this work

This research was supported in part by an appointment to theSandia National Laboratories Truman Fellowship in NationalSecurity Science and Engineering, sponsored by SandiaCorporation (a wholly owned subsidiary of Lockheed MartinCorporation) as Operator of Sandia National Laboratoriesunder its U.S. Department of Energy Contract No.DE-AC04-94AL85000.


Questions?

K. Carlberg, M. Barone, H. Antil. “Galerkin v. discrete-optimalprojection in nonlinear model reduction,” arXiv e-Print 1504.03749(2015).

∆t

p = 564p = 368p = 204

errorin

time-averaged

pressure

10−5 10−4 10−3 10−2 10−1 10010−4

10−3

10−2

10−1

100

p = 564p = 367p = 204

simulation

time(seconds)

∆t10−4 10−3 10−2 10−1

103

104

105

106

107

Figure : Discrete-optimal ROM performance.


Bibliography I

Abgrall, R. and Amsallem, D. (2015).Robust model reduction by l1-norm minimization andapproximation via dictionaries: Application to linear andnonlinear hyperbolic problems.Stanford University Preprint.

Carlberg, K. (2011).Model Reduction of Nonlinear Mechanical Systems viaOptimal Projection and Tensor Approximation.PhD thesis, Stanford University.


Bibliography II

Carlberg, K., Bou-Mosleh, C., and Farhat, C. (2011).Efficient non-linear model reduction via a least-squaresPetrov–Galerkin projection and compressive tensorapproximations.International Journal for Numerical Methods in Engineering,86(2):155–181.

Carlberg, K., Farhat, C., Cortial, J., and Amsallem, D. (2013).The GNAT method for nonlinear model reduction: effectiveimplementation and application to computational fluiddynamics and turbulent flows.Journal of Computational Physics, 242:623–647.


Bibliography III

LeGresley, P. A. (2006).Application of Proper Orthogonal Decomposition (POD) toDesign Decomposition Methods.PhD thesis, Stanford University.


Documents

Discrete optimality in nonlinear model reduction: analysis ...ktcarlb/presentations/kc_15_04_27... · Discrete optimality in nonlinear model reduction: analysis and application to