42
T : M. B F [email protected] Joint work with A. Nouy, O. Zahm CEMRACS Luminy, 2013

ˆ - CERMICScermics.enpc.fr/~lelievre/CEMRACS/slides/Billaud_Friess.pdf · ¶ Approximation intensor subsetfor u 2X ˇeu2SX ˆX Rank-r canonical tensors: Rr(X) = (Xr i=1 Os =1 ˚

  • Upload
    others

  • View
    4

  • Download
    0

Embed Size (px)

Citation preview

Page 1: ˆ - CERMICScermics.enpc.fr/~lelievre/CEMRACS/slides/Billaud_Friess.pdf · ¶ Approximation intensor subsetfor u 2X ˇeu2SX ˆX Rank-r canonical tensors: Rr(X) = (Xr i=1 Os =1 ˚

Tensor-based algorithms for the model reduction ofhigh dimensional problems: application to stochastic

fluid problems

M. Billaud [email protected]

Joint work with A. Nouy, O. Zahm

CEMRACSLuminy, 2013

Page 2: ˆ - CERMICScermics.enpc.fr/~lelievre/CEMRACS/slides/Billaud_Friess.pdf · ¶ Approximation intensor subsetfor u 2X ˇeu2SX ˆX Rank-r canonical tensors: Rr(X) = (Xr i=1 Os =1 ˚

Introduction Ideal algorithm (IA) Perturbed ideal algorithm (PA) Ad-Re-Di problem Oseen problem Conclusions

General context

High dimensional problem in tensor spaces:

Given b ∈ Y′, seek u ∈ X solution of Lu = b.

• L : X→ Y′ a linear and continuous isomorphism

• X = a⊗s

µ=1 Xµ||·||X (resp. Y) a tensor Hilbert space of dual X′ (resp. Y′).

• X (resp. Y) is equipped with the norm || · ||X (resp. || · ||Y)

Typical problems:• Stochastic partial diUerential equations (SPDE)

• Parametric partial diUerential equations

• High dimensional algebraic systems in tensor format arising from discretization

CEMRACS 2013 M. Billaud Friess (ECN) 2/ 37

Page 3: ˆ - CERMICScermics.enpc.fr/~lelievre/CEMRACS/slides/Billaud_Friess.pdf · ¶ Approximation intensor subsetfor u 2X ˇeu2SX ˆX Rank-r canonical tensors: Rr(X) = (Xr i=1 Os =1 ˚

Introduction Ideal algorithm (IA) Perturbed ideal algorithm (PA) Ad-Re-Di problem Oseen problem Conclusions

Application: Stochastic PDEs arising from Wuids

Problem: Find u : (x, ξ) ∈ Ω×Ξ→ u(x, ξ) in X = L2(Ξ, dPξ;V) solution of

L(u(·, ξ); ξ) = b(ξ), a.s. .

with uncertainties represented by m ∈ N random variables on (Ξ,B, Pξ): ξ ∈ Rm, and V anHilbert space of functions on Ω ⊂ Rd.

Considered examples:

• Reaction-Advection-DiUusion problem: non-symetric problem

L = −ν4+ c(ξ) · ∇+ a(ξ) with X = L2(Ξ, dPξ;H10(Ω))

• Oseen problem: non-symetric saddle point problem

L =

(−ν(ξ)4+ a(ξ) · ∇ ∇

∇· 0

)with X = L2(Ξ, dPξ;H1

0(Ω))× L2(Ξ, dPξ; L2(Ω))

DiXculty: Curse of dimensionality;Model reduction

• Reduced basis approaches [Rozza]

• Low rank tensor approximation (Proper Generalized Decomposition) [Nouy]

CEMRACS 2013 M. Billaud Friess (ECN) 3/ 37

Page 4: ˆ - CERMICScermics.enpc.fr/~lelievre/CEMRACS/slides/Billaud_Friess.pdf · ¶ Approximation intensor subsetfor u 2X ˇeu2SX ˆX Rank-r canonical tensors: Rr(X) = (Xr i=1 Os =1 ˚

Introduction Ideal algorithm (IA) Perturbed ideal algorithm (PA) Ad-Re-Di problem Oseen problem Conclusions

Low rank approximation

¶ Approximation in tensor subset for u ∈ X ≈ u ∈ SX ⊂ X

Rank-r canonical tensors:

Rr(X) =

r∑

i=1

s⊗µ=1

φµi ;φµi ∈ Xµ

with X = a

s⊗µ=1

||·||X

Other: Tucker tensors, Tensor train tensors, Hierarchical Tucker tensors [Khoromskij]

· Best approximation in SX

u ∈ ΠSX(u) = arg minv∈SX||v− u|| ; u ∈ arg min

v∈SX||Lv− b||∗

¸ Progressive constructions of approximations with Greedy approach [Temlyakov]

Limitation of the classical approach:× Bad convergence rate for usual norm || · ||∗ (ex.: || · ||2 for non symmetric operator L)

× Weakly coercive problems

CEMRACS 2013 M. Billaud Friess (ECN) 4/ 37

Page 5: ˆ - CERMICScermics.enpc.fr/~lelievre/CEMRACS/slides/Billaud_Friess.pdf · ¶ Approximation intensor subsetfor u 2X ˇeu2SX ˆX Rank-r canonical tensors: Rr(X) = (Xr i=1 Os =1 ˚

Introduction Ideal algorithm (IA) Perturbed ideal algorithm (PA) Ad-Re-Di problem Oseen problem Conclusions

Low rank approximation

¶ Approximation in tensor subset for u ∈ X ≈ u ∈ SX ⊂ X

Rank-r canonical tensors:

Rr(X) =

r∑

i=1

φi ⊗ ψi;φi ∈ V, ψi ∈ S

with X = L2(Ξ, dPξ)⊗ V = S⊗ V

; Deterministic/Stochastic separation s = 2Other: Tucker tensors, Tensor train tensors, Hierarchical Tucker tensors [Khoromskij]

· Best approximation in SX

u ∈ ΠSX(u) = arg minv∈SX||v− u|| ; u ∈ arg min

v∈SX||Lv− b||∗

¸ Progressive constructions of approximations with Greedy approach [Temlyakov]

Limitation of the classical approach:× Bad convergence rate for usual norm || · ||∗ (ex.: || · ||2 for non symmetric operator L)

× Weakly coercive problems

CEMRACS 2013 M. Billaud Friess (ECN) 4/ 37

Page 6: ˆ - CERMICScermics.enpc.fr/~lelievre/CEMRACS/slides/Billaud_Friess.pdf · ¶ Approximation intensor subsetfor u 2X ˇeu2SX ˆX Rank-r canonical tensors: Rr(X) = (Xr i=1 Os =1 ˚

Introduction Ideal algorithm (IA) Perturbed ideal algorithm (PA) Ad-Re-Di problem Oseen problem Conclusions

Fig. Reaction-diUusion-advection problem : comparison of convergence error for|| · ||∗ = || · ||2 for a R20 approximation

0 2 4 6 8 10 12 14 16 18 2010−5

10−4

10−3

10−2

10−1

100

r

||u−

u|| 2/||u|| 2

ReferenceApproximation

CEMRACS 2013 M. Billaud Friess (ECN) 5/ 37

Page 7: ˆ - CERMICScermics.enpc.fr/~lelievre/CEMRACS/slides/Billaud_Friess.pdf · ¶ Approximation intensor subsetfor u 2X ˇeu2SX ˆX Rank-r canonical tensors: Rr(X) = (Xr i=1 Os =1 ˚

Introduction Ideal algorithm (IA) Perturbed ideal algorithm (PA) Ad-Re-Di problem Oseen problem Conclusions

Main goal of the talk

Goal:

Present an approximation strategy to solve high dimensional PDEs (ex.: stochastic)in tensor subsets relying on best approximation problem formulated using ideal norms.

1 Ideal algorithm (IA)

2 Perturbed ideal algorithm (PA)

3 Ad-Re-Di problem

4 Oseen problem

5 Conclusions

CEMRACS 2013 M. Billaud Friess (ECN) 6/ 37

Page 8: ˆ - CERMICScermics.enpc.fr/~lelievre/CEMRACS/slides/Billaud_Friess.pdf · ¶ Approximation intensor subsetfor u 2X ˇeu2SX ˆX Rank-r canonical tensors: Rr(X) = (Xr i=1 Os =1 ˚

Introduction Ideal algorithm (IA) Perturbed ideal algorithm (PA) Ad-Re-Di problem Oseen problem Conclusions

Outline

1 Ideal algorithm (IA)

2 Perturbed ideal algorithm (PA)

3 Ad-Re-Di problem

4 Oseen problem

5 Conclusions

CEMRACS 2013 M. Billaud Friess (ECN) 7/ 37

Page 9: ˆ - CERMICScermics.enpc.fr/~lelievre/CEMRACS/slides/Billaud_Friess.pdf · ¶ Approximation intensor subsetfor u 2X ˇeu2SX ˆX Rank-r canonical tensors: Rr(X) = (Xr i=1 Os =1 ˚

Introduction Ideal algorithm (IA) Perturbed ideal algorithm (PA) Ad-Re-Di problem Oseen problem Conclusions

Ideal norm

Problem:

Given b ∈ Y′ Vnd the solution u ofLu = b

• L : X→ Y′ linear operator of adjoint L∗ : Y→ X′, b ∈ Y′

• Riesz operators RX : X→ X′ (resp. RY : Y→ Y′ )

∀u,w ∈ X 〈u,w〉X = 〈u, RXw〉X,X′ = 〈RXu,w〉X′,X = 〈RXu, RXu〉X′

• L is continuous

supv∈X

supw∈Y

〈Lv,w〉Y′,Y‖v‖X‖w‖Y

= β > 0.

• L is weakly coercive

infv∈X

supw∈Y

〈Lv,w〉Y′,Y‖v‖X‖w‖Y

= α > 0.

• We have the stability condition for L

α||Lu||Y′ ≤ ||u||X ≤ β||Lu||Y′

å Under these assumptions L is an isomorphism [Ern].

CEMRACS 2013 M. Billaud Friess (ECN) 8/ 37

Page 10: ˆ - CERMICScermics.enpc.fr/~lelievre/CEMRACS/slides/Billaud_Friess.pdf · ¶ Approximation intensor subsetfor u 2X ˇeu2SX ˆX Rank-r canonical tensors: Rr(X) = (Xr i=1 Os =1 ˚

Introduction Ideal algorithm (IA) Perturbed ideal algorithm (PA) Ad-Re-Di problem Oseen problem Conclusions

How to choose the norms || · ||X and || · ||Y′ ? [Cohen,Dahmen]

|| · ||X = ||L · ||Y′ ⇔ || · ||X′ = ||L∗ · ||Y

• This choice leads to a problem ideally conditioned i.e. α = β = 1.

• Possibility to choose a priori and arbitrary || · ||X ; "Goal oriented approximations"

Interpretation: Such a choice implies ∀v,w ∈ X

〈v,w〉X = 〈Lv, Lw〉Y′ = 〈Lv, R−1Y Lw〉Y′,Y = 〈v, R−1

X L∗R−1Y Lw〉X

⇒ IX = R−1X L∗R−1

Y L⇔ RY = LR−1X L∗ ⇔ RX = L∗R−1

Y L

Example: algebraic system

• L ∈ Rn×n, u, b ∈ Rn

• X = Y = Rn

• RX = I, RY = LL∗

• ||u||Y = ||L∗u||2 = ||L∗u||X

CEMRACS 2013 M. Billaud Friess (ECN) 9/ 37

Page 11: ˆ - CERMICScermics.enpc.fr/~lelievre/CEMRACS/slides/Billaud_Friess.pdf · ¶ Approximation intensor subsetfor u 2X ˇeu2SX ˆX Rank-r canonical tensors: Rr(X) = (Xr i=1 Os =1 ˚

Introduction Ideal algorithm (IA) Perturbed ideal algorithm (PA) Ad-Re-Di problem Oseen problem Conclusions

Best approximation problem

¶ Best approximation problem u ≈ u in SX ⊂ X

u ∈ ΠSX(u) = arg minv∈SX||v− u||X ⇔ ||Lu− b||Y′ = min

v∈SX||Lv− b||Y′

· Equivalent problem:

minv∈SX||Lv− b||Y′

• Non computable norm || · ||Y′

¸ Exact gradient type algorithm

We seek uk, ykk≥0 ⊂ SX × Y given u0h = 0 s.t.yk = R−1

Y (Luk − b)),

uk+1 ∈ ΠSX (uk − R−1X L∗yk).

• This ideal gradient type algorithm converges in one iteration.• R−1

Y (Lv− b) not aUordable in practice !• How to compute practically ΠSX ?

CEMRACS 2013 M. Billaud Friess (ECN) 10/ 37

Page 12: ˆ - CERMICScermics.enpc.fr/~lelievre/CEMRACS/slides/Billaud_Friess.pdf · ¶ Approximation intensor subsetfor u 2X ˇeu2SX ˆX Rank-r canonical tensors: Rr(X) = (Xr i=1 Os =1 ˚

Introduction Ideal algorithm (IA) Perturbed ideal algorithm (PA) Ad-Re-Di problem Oseen problem Conclusions

Best approximation problem

¶ Best approximation problem u ≈ u in SX ⊂ X

u ∈ ΠSX(u) = arg minv∈SX||v− u||X ⇔ ||Lu− b||Y′ = min

v∈SX||Lv− b||Y′

· Equivalent problem:

minv∈SX||R−1

Y (Lv− b)||Y

• Non computable norm || · ||Y′

¸ Exact gradient type algorithm

We seek uk, ykk≥0 ⊂ SX × Y given u0h = 0 s.t.yk = R−1

Y (Luk − b)),

uk+1 ∈ ΠSX (uk − R−1X L∗yk).

• This ideal gradient type algorithm converges in one iteration.• R−1

Y (Lv− b) not aUordable in practice !• How to compute practically ΠSX ?

CEMRACS 2013 M. Billaud Friess (ECN) 10/ 37

Page 13: ˆ - CERMICScermics.enpc.fr/~lelievre/CEMRACS/slides/Billaud_Friess.pdf · ¶ Approximation intensor subsetfor u 2X ˇeu2SX ˆX Rank-r canonical tensors: Rr(X) = (Xr i=1 Os =1 ˚

Introduction Ideal algorithm (IA) Perturbed ideal algorithm (PA) Ad-Re-Di problem Oseen problem Conclusions

Outline

1 Ideal algorithm (IA)

2 Perturbed ideal algorithm (PA)

3 Ad-Re-Di problem

4 Oseen problem

5 Conclusions

CEMRACS 2013 M. Billaud Friess (ECN) 11/ 37

Page 14: ˆ - CERMICScermics.enpc.fr/~lelievre/CEMRACS/slides/Billaud_Friess.pdf · ¶ Approximation intensor subsetfor u 2X ˇeu2SX ˆX Rank-r canonical tensors: Rr(X) = (Xr i=1 Os =1 ˚

Introduction Ideal algorithm (IA) Perturbed ideal algorithm (PA) Ad-Re-Di problem Oseen problem Conclusions

First step

To compute:

Find yk ∈ Y s.t. yk = R−1Y (Luk − b)

• Λδ : Y→ Y is a non linear mapping s.t ∀y ∈ L(SX − b); v ∈ SX we have

||Λδ(y)− y||Y′ ≤ δ||y||Y′ , δ ∈ (0, 1)

• yk is an approximation of R−1Y L(uk − b) "with" a precision δ

How ?• Preconditionned iterative solver [Powell & al.]

• Greedy construction in a Vxed small low-rank subset (ex.: R1) [Temlyakov]

CEMRACS 2013 M. Billaud Friess (ECN) 12/ 37

Page 15: ˆ - CERMICScermics.enpc.fr/~lelievre/CEMRACS/slides/Billaud_Friess.pdf · ¶ Approximation intensor subsetfor u 2X ˇeu2SX ˆX Rank-r canonical tensors: Rr(X) = (Xr i=1 Os =1 ˚

Introduction Ideal algorithm (IA) Perturbed ideal algorithm (PA) Ad-Re-Di problem Oseen problem Conclusions

First step

To compute:

Find yk ∈ Y s.t. yk = ΛδR−1Y (Luk − b)

• Λδ : Y→ Y is a non linear mapping s.t ∀y ∈ L(SX − b); v ∈ SX we have

||Λδ(y)− y||Y′ ≤ δ||y||Y′ , δ ∈ (0, 1)

• yk is an approximation of R−1Y L(uk − b) "with" a precision δ

How ?• Preconditionned iterative solver [Powell & al.]

• Greedy construction in a Vxed small low-rank subset (ex.: R1) [Temlyakov]

CEMRACS 2013 M. Billaud Friess (ECN) 12/ 37

Page 16: ˆ - CERMICScermics.enpc.fr/~lelievre/CEMRACS/slides/Billaud_Friess.pdf · ¶ Approximation intensor subsetfor u 2X ˇeu2SX ˆX Rank-r canonical tensors: Rr(X) = (Xr i=1 Os =1 ˚

Introduction Ideal algorithm (IA) Perturbed ideal algorithm (PA) Ad-Re-Di problem Oseen problem Conclusions

Second step

To compute:

Find uk ∈ SkX s.t. uk+1 ∈ Cε(uk − R−1

X (L∗yk)).

• Cε : X→ X is a non linear mapping giving an approx. of uk − R−1X (L∗yk)

− either with a Vxed rank r,Cε = ΠSX .

å In that case SkX = SX is Vxed at each iteration.

− or with a Vxed precision ε ∈ (0, 1) s.t.

Cε = ΠSkXand ||Cε(v)− v||X ≤ ε||v||X, ∀v ∈ X

å In that case SkX = Rrk(X) may change at each iteration.

How ?

1. s = 2 : Rank-r SVD, s > 2 : HOSVD, Alternating Minimization Algorithm

2. Greedy construction in a Vxed small low-rank subset: adaptativity, large rank rk, Vxedprecision

CEMRACS 2013 M. Billaud Friess (ECN) 13/ 37

Page 17: ˆ - CERMICScermics.enpc.fr/~lelievre/CEMRACS/slides/Billaud_Friess.pdf · ¶ Approximation intensor subsetfor u 2X ˇeu2SX ˆX Rank-r canonical tensors: Rr(X) = (Xr i=1 Os =1 ˚

Introduction Ideal algorithm (IA) Perturbed ideal algorithm (PA) Ad-Re-Di problem Oseen problem Conclusions

Summary of the algorithm

Set u0 = 0, Z00 = 0 and SY a low rank tensor subset.

Gradient loop: for k = 0 to K do

1. Projection: yk0 = argminy∈Zk0||y− rk||Y; with rk = R−1

Y (Luk − b)

2. Set m = 0;

Loop for Λd: while err(ykm, rk) ≤ δ do

a) m = m + 1;

b) Correction: wkm = arg min

y∈SY||ykm−1 + w− rk||Y;

c) Set Zkm = Zkm−1 + spanwkm;

d) Projection: ykm = arg miny∈Zkm

||y− rk||Y;

3. Compute uk+1 ∈ Cε(uk − R−1X L∗ykm)

Remark: Stopping criterion based on

err(ykm, rk) = ||rk − ykm||Y/||r

k||Y ≤ δ ; ||ykm − ykm+p||Y/||ykm+p||Y ≤ δ

CEMRACS 2013 M. Billaud Friess (ECN) 14/ 37

Page 18: ˆ - CERMICScermics.enpc.fr/~lelievre/CEMRACS/slides/Billaud_Friess.pdf · ¶ Approximation intensor subsetfor u 2X ˇeu2SX ˆX Rank-r canonical tensors: Rr(X) = (Xr i=1 Os =1 ˚

Introduction Ideal algorithm (IA) Perturbed ideal algorithm (PA) Ad-Re-Di problem Oseen problem Conclusions

Greedy approximations for Cε:

Weak greedy algorithm: [Temlyakov]

Let SX be a low rank tensor subset, Vnd umm≥1, with um ∈ SX + · · ·+ SX = SmX ⊂ Xfor u0 = 0

1) Correction wm: wm ∈ arg minv∈SX||ΛδR−1

Y (Lv− b + Lum−1)||Y

2) Update: um = um−1 + wm ∈ SmX

3) Stopping criterion: If m = r or ||Cε(v)− v||X ≤ ε||v||X then uk = um

Convergence result: [Billaud,Nouy,Zahm]

The weak greedy algorithm for a Vxed rank subset SX converges under some criterionsdepending on δ.

CEMRACS 2013 M. Billaud Friess (ECN) 15/ 37

Page 19: ˆ - CERMICScermics.enpc.fr/~lelievre/CEMRACS/slides/Billaud_Friess.pdf · ¶ Approximation intensor subsetfor u 2X ˇeu2SX ˆX Rank-r canonical tensors: Rr(X) = (Xr i=1 Os =1 ˚

Introduction Ideal algorithm (IA) Perturbed ideal algorithm (PA) Ad-Re-Di problem Oseen problem Conclusions

Convergence of the overall algorithm ...

... with Vxed rank:

Proposition: [Billaud,Nouy,Zahm]

Assuming 0 < 2δ < 1 then the sequence ukk≥0 is s.t.

||Ek||X ≤ (2δ)k||E0||X +1

1− 2δ||EΠ||X, ∀k > 0

with Ek = (uk − u) and EΠ = (u−ΠSX (u))

... with Vxed precision:

Proposition:

Assuming 0 < δ(1 + ε) < 1 then the sequence ukk≥0 is s.t.

||Ek||X ≤ ((1 + ε)δ)k||E0||X +ε

1− (1 + ε)δ||u||X, ∀k > 0.

Remarks:• Convergence in two phases : decreasing and stagnation for both kind of algorithms.

• Practical interest of the second algorithm

CEMRACS 2013 M. Billaud Friess (ECN) 16/ 37

Page 20: ˆ - CERMICScermics.enpc.fr/~lelievre/CEMRACS/slides/Billaud_Friess.pdf · ¶ Approximation intensor subsetfor u 2X ˇeu2SX ˆX Rank-r canonical tensors: Rr(X) = (Xr i=1 Os =1 ˚

Introduction Ideal algorithm (IA) Perturbed ideal algorithm (PA) Ad-Re-Di problem Oseen problem Conclusions

Outline

1 Ideal algorithm (IA)

2 Perturbed ideal algorithm (PA)

3 Ad-Re-Di problem

4 Oseen problem

5 Conclusions

CEMRACS 2013 M. Billaud Friess (ECN) 17/ 37

Page 21: ˆ - CERMICScermics.enpc.fr/~lelievre/CEMRACS/slides/Billaud_Friess.pdf · ¶ Approximation intensor subsetfor u 2X ˇeu2SX ˆX Rank-r canonical tensors: Rr(X) = (Xr i=1 Os =1 ˚

Introduction Ideal algorithm (IA) Perturbed ideal algorithm (PA) Ad-Re-Di problem Oseen problem Conclusions

Test case

Stochastic reaction-advection-diUusion problem:We seek u(x, ξ) satisfying a.e. in Ξ

−4u + c∇u + au = f, in Ωu = 0, on ∂Ω

• f(x) = 1Ω1(x)− 1Ω2(x)

• c(x, ξ) = ξ1(12 − x2, x1 − 1

2 ), ξ1 ∈ U(−350, 350)• a(x, ξ) = ξ2, ξ2 ∈ lnU(0.1, 100)

Ω

Ω2

Ω1

x1 = 0x2 = 0

x2 = 1

x1 = 1

Discretization:• Approx. x = (x1, x2): Q1 with N = 1521 nodes ;VN ⊂ V

• Approx. ξ: piecewise polynomial chaos of degree 5 with the dimension S = 72 ;SP ⊂ S

Algebraic system: dimension s = 2

Lu = b

with u ∈ X = RN ⊗ RP, L ∈ RN×N ⊗ RP×P and b ∈ RN ⊗ RP.

CEMRACS 2013 M. Billaud Friess (ECN) 18/ 37

Page 22: ˆ - CERMICScermics.enpc.fr/~lelievre/CEMRACS/slides/Billaud_Friess.pdf · ¶ Approximation intensor subsetfor u 2X ˇeu2SX ˆX Rank-r canonical tensors: Rr(X) = (Xr i=1 Os =1 ˚

Introduction Ideal algorithm (IA) Perturbed ideal algorithm (PA) Ad-Re-Di problem Oseen problem Conclusions

Tested norms:

1. Canonical norm: ||v||22 =P∑

i=1

N∑i=1

(vij)2 : RX = IN ⊗ IP.

2. Weighted norm: ||v||2w =P∑

i=1

N∑i=1

(w(xi)vij)2 : RX = diag(w(xi))2 ⊗ IP,

w(x) = 1000 · 1D(x) + 1Ω\D(x).

; "Goal oriented": measure a quantity of interest localized in D:

Q(v) =1|D|

∫Dvdx.

Ω

D

x1 = 0x2 = 0

x2 = 1

x1 = 1

Confronted approaches: Approximation in Rr(X), with r Vxed.

X Singular Value Decomposition (SVD): Ideal rank-r reference approximation

X Classical algorithm (CA):minv∈SX||Lv− b||2

X Perturbed algorithm (PA) : with δ ∈ 0.01, 0.05, 0.2, 0.5, 0.9 with Vxed rank r

CEMRACS 2013 M. Billaud Friess (ECN) 19/ 37

Page 23: ˆ - CERMICScermics.enpc.fr/~lelievre/CEMRACS/slides/Billaud_Friess.pdf · ¶ Approximation intensor subsetfor u 2X ˇeu2SX ˆX Rank-r canonical tensors: Rr(X) = (Xr i=1 Os =1 ˚

Introduction Ideal algorithm (IA) Perturbed ideal algorithm (PA) Ad-Re-Di problem Oseen problem Conclusions

First results

Fig.1 Convergence error for || · ||2 and || · ||w with SX = R20(X), − δ = 0.9, − δ = 0.5,− δ = 0.2, −δ = 0.05, − δ = 0.01, − SVD, - - - CA

0 5 10 15 20

10−4

10−2

100

r

||u−

u|| 2/||u|| 2

0 5 10 15 2010−7

10−4

10−1

r

||u−

u|| w/||u|| w

• Better result with PA then for CA for both norms

• Convergence closer to SVD for decreasing δ → 0

• Convergence slightly deteriorated for the w-norm

CEMRACS 2013 M. Billaud Friess (ECN) 20/ 37

Page 24: ˆ - CERMICScermics.enpc.fr/~lelievre/CEMRACS/slides/Billaud_Friess.pdf · ¶ Approximation intensor subsetfor u 2X ˇeu2SX ˆX Rank-r canonical tensors: Rr(X) = (Xr i=1 Os =1 ˚

Introduction Ideal algorithm (IA) Perturbed ideal algorithm (PA) Ad-Re-Di problem Oseen problem Conclusions

Interest of a weighted norm

Fig.2 Convergence error with rank for both || · ||2 and || · ||w with− uw, − u2

5 10 15 20

10−4

10−2

100

r

||u−

u|| 2/||u|| 2

5 10 15 20

10−4

10−2

100

r||u−

u|| w/||u|| w

• Similar approximations when compared to the reference with the 2-norm

• uw is a better approximation when computed and compared to reference with w-norm.

CEMRACS 2013 M. Billaud Friess (ECN) 21/ 37

Page 25: ˆ - CERMICScermics.enpc.fr/~lelievre/CEMRACS/slides/Billaud_Friess.pdf · ¶ Approximation intensor subsetfor u 2X ˇeu2SX ˆX Rank-r canonical tensors: Rr(X) = (Xr i=1 Os =1 ˚

Introduction Ideal algorithm (IA) Perturbed ideal algorithm (PA) Ad-Re-Di problem Oseen problem Conclusions

Fig.3 Comparison of Vrst spatial modes of u2 and uw

• DiUerent spatial modes of uw and u2• Features diUerently captured

CEMRACS 2013 M. Billaud Friess (ECN) 22/ 37

Page 26: ˆ - CERMICScermics.enpc.fr/~lelievre/CEMRACS/slides/Billaud_Friess.pdf · ¶ Approximation intensor subsetfor u 2X ˇeu2SX ˆX Rank-r canonical tensors: Rr(X) = (Xr i=1 Os =1 ˚

Introduction Ideal algorithm (IA) Perturbed ideal algorithm (PA) Ad-Re-Di problem Oseen problem Conclusions

Fig.4 Comparison of the mean value and variance of u2 and uw

5 10 15 2010−14

10−7

100

r

error

Mean of Q(u)

uwu2

5 10 15 2010−14

10−7

100

r

error

Variance of Q(u)

uwu2

• Better mean and variance for uw than u2

CEMRACS 2013 M. Billaud Friess (ECN) 23/ 37

Page 27: ˆ - CERMICScermics.enpc.fr/~lelievre/CEMRACS/slides/Billaud_Friess.pdf · ¶ Approximation intensor subsetfor u 2X ˇeu2SX ˆX Rank-r canonical tensors: Rr(X) = (Xr i=1 Os =1 ˚

Introduction Ideal algorithm (IA) Perturbed ideal algorithm (PA) Ad-Re-Di problem Oseen problem Conclusions

Gradient algorithm

Fig.5 Convergence study of the gradient type algorithm for diUerent δ for the 2-norm and SX = R10(X)

5 10 15 20 25 30100

101

102

k

||u−

uk|| 2/||u−

ΠSX(u

)|| 2

δ = 9.0e− 01δ = 5.0e− 01δ = 2.0e− 01δ = 5.0e− 02δ = 1.0e− 02

• Convergence behavior consistent with theoric result (decreasing +stagnation)

• Good result, either for "large" δ (near 0.5)

• Similar result for both 2-norm and w-norm

CEMRACS 2013 M. Billaud Friess (ECN) 24/ 37

Page 28: ˆ - CERMICScermics.enpc.fr/~lelievre/CEMRACS/slides/Billaud_Friess.pdf · ¶ Approximation intensor subsetfor u 2X ˇeu2SX ˆX Rank-r canonical tensors: Rr(X) = (Xr i=1 Os =1 ˚

Introduction Ideal algorithm (IA) Perturbed ideal algorithm (PA) Ad-Re-Di problem Oseen problem Conclusions

Tab.1 Measured convergence rates of the linear phase for gradient type algorithm.

HHHHr

δ0.90 0.50 0.20 0.05 0.01

4 0.78 0.36 ≈ 0 ≈ 0 ≈ 010 0.82 0.42 0.183 ≈ 0 ≈ 020 0.86 0.48 0.197 0.051 0.011

Tab.2 Measured stagnation values for gradient type algorithm.

δ 0.90 0.50 0.20 0.05 0.012δ/(1− 2δ) - - 6.6e-1 1.1e-1 2.1e-2

4 3.3e-1 5.6e-2 4.9e-3 3.5e-4 3.0e-510 5.2e-1 1.3e-1 1.7e-2 1.8e-3 3.3e-520 6.4e-1 1.5e-1 1.9e-2 1.2e-3 7.3e-5

• Convergence rate near δ than 2δ

• Stagnation values overestimated: smaller than theoretical bound 2δ/(1− 2δ)

CEMRACS 2013 M. Billaud Friess (ECN) 25/ 37

Page 29: ˆ - CERMICScermics.enpc.fr/~lelievre/CEMRACS/slides/Billaud_Friess.pdf · ¶ Approximation intensor subsetfor u 2X ˇeu2SX ˆX Rank-r canonical tensors: Rr(X) = (Xr i=1 Os =1 ˚

Introduction Ideal algorithm (IA) Perturbed ideal algorithm (PA) Ad-Re-Di problem Oseen problem Conclusions

Outline

1 Ideal algorithm (IA)

2 Perturbed ideal algorithm (PA)

3 Ad-Re-Di problem

4 Oseen problem

5 Conclusions

CEMRACS 2013 M. Billaud Friess (ECN) 26/ 37

Page 30: ˆ - CERMICScermics.enpc.fr/~lelievre/CEMRACS/slides/Billaud_Friess.pdf · ¶ Approximation intensor subsetfor u 2X ˇeu2SX ˆX Rank-r canonical tensors: Rr(X) = (Xr i=1 Os =1 ˚

Introduction Ideal algorithm (IA) Perturbed ideal algorithm (PA) Ad-Re-Di problem Oseen problem Conclusions

General framework

Problem:

Given b = (f, g) ∈ Y′ = X′1 ×M′2, Vnd (u, p) ∈ X = X2 ×M1 solution ofAu + B∗1 p = f,B2u = g, ⇔ Lu = b.

• Continous and linear operators A : X2 → X′1, Bi : Xi → M′i with A∗ : X1 → X′2, B∗i : Mi → Xi

• A is weakly coercive:

infu∈K2

supv∈K1

〈Av,w〉X′1,X1||u||X2 ||v||X1

≥ α > 0.

with Ki =v ∈ Xi; ∀q ∈ Mi; 〈Biv, q〉M′

i ,Mi= 0

• Bi satisVes the inf-sup condition:

infq∈Mi

supv∈Xi

〈Biv, q〉M′i ,Mi

||v||Xi ||q||Mi

= βi > 0.

• The product space X is equipped with 〈·, ·〉X = 〈·, ·〉X2 + 〈·, ·〉M1 .

å Under these assumptions L is an isomorphism [Bernardi,Ern].

CEMRACS 2013 M. Billaud Friess (ECN) 27/ 37

Page 31: ˆ - CERMICScermics.enpc.fr/~lelievre/CEMRACS/slides/Billaud_Friess.pdf · ¶ Approximation intensor subsetfor u 2X ˇeu2SX ˆX Rank-r canonical tensors: Rr(X) = (Xr i=1 Os =1 ˚

Introduction Ideal algorithm (IA) Perturbed ideal algorithm (PA) Ad-Re-Di problem Oseen problem Conclusions

Best approximation problem

¶ Best approximation problem in SX2 ⊂ X2, and in SM1 ⊂ M1

ΠSX2(u)×ΠSM1

(p) = arg min(v,q)∈SX2×SM1

||v− u||2X2 + ||q− p||2M1

· Equivalent approximation problem:arg min

(v,q)∈SX2×SM1

||R−1Y (L(v, q)− (f, g))||Y

¸ Perturbed gradient type algorithm

Seek uk, pk, ykk≥0 ⊂ SX2 × SM1 × Y given u0, p0 s.t.yk = (vk, qk) = ΛδR−1

Y (Luk − b),

uk+1 ∈ ΠSX2(uk − R−1

X2(A∗vk + B∗2 q

k)),

pk+1 ∈ ΠSM1(pk − R−1

M1(B1vk)).

• DiUerent approximations are constructed for p and u.

• Problem fully coupled when computing yk.

• Perturbed algorithm:

1. approximation with "δ-precision" of yk

2. approximation with a Vxed rank (ru, rp) or Vxed precision (η, µ) for u and p separately.

CEMRACS 2013 M. Billaud Friess (ECN) 28/ 37

Page 32: ˆ - CERMICScermics.enpc.fr/~lelievre/CEMRACS/slides/Billaud_Friess.pdf · ¶ Approximation intensor subsetfor u 2X ˇeu2SX ˆX Rank-r canonical tensors: Rr(X) = (Xr i=1 Os =1 ˚

Introduction Ideal algorithm (IA) Perturbed ideal algorithm (PA) Ad-Re-Di problem Oseen problem Conclusions

Convergence results

Proposition 1: Vxed rank

Assuming 0 < δ < 1/√8 then the sequence uk, pkk≥0 is s.t.

||Ek||X ≤ (√8δ)k||E0||X +

1

1−√8δ||EΠ||X, ∀k > 0

with Ek = (uk − u, pk − p), EΠ = (u−ΠSX2(u), p−ΠSM1

(p))

Proposition 2: Vxed precision η, µ

Assuming 0 < δ(1 + ε) < 1/√2 then the sequence uk, pkk≥0 is s.t.

||Ek||X ≤ (√2δ(1 + ε))k||E0||X +

ε

1−√2δ(1 + ε)

||u||X, ∀k > 0

with ε = max(η, µ), u = (u, p)

• Two phases of convergence : decreasing, stagnation

• A priori estimate when iterates are computed with Vxed precision depend on the more coarseapproximation.

• Global error estimation due to R−1Y

CEMRACS 2013 M. Billaud Friess (ECN) 29/ 37

Page 33: ˆ - CERMICScermics.enpc.fr/~lelievre/CEMRACS/slides/Billaud_Friess.pdf · ¶ Approximation intensor subsetfor u 2X ˇeu2SX ˆX Rank-r canonical tensors: Rr(X) = (Xr i=1 Os =1 ˚

Introduction Ideal algorithm (IA) Perturbed ideal algorithm (PA) Ad-Re-Di problem Oseen problem Conclusions

Test case

Oseen equations with uncertain parameters:

Given (Ξ,B(Ξ), Pξ) a probability space, Vnd

u ∈ V = L2(Ξ; [H1(Ω)]d), p ∈ P = L2(Ξ; L2(Ω))

satisfying a.e. in Ξ−ν4u + a∇u + p = 0, in Ω

∇ · u = 0, in Ωu = u, on ∂Ω

Ω

x1 = 0x2 = 0

x2 = 1

x1 = 1

u = (1, 0)

u = (0, 0) u = (0, 0)

u = (0, 0)

Uncertainties on a and ν are represented by random variables

• ν(ξ) = ν0 + ξ1, ξ1 ∼ U(−1, 1)• a(ξ) = a0(1 + ξ2), ξ2 ∼ N(0, 1)

Discretization: Stochastic Galerkin approach

• Space: P2 − P1 ; n = 2189, m = 568 nodes

• Uncertainties: polynomial chaos of degree 3 ; p = 10

å Vn ⊗ Sp ⊂ V and Vm ⊗ Sp ⊂ P

CEMRACS 2013 M. Billaud Friess (ECN) 30/ 37

Page 34: ˆ - CERMICScermics.enpc.fr/~lelievre/CEMRACS/slides/Billaud_Friess.pdf · ¶ Approximation intensor subsetfor u 2X ˇeu2SX ˆX Rank-r canonical tensors: Rr(X) = (Xr i=1 Os =1 ˚

Introduction Ideal algorithm (IA) Perturbed ideal algorithm (PA) Ad-Re-Di problem Oseen problem Conclusions

Test case

Oseen equations with uncertain parameters:

Given (Ξ,B(Ξ), Pξ) a probability space, Vnd

u ∈ V = L2(Ξ; [H10(Ω)]d), p ∈ P = L2(Ξ; L2(Ω))

satisfying a.e. in Ξ−ν4u + a∇u + p = 0, in Ω

∇ · u = 0, in Ωu = 0, on ∂Ω

Ω

x1 = 0x2 = 0

x2 = 1

x1 = 1

u = (1, 0)

u = (0, 0) u = (0, 0)

u = (0, 0)

Uncertainties on a and ν are represented by random variables

• ν(ξ) = ν0 + ξ1, ξ1 ∼ U(−1, 1)• a(ξ) = a0(1 + ξ2), ξ2 ∼ N(0, 1)

Discretization: Stochastic Galerkin approach

• Space: P2 − P1 ; n = 2189, m = 568 nodes

• Uncertainties: polynomial chaos of degree 3 ; p = 10

å Vn ⊗ Sp ⊂ V and Vm ⊗ Sp ⊂ P

CEMRACS 2013 M. Billaud Friess (ECN) 30/ 37

Page 35: ˆ - CERMICScermics.enpc.fr/~lelievre/CEMRACS/slides/Billaud_Friess.pdf · ¶ Approximation intensor subsetfor u 2X ˇeu2SX ˆX Rank-r canonical tensors: Rr(X) = (Xr i=1 Os =1 ˚

Introduction Ideal algorithm (IA) Perturbed ideal algorithm (PA) Ad-Re-Di problem Oseen problem Conclusions

Algebraic system: dimension s = 2

Find u ∈ X2 = Rn ⊗ Rp and p ∈ M1 = Rn ⊗ RP s.t.(A B∗

B 0

)(up

)=

(fg

)with A ∈ Rn×n ⊗ RP×P, B ∈ Rm×n ⊗ RP×P and f ∈ Rn ⊗ RP, g ∈ Rm ⊗ RP.

Canonical norm: ||v||22 =

P∑i=1

N∑i=1

(vij)2

⇒ RX2 = In ⊗ IP, RM1 = Im ⊗ IP ⇒ RY =

(AR−1

X2 A∗ + B∗R−1

M1B AR−1

X2 B∗

BR−1X2 A

∗ BR−1X2 B∗

).

Confronted approaches: Approximations with Vxed precisions η, µ

• Singular Value Decomposition (SVD): Ideal rank-r reference approximation since s = 2

• Perturbed Algorithm (PA): δ ∈ 0.05, 0.1, 0.2, 0.5, 0.7 with η = 1.e− 6 and µ = 1.e− 4

CEMRACS 2013 M. Billaud Friess (ECN) 31/ 37

Page 36: ˆ - CERMICScermics.enpc.fr/~lelievre/CEMRACS/slides/Billaud_Friess.pdf · ¶ Approximation intensor subsetfor u 2X ˇeu2SX ˆX Rank-r canonical tensors: Rr(X) = (Xr i=1 Os =1 ˚

Introduction Ideal algorithm (IA) Perturbed ideal algorithm (PA) Ad-Re-Di problem Oseen problem Conclusions

First results

Fig.6 Comparison of the relative convergence to reference in || · ||2 for both pressure and velocity

0 2 4 6 8 1010−16

10−12

10−8

10−4

100

modes m

||u−

u ref|| X

2/||u

ref||

X2

2 4 6 8 1010−16

10−12

10−8

10−4

100

modes m

||p−

p ref|| M

1/||p

ref||

M1

δ=0.7δ=0.5δ=0.2δ=0.1δ=0.05SVD

• Good agreement with the reference

• InWuence of the pressure approximation on the velocity

CEMRACS 2013 M. Billaud Friess (ECN) 32/ 37

Page 37: ˆ - CERMICScermics.enpc.fr/~lelievre/CEMRACS/slides/Billaud_Friess.pdf · ¶ Approximation intensor subsetfor u 2X ˇeu2SX ˆX Rank-r canonical tensors: Rr(X) = (Xr i=1 Os =1 ˚

Introduction Ideal algorithm (IA) Perturbed ideal algorithm (PA) Ad-Re-Di problem Oseen problem Conclusions

Fig.7 Comparison of both mean and expectations of velocities and pressure for the SVD reference (left)

and the computed approximation PA (right)

• Mean proVles in good agreement with the deterministic proVles

• Good agreement between the two approaches

CEMRACS 2013 M. Billaud Friess (ECN) 33/ 37

Page 38: ˆ - CERMICScermics.enpc.fr/~lelievre/CEMRACS/slides/Billaud_Friess.pdf · ¶ Approximation intensor subsetfor u 2X ˇeu2SX ˆX Rank-r canonical tensors: Rr(X) = (Xr i=1 Os =1 ˚

Introduction Ideal algorithm (IA) Perturbed ideal algorithm (PA) Ad-Re-Di problem Oseen problem Conclusions

Gradient algorithm

Fig.8 Convergence study of the gradient type algorithm for diUerent δ for the 2-norm and with the resp.

precisions η = 10−6, µ = 10−4

0 5 10 15 2010−5

10−4

10−3

10−2

10−1

100

Iterations k

||u−

uk|| X/||u|| X

δ=0.7δ=0.5δ=0.2δ=0.1δ=0.05

Tab.3 Measured convergence rates of the linear phase forgradient type algorithm.

δ 0.70 0.50 0.20 0.1 0.05Th. 0.99 0.71 0.28 0.14 0.07Me. 0.37 0.27 0.06 0.09 0.03

Tab.4 Measured stagnation values for gradient typealgorithm.

δ 0.70 0.50 0.20 0.1 0.05Th. (10−4) 100 3.41 1.39 1.16 1.07Me. (10−5) 5.53 5.28 5.08 5.04 5.03

å Both rate of convergence and stagnation value

overestimated

CEMRACS 2013 M. Billaud Friess (ECN) 34/ 37

Page 39: ˆ - CERMICScermics.enpc.fr/~lelievre/CEMRACS/slides/Billaud_Friess.pdf · ¶ Approximation intensor subsetfor u 2X ˇeu2SX ˆX Rank-r canonical tensors: Rr(X) = (Xr i=1 Os =1 ˚

Introduction Ideal algorithm (IA) Perturbed ideal algorithm (PA) Ad-Re-Di problem Oseen problem Conclusions

Outline

1 Ideal algorithm (IA)

2 Perturbed ideal algorithm (PA)

3 Ad-Re-Di problem

4 Oseen problem

5 Conclusions

CEMRACS 2013 M. Billaud Friess (ECN) 35/ 37

Page 40: ˆ - CERMICScermics.enpc.fr/~lelievre/CEMRACS/slides/Billaud_Friess.pdf · ¶ Approximation intensor subsetfor u 2X ˇeu2SX ˆX Rank-r canonical tensors: Rr(X) = (Xr i=1 Os =1 ˚

Introduction Ideal algorithm (IA) Perturbed ideal algorithm (PA) Ad-Re-Di problem Oseen problem Conclusions

Overview

Conclusion: Robust model reduction approach

X Ideal minimal residual formulation with ideal norms; Convergence improvement

X Validation on SPDEs with application arising from Wuids mechanics

X Arbitrary choice of the norm associated to a speciVc problem; Goal oriented

Need of improvement: Algorithm for R−1Y

• Preconditioned solvers [Powell]

• Approximation of R−1Y in low tensor subsets [Giraldi,Nouy,Legrain]

• Stopping criterion for the dual problem; Still an open problem

Futher exploration:

• "Real" goal oriented explorations; PhD of O.Zahm, in collaboration with A. Nouy

• Uzawa algorithm for HD saddle point problems in collaboration with V. Erlacher

CEMRACS 2013 M. Billaud Friess (ECN) 36/ 37

Page 41: ˆ - CERMICScermics.enpc.fr/~lelievre/CEMRACS/slides/Billaud_Friess.pdf · ¶ Approximation intensor subsetfor u 2X ˇeu2SX ˆX Rank-r canonical tensors: Rr(X) = (Xr i=1 Os =1 ˚

Introduction Ideal algorithm (IA) Perturbed ideal algorithm (PA) Ad-Re-Di problem Oseen problem Conclusions

Overview

Conclusion: Robust model reduction approach

X Ideal minimal residual formulation with ideal norms; Convergence improvement

X Validation on SPDEs with application arising from Wuids mechanics

X Arbitrary choice of the norm associated to a speciVc problem; Goal oriented

Need of improvement: Algorithm for R−1Y

• Preconditioned solvers [Powell]

• Approximation of R−1Y in low tensor subsets [Giraldi,Nouy,Legrain]

• Stopping criterion for the dual problem; Still an open problem

Futher exploration:

• "Real" goal oriented explorations; PhD of O.Zahm, in collaboration with A. Nouy

• Uzawa algorithm for HD saddle point problems in collaboration with V. Erlacher

Thank you for your attention

CEMRACS 2013 M. Billaud Friess (ECN) 36/ 37

Page 42: ˆ - CERMICScermics.enpc.fr/~lelievre/CEMRACS/slides/Billaud_Friess.pdf · ¶ Approximation intensor subsetfor u 2X ˇeu2SX ˆX Rank-r canonical tensors: Rr(X) = (Xr i=1 Os =1 ˚

Bibliography

[Bernardi] C. Bernardi, C. Canuto, Y. Maday: Generalized inf-sup conditions for Chebyshev spectralapproximation of the Stokes problem, SIAM J. Numer. Anal., Vol.25, Issue. 6, pp. 1237–1271, 1988.

[Billaud] Billaud Friess, M., Nouy, A. and Zahm, O., A tensor approximation method based on idealminimal residual formulations for the solution of high dimensional problems, In revision, 2013

[Cohen] Cohen, A., Dahmen, W. and Welper, G., Adaptivity and Variational Stabilization forConvection-DiUusion Equations, Preprint, 2011

[Dahmen] Dahmen, W., Huang, C. and Schwab, C., Adaptative Petrov-Galerkin methods for Vrst ordertransport equations, IGPM Report 321, RWTH Aachen, Volume 150, Pages 425–467, 2011

[Ern] A. Ern and J.-L. Guermond, Theory and practice of Vnite elements, volume 159 of appliedmathematical sciences, 2004.

[Giraldi] L. Giraldi, A. Nouy, G. Legrain: Low-rank approximate inverse for preconditioningtensor-structured linear systems, arXiv:1304.6004, 2013.

[Nouy] Nouy, A., Proper Generalized Decompositions and Separated Representations for the NumericalSolution of High Dimensional Stochastic Problems, Archives of Computational Methods inEngineering, Volume 17, Number 4, 2010

[Powell] C.E. Powell, D.J. Silvester: Preconditionning steady-state Navier-Stokes equations with randomdata, MIMS EPrint 2012.35, 2012.

[Temlyakov] Temlyakov, A., Greedy approximation, 2011