Upload
others
View
20
Download
0
Embed Size (px)
Citation preview
Intro POD Results POD Problem extras
Proper Orthogonal Decomposition (POD)
Saifon Chaturantabut
Advisor: Dr. SorensenCAAM699
Department of Computational and Applied MathematicsRice University
September 5, 2008
Saifon Chaturantabut POD
Intro POD Results POD Problem extras
Outline
1 Introduction/MotivationLow Rank Approximation and PODModel Reduction for ODEs and PDEs
2 Proper Orthogonal Decomposition(POD)Introduction to PODPOD in Euclidean SpacePOD in General Hilbert Space
3 Numerical ResultsSolutions from Full and Reduced Systems of PDE: 1DBurgers’Equation
4 Problem: POD for PDEs with nonlinearitiesNonlinear ApproximationError and Computational Time
Saifon Chaturantabut POD
Intro POD Results POD Problem extras Low Rank Approx and POD Model Reduction for ODEs and PDEs
Problem in finite dimensional spaceGiven y1, . . . , yn ∈ Rm. Let Y = spany1, y2, . . . , yn ⊂ Rm;r = rank(Y )
y1, . . . , yn ∈ Rm possible (almost) linearly dependent⇒ NOT agood basis for Y
Goal: Find orthonormal basis vectors φ1, . . . , φk that bestapproximates Y , for given k < rSolution: Use Low Rank ApproximationForm a matrix of known data:
Y =
| |y1 . . . yn| |
∈ Rm×n.
Saifon Chaturantabut POD
Intro POD Results POD Problem extras Low Rank Approx and POD Model Reduction for ODEs and PDEs
Low Rank ApproximationSingular Value Decomposition (SVD)
Let Y ∈ Rm×n, r = rank(Y ), and k < r .
Problem: Low Rank Approximation
minY‖Y − Y‖2
F : rank(Y ) = k
Solution
Y ∗ = Uk Σk V Tk with min error ‖Y − Y ∗‖2
F =∑r
i=k+1 σ2i
where Y = UΣV T is SVD of Y ; Σ = diag(σ1, . . . , σr ) ∈ Rr×r withσ1 ≥ σ2 ≥ . . . ≥ σr > 0
Optimal orthonormal basis of rank k =: POD basis
Y ∗ =∑k
i=1 σiuivTi ⇒ φk
i=1 = uiki=1
Saifon Chaturantabut POD
Intro POD Results POD Problem extras Low Rank Approx and POD Model Reduction for ODEs and PDEs
EX: Image Compression via SVD
1 % Low Rank Approximation
source: http://demonstrations.wolfram.com/demonstrations.wolfram.com
Saifon Chaturantabut POD
Intro POD Results POD Problem extras Low Rank Approx and POD Model Reduction for ODEs and PDEs
EX: Image Compression via SVD
50 % Low Rank Approximation
source: http://demonstrations.wolfram.com/demonstrations.wolfram.com
Saifon Chaturantabut POD
Intro POD Results POD Problem extras Low Rank Approx and POD Model Reduction for ODEs and PDEs
Model Reduction for ODEs
Full-order system (dim = N)
ddt
y(t) = Ky(t) + g(t) + N(y(t))⇒ y(t)
Reduced-order system (dim = k < N)
Let y = Uk y, for Uk ∈ RN×k , with orthonormal columns:UT
k Uk = I ∈ Rk×k .
Ukddt y(t) = KUk y(t) + g(t) + N(Uk y(t))⇒ d
dt y(t) = UTk KUk︸ ︷︷ ︸ y(t) + UT
k g(t)︸ ︷︷ ︸+UTk N(Uk y(t))
ddt
y(t) = Ky(t) + g(t) + UTk N(Uk y(t))⇒ y(t)
How to construct Uk? .....Use POD!
Saifon Chaturantabut POD
Intro POD Results POD Problem extras Low Rank Approx and POD Model Reduction for ODEs and PDEs
Model Reduction for PDEsEx. Unsteady 1D Burgers’Equation
∂
∂ty(x , t)− ν ∂
2
∂x2 y(x , t) +∂
∂x
(y(x , t)2
2
)= 0 x ∈ [0,1], t ≥ 0
y(0, t) = y(1, t) = 0, t ≥ 0, y(x ,0) = y0(x), x ∈ [0,1],
Discretized system (Galerkin)
Full-order system (dim = N): FE basis ϕiNi=1
Mhddt
y(t) + νKhy(t)−Nh(y(t)) = 0⇒ yh(x , t) =∑N
i=1 ϕi (x)yi (t)
Reduced-order system (dim = k < N): orthonormal basis φiki=1
Mddt
y(t) + νKy(t)− N(y(t)) = 0⇒ yh(x , t) =∑k
i=1 φi (x)yi (t)
How to construct φiki=1? .....Use POD!Saifon Chaturantabut POD
Intro POD Results POD Problem extras Intro POD in Euclidean Space POD in General Hilbert Space
Proper Orthogonal Decomposition(POD)
POD is a method for finding alow-dimensional approximaterepresentation of:
large-scale dynamicalsystems, e.g. signalanalysis, turbulent fluid flowlarge data set, e.g. imageprocessing
≡ SVD in Euclidean space.Extracts basis functionscontaining characteristicsfrom the system of interestGenerally gives a goodapproximation withsubstantially lower dimension
0 0.5 1−2
−1
0 POD basis # 1
0 0.5 1−4
−2
0
2 POD basis # 2
0 0.5 1−4
−2
0
2 POD basis # 3
0 0.5 1−4
−2
0
2 POD basis # 4
0 0.2 0.4 0.6 0.8 1 0
0.5
1
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
t
x
Sol of Full System (FE):dim = 100
y
Figure: Plots of the first 4 POD basisand the space of solutions
Saifon Chaturantabut POD
Intro POD Results POD Problem extras Intro POD in Euclidean Space POD in General Hilbert Space
Definition of POD
Let X be Hilbert Space(I.e. Complete Inner Product Space) withinner product 〈·, ·〉 and norm ‖ · ‖ =
√〈·, ·〉
Given y1, . . . , yn ∈ X . Define yi ≡ snapshot i , ∀i . LetY ≡ spany1, y2, . . . , yn ⊂ X .Given k ≤ n, POD generates a set of orthonormal basis ofdimension k , which minimizes the error from approximating thesnapshots:
POD basis ≡ Optimal solution of:
minφk
i=1
n∑j=1
‖yj − yj‖2, s.t. 〈φi , φj〉 = δij
where yj (x) =∑k
i=1〈yj , φi〉φi (x), an approximation of yj usingφk
i=1.Solve by SVD
Saifon Chaturantabut POD
Intro POD Results POD Problem extras Intro POD in Euclidean Space POD in General Hilbert Space
Derivation for POD in Euclidean Space: E.g. X = Rm
minφki=1
∑nj=1 ‖yj −
∑ki=1(yT
j φi )φi‖22 ≡ E(φ1, ..., φk )
s.t .
φTi φj = δij =
1 if i = j0 if i 6= j i , j = 1, ..., k
Lagrange Function:
L(φ1, . . . , φk , λ11, . . . , λij , . . . , λkk ) = E(φ1, ..., φk ) +k∑
i,j=1
λij (φTi φj − δij )
KKT Necessary Conditions:
∂∂φi
L = 0⇔ ∑n
j=1 yj (yTj φi ) = λiiφi
λij = 0, ifi 6= j
.
φTi φj = δij
Saifon Chaturantabut POD
Intro POD Results POD Problem extras Intro POD in Euclidean Space POD in General Hilbert Space
Optimal Condition: Symmetric m-by-m eigenvalue problem
YY Tφi = λiφi
where λi = λii ,Y =
| |y1 . . . yn| |
for i = 1, . . . , k
Error for POD basis:n∑
j=1
‖yi −k∑
i=1
(yTj φi )φi‖2
2 =r∑
i=k+1
λi
λiφi = YY Tφi =∑n
j=1 yj (yTj φi )⇒ λi =
∑nj=1(yT
j φi )2
Since φTi φj = δij and span(Y ) = spanφ1, . . . , φr (from SVD),
then yj =∑r
i=1(yTj φi )φi , forj = 1, . . . ,n, and
n∑j=1
‖yj −k∑
i=1
(yTj φi )φi‖2
2 =n∑
j=1
r∑i=k+1
(yTj φi )
2 =r∑
i=k+1
λi
Saifon Chaturantabut POD
Intro POD Results POD Problem extras Intro POD in Euclidean Space POD in General Hilbert Space
SOLUTION for POD basis in Rm
Recall Optimality Condition and the POD Error:YY Tφi = λiφi , i = 1, . . . , k∑n
j=1 ‖yj −∑k
i=1(yTj φi )φi‖2
2 =∑r
i=k+1 λi
Optimal solution:
POD basis: φ∗i ki=1 = uik
i=1
Lagrange multiplier: λ∗i = σ2i , i = 1, . . . , k ,
can be obtained by the SVD of Y ∈ Rm×n or EVD ofYY T ∈ Rm×m, i.e.,
Y = UΣV T ⇒ YY T ui = σ2i ui , i = 1, . . . , r ,
where Σ = diag(σ1, . . . , σr ) ∈ Rr×r with σ1 ≥ σ2 ≥ . . . ≥ σr > 0;U = [u1, . . . ,ur ] ∈ Rm×r and V = [v1, . . . , vr ] ∈ Rn×r haveorthonormal columns.This is equivalent to SVD solution for Low Rank Approximation
Saifon Chaturantabut POD
Intro POD Results POD Problem extras Intro POD in Euclidean Space POD in General Hilbert Space
SOLUTION for POD basis in a Hilbert Space XTwo approaches:
1 Define linear symmetric operator F (w) =∑n
j=1〈w , yj〉yj , w ∈ X .Find eigenfunction ui ∈ X :
EVD: F (ui ) = σ2i ui∑n
j=1〈ui , yj〉yj = σ2i ui ∼ (cf. YY T ui = σ2
i ui )
Sol: φ∗i = ui , λ∗i = σ2
i , i = 1, . . . , k
2 Define linear symmetric operator L = [〈yi , yj〉] ∈ Rn×n. Findeigenvector vi ∈ Rn:
EVD: L vi = σ2i vi
[〈yi , yj〉]vi = σ2i vi ∼ (cf. Y T Yvi = σ2
i vi )
Sol: φ∗i = 1σi
∑nj=1(vi )jyj , λ
∗i = σ2
i ∼ Uk = YVk Σ−1k
NOTE: vi ∈ Rn,but ui , yj ∈ X
Saifon Chaturantabut POD
Intro POD Results POD Problem extras Intro POD in Euclidean Space POD in General Hilbert Space
Remark: Practical way for computing the POD basis for discretized PDEs
Let ϕiNi=1 ⊂ X ≡ H1(Ω) be finite element(FE) basis.
FE snapshots(solutions): yjni=j ∈ X at time tjn
j=1.
yj = y(x , tj ) =N∑
i=1
Yij (tj )ϕi (x).
L = [〈yi , yj〉] = [m∑
k,`=1
Yik Yj`〈ϕi , ϕj〉] = Y T MY ∈ Rn×n,
where M = [〈ϕi , ϕj〉] ∈ RN×N .POD basis:
φ∗i =1σi
n∑j=1
(vi )jyj ,
whereL vi = σ2
i vi ,
with v1, v2, . . . , vk corresponding to σ1 ≥ σ2 ≥ . . . ≥ σk > 0.
Saifon Chaturantabut POD
Intro POD Results POD Problem extras Solutions from Full and Reduced Systems of PDE: 1D Burgers’Equation
Numerical Results for PDEs
Recall the 1D Burgers’Equation x ∈ [0, 1], t ≥ 0:∂∂t y(x, t)− ν ∂2
∂x2 y(x, t) + ∂∂x
(y(x,t)2
2
)= 0,
y(0, t) = y(1, t) = 0, t ≥ 0,
y(x, 0) = y0(x), x ∈ [0, 1].
0
0.5
1
0
0.5
1
1.5
20
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
x
Sol of Reduced System (Direct POD):dim = 6
t
y
0
0.5
1
0
0.5
1
1.5
20
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
x
Sol of Full System (Finite Element):dim = 100
t
y
0 10 20 30 40 50 60 7010−15
10−10
10−5
100 Singular values of the Snapshots
0 0.2 0.4 0.6 0.8 10
0.5
1
x
y(x,
t)
Sol of Full System (Finite Element) :dim = 100
t= 0t= 0.2029t= 0.98551t= 1.2174t= 2
0 0.2 0.4 0.6 0.8 10
0.20.40.60.8
x
y(x,
t)
Sol of Reduced System (Direct POD):dim = 1
t= 0t= 0.2029t= 0.98551t= 1.2174t= 2
0 0.2 0.4 0.6 0.8 10
0.20.40.60.8
x
y(x,
t)
Sol of Reduced System (Direct POD):dim = 2
t= 0t= 0.2029t= 0.98551t= 1.2174t= 2
0 0.2 0.4 0.6 0.8 10
0.20.40.60.8
x
y(x,
t)
Sol of Reduced System (Direct POD):dim = 3
t= 0t= 0.2029t= 0.98551t= 1.2174t= 2
0 0.2 0.4 0.6 0.8 10
0.20.40.60.8
x
y(x,
t)
Sol of Reduced System (Direct POD):dim = 4
t= 0t= 0.2029t= 0.98551t= 1.2174t= 2
0 0.2 0.4 0.6 0.8 10
0.20.40.60.8
x
y(x,
t)
Sol of Reduced System (Direct POD):dim = 5
t= 0t= 0.2029t= 0.98551t= 1.2174t= 2
Figure: Direct PODSaifon Chaturantabut POD
Intro POD Results POD Problem extras Nonlinear Approximation Error and CPU Time
Problem: POD for PDEs with nonlinearities
If we apply the POD basis directly to construct a discretized system,the original system of order N:
Mhddt y(t) + νKhy(t)− Nh(y(t)) = 0
become a system of order k N:
M ddt y(t) + νKy(t)− N(y(t)) = 0,
where the nonlinear term :
N(y(t)) = UTk︸︷︷︸
k×N
Nh(Uk y(t))︸ ︷︷ ︸N×1
⇒ Computational Complexity still depends on N!!
Saifon Chaturantabut POD
Intro POD Results POD Problem extras Nonlinear Approximation Error and CPU Time
Nonlinear Approximation
Recall the problem from bf Direct POD :N(y(t)) = UT︸︷︷︸
k×N
Nh(Uy(t))︸ ︷︷ ︸N×1
WANT:
N(y(t))← C︸︷︷︸k×nm
N(y(t))︸ ︷︷ ︸nm×1
99K k ,nm N 99K Independent of N
2 Approaches:Precomputing Technique: Simple nonlinearEmpirical Interpolation Method (EIM): General nonlinear
Saifon Chaturantabut POD
Intro POD Results POD Problem extras Nonlinear Approximation Error and CPU Time
ACCURACY vs. COMPLEXITY
0 5 10 1510−7
10−6
10−5
10−4
10−3
10−2
10−1
100
101
Number of POD basis used
Avera
ge R
ela
tive E
rror
Relative Error:= sumt||yFE(t)−y(t)||/||yFE(t)|| of Reconstruct Snapshots using POD
Direct PODPrecompute PODEIM−POD
0 5 10 150
0.05
0.1
0.15
0.2
0.25
0.3
0.35
Number of POD basis usedcp
u t
ime
(se
c)
CPU TIME for solving reduced−order ODE system
Direct PODPrecompute PODEIM−POD
Figure: LEFT: Error Eavg = 1nt
∑nti=1‖yh(·,ti )−yh(·,ti )‖X‖yh(·,ti )‖X
.RIGHT: CPU time (sec)
Saifon Chaturantabut POD
Intro POD Results POD Problem extras Nonlinear Approximation Error and CPU Time
QUESTION ?
Saifon Chaturantabut POD
Intro POD Results POD Problem extras Empirical Interpolation Method (EIM) Solutions from EIM Conclusions and Future Works
Empirical Interpolation Method (EIM) [Patera; 2004]
Approximates nonlinear parametrized functionsLet s(x ;µ) be a parametrized function with spatial variable x ∈ Ωand parameter µ ∈ D .Function approximation from EIM:
s(x ;µ) =nm∑
m=1
qm(x)βm(µ),
where spanqmnmm=1 'M s ≡ s(·;µ) : µ ∈ D and βm(µ) is
specified from the interpolation points zmnmm=1, in Ω:
s(zi ;µ) =nm∑
m=1
qm(zi )βm(µ),
for i = 1, . . . ,nm.
Saifon Chaturantabut POD
Intro POD Results POD Problem extras Empirical Interpolation Method (EIM) Solutions from EIM Conclusions and Future Works
EIM: Numerical Example
s(x ;µ) = (1− x)cos(3πµ(x + 1))e−(1+x)µ,
where x ∈ [−1,1] and µ ∈ [1, π].
−1 −0.5 0 0.5 1−1.5
−1
−0.5
0
0.5
1
1.5
2
x
s(x;
µ)
Plot of Approximate Functions (dim = 10) with Exact Functions (in black solid line)
µ= 1 µ= 1.1713 µ= 1.3855 µ= 3.1416
Figure: Approximate Function from EIM with 10 POD basisSaifon Chaturantabut POD
Intro POD Results POD Problem extras Empirical Interpolation Method (EIM) Solutions from EIM Conclusions and Future Works
Plots of Numerical Solutions from 3Approaches
Direct POD
Precomputed POD
EIM-POD
0 10 20 30 40 50 60 7010−15
10−10
10−5
100 Singular values of the Snapshots
Figure: SVD
0 0.2 0.4 0.6 0.8 10
0.5
1
x
y(x
,t)
Sol of Full System (Finite Element) :dim = 100
t= 0t= 0.2029t= 0.98551t= 1.2174t= 2
0 0.2 0.4 0.6 0.8 10
0.20.40.60.8
x
y(x
,t)
Sol of Reduced System (Direct POD):dim = 1
t= 0t= 0.2029t= 0.98551t= 1.2174t= 2
0 0.2 0.4 0.6 0.8 10
0.20.40.60.8
x
y(x
,t)
Sol of Reduced System (Direct POD):dim = 2
t= 0t= 0.2029t= 0.98551t= 1.2174t= 2
0 0.2 0.4 0.6 0.8 10
0.20.40.60.8
x
y(x
,t)
Sol of Reduced System (Direct POD):dim = 3
t= 0t= 0.2029t= 0.98551t= 1.2174t= 2
0 0.2 0.4 0.6 0.8 10
0.20.40.60.8
xy(x
,t)
Sol of Reduced System (Direct POD):dim = 4
t= 0t= 0.2029t= 0.98551t= 1.2174t= 2
0 0.2 0.4 0.6 0.8 10
0.20.40.60.8
x
y(x
,t)
Sol of Reduced System (Direct POD):dim = 5
t= 0t= 0.2029t= 0.98551t= 1.2174t= 2
Figure: Direct POD
Saifon Chaturantabut POD
Intro POD Results POD Problem extras Empirical Interpolation Method (EIM) Solutions from EIM Conclusions and Future Works
0 0.2 0.4 0.6 0.8 10
0.5
1
x
y(x
,t)
Sol of Full System (Finite Element) :dim = 100
t= 0t= 0.2029t= 0.98551t= 1.2174t= 2
0 0.2 0.4 0.6 0.8 10
0.20.40.60.8
xy(x
,t)
Sol of Reduced System (Precompute POD):dim = 1
t= 0t= 0.2029t= 0.98551t= 1.2174t= 2
0 0.2 0.4 0.6 0.8 10
0.20.40.60.8
x
y(x
,t)
Sol of Reduced System (Precompute POD):dim = 2
t= 0t= 0.2029t= 0.98551t= 1.2174t= 2
0 0.2 0.4 0.6 0.8 10
0.20.40.60.8
x
y(x
,t)
Sol of Reduced System (Precompute POD):dim = 3
t= 0t= 0.2029t= 0.98551t= 1.2174t= 2
0 0.2 0.4 0.6 0.8 10
0.20.40.60.8
x
y(x
,t)
Sol of Reduced System (Precompute POD):dim = 4
t= 0t= 0.2029t= 0.98551t= 1.2174t= 2
0 0.2 0.4 0.6 0.8 10
0.20.40.60.8
x
y(x
,t)
Sol of Reduced System (Precompute POD):dim = 5
t= 0t= 0.2029t= 0.98551t= 1.2174t= 2
Figure: Precomputed POD
0 0.2 0.4 0.6 0.8 10
0.5
1
x
y(x
,t)
Sol of Full System (Finite Element) :dim = 100
t= 0t= 0.2029t= 0.98551t= 1.2174t= 2
0 0.2 0.4 0.6 0.8 10
0.20.40.60.8
x
y(x
,t)
Sol of Reduced System (EIM−POD):dim = 1
t= 0t= 0.2029t= 0.98551t= 1.2174t= 2
0 0.2 0.4 0.6 0.8 10
0.20.40.60.8
x
y(x
,t)
Sol of Reduced System (EIM−POD):dim = 2
t= 0t= 0.2029t= 0.98551t= 1.2174t= 2
0 0.2 0.4 0.6 0.8 10
0.20.40.60.8
x
y(x
,t)
Sol of Reduced System (EIM−POD):dim = 3
t= 0t= 0.2029t= 0.98551t= 1.2174t= 2
0 0.2 0.4 0.6 0.8 10
0.20.40.60.8
xy(x
,t)
Sol of Reduced System (EIM−POD):dim = 4
t= 0t= 0.2029t= 0.98551t= 1.2174t= 2
0 0.2 0.4 0.6 0.8 10
0.20.40.60.8
x
y(x
,t)
Sol of Reduced System (EIM−POD):dim = 5
t= 0t= 0.2029t= 0.98551t= 1.2174t= 2
Figure: EIM-POD
Saifon Chaturantabut POD
Intro POD Results POD Problem extras Empirical Interpolation Method (EIM) Solutions from EIM Conclusions and Future Works
Conclusions
Unique Contribution: Cleardescription of the EIM⇒ SuccessfulImplementation of EIM with POD
EIM is comparable to widely acceptedmethods, such as precomputingtechnique
The results suggest that EIM withPOD basis is a promising modelreduction technique for more generalnonlinear PDEs.
Future WorkExtend to higher dimensions
Extend to PDEs with more generalnonlinearities
Apply to practical problem, e.g.optimal control problem
0 10 20 30 40 50 60 7010−8
10−6
10−4
10−2
100
102
Number of POD basis used
Aver
age
Rel
ativ
e Er
ror
Relative Error:=(1/nt)sumi=1n
t (||yFE(t)−y(t)||/||yFE(t)||)2 of Reconstructed Snapshots using different dim of basis from EIM−POD
EIM−POD dim=1EIM−POD dim=2EIM−POD dim=4EIM−POD dim=6EIM−POD dim=10EIM−POD dim=20Direct PODPrecompute POD
0 10 20 30 40 50 60 700
0.2
0.4
0.6
0.8
1
1.2
1.4
Number of POD basis used
CP
U ti
me
(sec
)
CPU TIME for solving reduced−order ODE system
EIM−POD dim=1EIM−POD dim=2EIM−POD dim=4EIM−POD dim=6EIM−POD dim=10EIM−POD dim=20Direct PODPrecompute POD
Figure: Plots of the first 4 POD basisand the space of solutions
Saifon Chaturantabut POD
Intro POD Results POD Problem extras Empirical Interpolation Method (EIM) Solutions from EIM Conclusions and Future Works
Overview
Nonlinear PDE
⇓ L99 FE-Galerkin
FULL Discretized System (ODE)dim = N
⇓ L99 Solve ODE
SNAPSHOTS: y(x, t`)ns`=1
⇓ L99 POD
POD basis: φiki=1
⇓ L99 POD-Galerkin
REDUCED Discretized System (ODE)Linear Term: dim = k N
Non-Linear Term: dim ∼ N −→ −→(i) Memory Saving
(ii) Accuracy
⇓ L99
Nonlinear Approx-EIM
-Precomputing
REDUCED Discretized System (ODE)Linear Term: dim = k N
Non-Linear Term: dim = nm N −→ −→(iii) Efficiency(Time Saving)
Saifon Chaturantabut POD