Upload
others
View
15
Download
0
Embed Size (px)
Citation preview
On Efficiently Updating Singular Value Decomposition Based Reduced Order Models
Ralf Zimmermann
GAMM Workshop Applied and Numerical Linear Algebra with Special Emphasis on Model Reduction
Bremen, Sep. 22.-23. 2011
2
1. Input: CFD snaphots
Flow solutions at different flow conditions
The POD-based ROM approach
m
3
1. Input: CFD snaphots
Flow solutions at different flow conditions
2. POD BasisOrthonomal basis ordered by infor- mation content spanning the same space
The POD-based ROM approach
m
4
3. Order Reduction Select POD components with largest information content
3. Order Reduction Select POD components with largest information content
1. Input: CFD snaphots
Flow solutions at different flow conditions
2. POD BasisOrthonomal basis ordered by infor- mation content spanning the same space
The POD-based ROM approach
mm ~
m
5
3. Order Reduction Select POD components with largest information content
3. Order Reduction Select POD components with largest information content
4. Prediction step Determine POD-ROM coefficients by interpolation / solving low-order PDEs / least-squares optimization
4. Prediction step Determine POD-ROM coefficients by interpolation / solving low-order PDEs/ least-squares optimization
1. Input: CFD snaphots
Flow solutions at different flow conditions
2. POD BasisOrthonomal basis ordered by infor- mation content spanning the same space
The POD-based ROM approach
mm ~
m
6
3. Order Reduction Select POD components with largest information content
3. Order Reduction Select POD components with largest information content
5. Output: approximated flow field
Untried flow condition
1. Input: CFD snaphots
Flow solutions at different flow conditions
2. POD BasisOrthonomal basis ordered by infor- mation content spanning the same space
The POD-based ROM approach
mm ~4. Prediction step Determine POD-ROM coefficients by interpolation / solving low-order PDEs / least-squares optimization
4. Prediction step Determine POD-ROM coefficients by interpolation / solving low-order PDEs/ least-squares optimization
m
7
POD based reduced order models are a powerful tool…
Industrial aircraft configurationROM Speed-up factor > 300
Grid size ~9Mio, subsonic Ma = 0.2
Snapshot data at AoA = -1°, 0°, 1°, 2°
Prediction at AoA = 7° (Extrapolation!)
8
How to compute POD/SVD of augmented data set efficiently?
… but treating large snapshots may become a challenge!
POD/SVD representation known
Incoming new snapshots
9
Nomenclature
Tm
mm
mm
Tmmm
m
j
jmm
mnmm
VUY
WWAWAWAYY
WA
RWWY
:SVD
),,...,(:),...,(1 :Centering
:averageSnapshot
),...,( :matrixSnapshot
11
1
1
1
1,)( :contentn Informatio Relative1
21
2
mrrric mm m
i i
mr
i i
10
Discard columns corresponding to small singular values:
Definition: The data setis called reduced-order model of order of The ratio is called the compression rate.
m
mmmm
r
mrmr
mrnr
m
Tmmmm
RVVVRUUU
VUY
000000
,,...,,,...,
,
111
mmmmm rmnAVU ,,,,,,mr .mY
mrm
Reduced-order representation
11
Given: ROM of
p new snapshot observations
Task: Compute ROMof
Requirement:use only the previous stage ROM and the incoming snapshots!
! Keep computational costs depending on as low as possible
mmmmm rmnAVU ,,,,,, .mnm RY
.pmY
pmm WW ,...,1
pmpmpmpmpm rpmnAVU ,,,,,,
The SVD basis update problem
mn n
12
Objective
POD
MO
DES
=SVD
SNA
PSH
OTS
Efficient SVD basis update• Update SVD basis without having to store the initial snapshots
13
Objective
Efficient SVD basis update• Update SVD basis without having to store the initial snapshots
POD
MO
DES
=SVD
SNA
PSH
OTS
=
SNA
PSH
OTS
SVD+ +PO
D M
OD
ES
14
Objective
POD
MO
DES
=SVD
SNA
PSH
OTS
SNA
PSH
OTS
+
SNA
PSH
OTS
=SVDPO
D M
OD
ES
Efficient SVD basis update• Update SVD basis without having to store the initial snapshots
15
Updating the snapshot mean
Shift vector:
Shift update snapshots to the previous-stage center:
To do: Decompose
SVD basis update strategies
pmmpm
pm
mi
impmpm
TAA
WpAT
1
1:
piAWW mimim ,...,1,:
,1,!
Tpmpmpm
Tpmpmmpm VUTWYY
16
Updating the snapshot mean
Shift vector:
Shift update snapshots to the previous-stage center:
To do: Decompose
SVD basis update strategies
pmmpm
pm
mi
impmpm
TAA
WpAT
1
1:
piAWW mimim ,...,1,:
,1,!
Tpmpmpm
Tpmpmmpm VUTWYY
start here
17
Setting
it holds
SVD basis update strategies (A): two-steps SVD1,2
ppmpTpnm IBYX ,0,0,
prTmmm
Tm
mVUXBWXWY 0, and ,
1 M. Brand: “Fast low-rank modifications of the thin SVD”, Lin. Alg. and its Appl. 415, 20062 P. Hall et al.: “Merging and splitting eigenspace models”, IEEE Trans. Pattern analysis
and Machine Intelligence, 22(9), 2000
18
Setting
it holds
SVD basis update strategies (A): two-steps SVD1,2
ppmpTpnm IBYX ,0,0,
prTmmm
Tm
mVUXBWXWY 0, and ,
1 M. Brand: “Fast low-rank modifications of the thin SVD”, Lin. Alg. and its Appl. 415, 20062 P. Hall et al.: “Merging and splitting eigenspace models”, IEEE Trans. Pattern analysis
and Machine Intelligence, 22(9), 2000
rank-p SVD update problem
19
Setting
it holds
Factoring out orthogonal components:
SVD basis update strategies (A): two-steps SVD1,2
ppmpTpnm IBYX ,0,0,
prTmmm
Tm
mVUXBWXWY 0, and ,
1 M. Brand: “Fast low-rank modifications of the thin SVD”, Lin. Alg. and its Appl. 415, 20062 P. Hall et al.: “Merging and splitting eigenspace models”, IEEE Trans. Pattern analysis
and Machine Intelligence, 22(9), 2000
),0()0,(
00
, ppmp
prTm
ppm
mT
IV
IWUBWX
m
20
Setting
it holds
Factoring out orthogonal components:
where e.g. via Gram Schmidt
SVD basis update strategies (A): two-steps SVD1,2
ppmpTpnm IBYX ,0,0,
prTmmm
Tm
mVUXBWXWY 0, and ,
1 M. Brand: “Fast low-rank modifications of the thin SVD”, Lin. Alg. and its Appl. 415, 20062 P. Hall et al.: “Merging and splitting eigenspace models”, IEEE Trans. Pattern analysis
and Machine Intelligence, 22(9), 2000
),0()0,(
0,
ppmp
prTm
Torth
Tmm
orthmT
IV
PPWU
PUBWXm
)(orth),( PPWUUWP orthTm
21
Setting
it holds
Factoring out orthogonal components:
SVD basis update strategies (A): two-steps SVD1,2
ppmpTpnm IBYX ,0,0,
prTmmm
Tm
mVUXBWXWY 0, and ,
1 M. Brand: “Fast low-rank modifications of the thin SVD”, Lin. Alg. and its Appl. 415, 20062 P. Hall et al.: “Merging and splitting eigenspace models”, IEEE Trans. Pattern analysis
and Machine Intelligence, 22(9), 2000
),0()0,(
0,
ppmp
prTm
Torth
Tmm
orthmT
IV
PPWU
PUBWXm
orthonormal columns
22
Setting
it holds
Factoring out orthogonal components:
SVD basis update strategies (A): two-steps SVD1,2
ppmpTpnm IBYX ,0,0,
prTmmm
Tm
mVUXBWXWY 0, and ,
1 M. Brand: “Fast low-rank modifications of the thin SVD”, Lin. Alg. and its Appl. 415, 20062 P. Hall et al.: “Merging and splitting eigenspace models”, IEEE Trans. Pattern analysis
and Machine Intelligence, 22(9), 2000
),0()0,(
0,
ppmp
prTm
Torth
Tmm
orthmT
IV
PPWU
PUBWXm
orthonormal columns
23
Setting
it holds
Factoring out orthogonal components:
SVD basis update strategies (A): two-steps SVD1,2
ppmpTpnm IBYX ,0,0,
prTmmm
Tm
mVUXBWXWY 0, and ,
1 M. Brand: “Fast low-rank modifications of the thin SVD”, Lin. Alg. and its Appl. 415, 20062 P. Hall et al.: “Merging and splitting eigenspace models”, IEEE Trans. Pattern analysis
and Machine Intelligence, 22(9), 2000
),0()0,(
0,
ppmp
prTm
Torth
Tmm
orthmT
IV
PPWU
PUBWXm
size )()( prpr mm
24
Setting
it holds
Factoring out orthogonal components:
SVD basis update strategies (A): two-steps SVD1,2
ppmpTpnm IBYX ,0,0,
prTmmm
Tm
mVUXBWXWY 0, and ,
1 M. Brand: “Fast low-rank modifications of the thin SVD”, Lin. Alg. and its Appl. 415, 20062 P. Hall et al.: “Merging and splitting eigenspace models”, IEEE Trans. Pattern analysis
and Machine Intelligence, 22(9), 2000
),0()0,(~~~,
ppmp
prTmT
orthmT
IV
VUPUBWXm
25
Setting
it holds
Factoring out orthogonal components:
SVD basis update strategies (A): two-steps SVD1,2
ppmpTpnm IBYX ,0,0,
prTmmm
Tm
mVUXBWXWY 0, and ,
1 M. Brand: “Fast low-rank modifications of the thin SVD”, Lin. Alg. and its Appl. 415, 20062 P. Hall et al.: “Merging and splitting eigenspace models”, IEEE Trans. Pattern analysis
and Machine Intelligence, 22(9), 2000
),0()0,(~~~,
ppmp
prTmT
orthmT
IV
VUPUBWXm
26
Repeat for shifting to new center:
SVD basis update strategies (A): two-steps SVD1,2
1 M. Brand: “Fast low-rank modifications of the thin SVD”, Lin. Alg. and its Appl. 415, 20062 P. Hall et al.: “Merging and splitting eigenspace models”, IEEE Trans. Pattern analysis
and Machine Intelligence, 22(9), 2000
,1,!
Tpmpmpm
Tpmpmmpm VUTWYY
SVD now known
27
Comments: ‘re-orthogonalization’ expensive, additional (large-scale) SVD requiredless robust via Gram-Schmidt
Parallelization is more involved
SVD basis update strategies (A): two-steps SVD1,2
1 M. Brand: “Fast low-rank modifications of the thin SVD”, Lin. Alg. and its Appl. 415, 20062 P. Hall et al.: “Merging and splitting eigenspace models”, IEEE Trans. Pattern analysis
and Machine Intelligence, 22(9), 2000
)(orth),( PPWUUWP orthTm
28
Objective:
First step: Compute SVD of
Reduce to symmetric EVD
SVD basis update strategies (B): EVD3+SVD
WWYWWYYYWYWY
Tm
T
Tmm
Tm
mT
m ,,
3 Generalize ideas from: J.R. Bunch, C.P. Nielsen: “Updating the singular value decomposition”, Numer. Math. 31, 1978
,1,!
Tpmpmpm
Tpmpmmpm VUTWYY
WYm ,
29
Objective:
First step: Compute SVD of
Reduce to symmetric EVD
SVD basis update strategies (B): EVD3+SVD
WWVUW
WUVVVWYWY TT
mmmT
Tmmm
Tmmm
mT
m
2
,,
3 Generalize ideas from: J.R. Bunch, C.P. Nielsen: “Updating the singular value decomposition”, Numer. Math. 31, 1978
,1,!
Tpmpmpm
Tpmpmmpm VUTWYY
WYm ,
Exploit previous stage SVD
30
Objective:
First step: Compute SVD of
Reduce to symmetric EVD
SVD basis update strategies (B): EVD3+SVD
pp
Tm
Tmm
T
Tmmm
ppm
mT
m IV
WWUWWU
IV
WYWY0
00
0,,
2
3 Generalize ideas from: J.R. Bunch, C.P. Nielsen: “Updating the singular value decomposition”, Numer. Math. 31, 1978
,1,!
Tpmpmpm
Tpmpmmpm VUTWYY
WYm ,
factor out
31
Objective:
First step: Compute SVD of
Reduce to symmetric EVD
SVD basis update strategies (B): EVD3+SVD
pp
Tm
Tmm
T
Tmmm
ppm
mT
m IV
WWUWWU
IV
WYWY0
00
0,,
2
3 Generalize ideas from: J.R. Bunch, C.P. Nielsen: “Updating the singular value decomposition”, Numer. Math. 31, 1978
,1,!
Tpmpmpm
Tpmpmmpm VUTWYY
WYm ,
size )()( prpr mm
32
Objective:
First step: Compute SVD of
Reduce to symmetric EVD
SVD basis update strategies (B): EVD3+SVD
pp
TmT
ppm
mT
m IV
QQI
VWYWY
00~~~
00
,,
3 Generalize ideas from: J.R. Bunch, C.P. Nielsen: “Updating the singular value decomposition”, Numer. Math. 31, 1978
,1,!
Tpmpmpm
Tpmpmmpm VUTWYY
WYm ,
33
Objective:
First step: Compute SVD of
Reduce to symmetric EVD
Compute left singular vectors via
SVD basis update strategies (B): EVD3+SVD
pp
TmT
ppm
mT
m IV
QQI
VWYWY
00~~~
00
,,
3 Generalize ideas from: J.R. Bunch, C.P. Nielsen: “Updating the singular value decomposition”, Numer. Math. 31, 1978
,1,!
Tpmpmpm
Tpmpmmpm VUTWYY
WYm ,
1~~,ˆ QWUU mm
34
Use Brand’s method for shifting to new center:
SVD basis update strategies (B): EVD3 + SVD
,1,!
Tpmpmpm
Tpmpmmpm VUTWYY
SVD now known
3 Generalize ideas from: J.R. Bunch, C.P. Nielsen: “Updating the singular value decomposition”, Numer. Math. 31, 1978
35
Objective:
Let
Reduce to symmetric EVD, exploit previous stage SVD in matrix products
Compute left singular vectors via
SVD basis update strategies (C): one-step EVD
)()(~~~ pmpmTT RQQXX
,1,!
Tpmpmpm
Tpmpmmpm VUTWYY
)(1,: pmnTpmpmm RTWYX
1~~ QXU pm
36
Comments:Straight forward parallelization! (only standard matrix products required in parallel)
SVD basis update strategies (B) and (C)
37
Count flops for matrix product
Strategy (B) is more efficient than strategy (A):
Analysis of Computational costs
nmp pmmn RBRAAB ,,
))((
))(()(flops(B)flops(A)2
221
npmpO
PorthOnppprm
38
The ranking of Strategy (B) vs. (C) depends on the compression rate!
Assumption:
Then the computational costs differ by
Analysis of Computational costs
2))(32()2(flops(B)flops(C) 222 ppmmpxmxxn
rate)on (compressi , mr
mpmmxpxmprr
39
The ranking of Strategy (B) vs. (C) depends on the compression rate!
Solving the quadratic equation shows thatStrategy (B) is more efficient than Strategy (C) if
In practical implementations, use switch to select the most efficient update strategy.
Analysis of Computational costs
710)1(10)1(30 2241 ppmpmpmx m
40
Rules of thumb:For highly compressed models, Strategy (B) is more efficient than Strategy (C)For weakly compressed models, the opposite holds true
More preciselyStrategy (B) is more efficient than Strategy (C), if and
Strategy (C) is more efficient than Strategy (B), if
Analysis of Computational costs
21x
.32x
.12 pm
41
Uncompressed models
Example: Updating SVDs of Random matrices
100,500,000,100,, pmnRWRY pnmn
Method Reconstruction error time (sec)
Strategy (A), SVD-orth 2.504e-12 7.88
Strategy (A), QR-orth 2.333e-12 8.15
Strategy (B) 2.342e-12 6.32
Strategy (C) 2.279e-12 4.42
599,4991 pmm rmr
42
Compression level
Example: Updating SVDs of Random matrices
100,500,000,100,, pmnRWRY pnmn
Method Reconstruction error time (sec)
Strategy (A), SVD-orth 92.21 4.18
Strategy (A), QR-orth 92.21 4.23
Strategy (B) 92.21 1.93
Strategy (C) 93.41 3.54
4.0,200,200 mr
pmmmxrr
43
The update strategies following the symmetric EVD approach are more efficient than the SVD update known from literature (Brand, Hall et al.)(Assumption: n>>m)
Industrial point of view: SVD/EVD performed by ‘black box function’
Most efficient: choose method depending on the compression rate
The examples suggest that all methods share a similar level of accuracy(SVD-approach may suffer from orthogonal out-factoring,EVD-approaches may suffer from squaring the condition number)
Summary
44
For details and additional references, see:“A comprehensive comparison of various algorithms for efficiently updating singular value decomposition based reduced order models”, DLR IB 124-2011/3
Freely available at DLR’s electronic library:http://elib.dlr.de/70251
Summary
45
Thank you for your attention!