Upload
wayne-gayden
View
218
Download
1
Embed Size (px)
Citation preview
A Rank-Revealing Method for Low Rank Matrices withUpdating, Downdating, and Applications
Tsung-Lin Lee(Michigan State University)
2007 AMS Session Meeting, Chicago
joint work with Tien-Yien Li and Zhonggang Zeng
Rank determination problems appear in
1. Image Processing
2. Information Retrieval
3. Matrix Approximation
4. Least Squares Problems
5. Numerical Polynomial Algebra
……
1
k
k
The rank
gap:
kAranknkk
)(11
Numerical rank: nmR nm ,
the rank decision threshold :
the approxi-rank w.r.t. the threshold : (numerical rank)
)(rank
2k
kr k,rank(A) Assume
2})(|{12})(|{
minmin BABArBrankB
rrBrankB
Mirsky Theorem:
1k
A
k
2k
The numerical rank w.r.t. threshold :
)(min2
BrankArankAB
1k
A
k
SVD Algorithm (Golub-Reinsch)
In some applications, the matrix is large.
-The rank is close to full. (high rank)
-The rank is close to zero. (low rank)
=> efficient when the matrix size is moderate.
The goal: An efficient and stable algorithm
The updating and downdating problems
=> It can’t solve them efficiently.
1989 Tony Chan => Rank Revealing QR algorithm for high rank matrices
Updating problem:
krank )(
?)ˆ( rank
?)ˆ( rank
Downdating problem:
krank )(
?)ˆ( rank
?)ˆ( rank
1992 G.W. Stewart => rank revealing UTV decomposition. (URV/ULV)
1. Updating problems are applicable.
2. Downdating problems are difficult.
=> re-compute the UTV decomposition
F.D. Fierro, P.C. Hansen and P.S. K. Hansen (1999)UTV tools: Matlab templates for rank-revealing UTV decomposition
2005, T.Y. Li and Zhonggang Zeng
=> rank-revealing algorithm for high rank matrices
1. The approxi-rank.2. The approxi-kernel.3. The method is more efficient and robust.4. Algorithms for updating and downdating problems are straightforward, stable and efficient.
kernel-approxileftkernel-approxi
range-approxi rowspace-approxi
Tnkk
n
k
k
mkk vvvvuuuu
111
1
11
nkk
11
Tsung-Lin Lee, T.Y. Li and Zhonggang Zeng
=> rank-revealing algorithm for low rank matrices
1. The approxi-rank.2. The approxi-range.3. The approxi-rowspace.4. The projections of left and right kernel.5. USV+E decomposition.6. The method is robust and more efficient.7. Algorithms for updating and downdating problems are straightforward, stable and efficient.
= +
2
21
1
j
jj
jT
jT
j
xA
xy
yA
yx
1v
1u
Stop when thresholdepsyA
j
jT
:,
2
21
Power iteration on T
Random 0y
,3,2,1for j
0y
1y
2y
4y1u
epsO
0
mR
3y
},,,{ 21 kuuuspan approxi-range
},,,{)( 211 kuuuspanrangez
nnkk 1121
0' k
The implicit singular value deflation:
0'''1121
nkk
Tzz 11'
Tp UUE kzzU ,,1
USV+E decomp.
TT LQU LQ pT EULQ
)(1 rangez
Tzz 11' )'(2 rangez
TT zzzz 2211'' )''(3 rangez
Tkk
Tp zzzzE 11
2pE
perturbation= +
USV+E decomposition
kk
approxi-range
approxi-rowspace
Numerical experiments and comparisonsMatlab 7.0, on Dell PC Pentium D 3.2MHz CPU, 1GB RAM
2n
n,20
2
400x200 800x400 1600x800 3200x1600
time error time error time error time error
SVD 0.31 3e-9 2.19 4e-9 16.6 3e-9 144. 7e-9
lurv 0.66 3e-9 1.52 4e-9 5.97 3e-9 32.5 7e-9
lulv 0.56 4e-9 1.52 6e-9 6.03 5e-9 31.9 5e-9
larank 0.05 3e-9 0.11 4e-9 0.39 3e-9 1.81 4e-9
3111
10 e
10)(,81 ranke
A U= + pE
TV
Row updating
A U= + pE
TV
1
TV~
A U= + pE
TV
1
Tv~
2aVVa T
aVVa
vT
~
Ta
row downdating
1)ˆ( korkrank )()ˆ( rowrow
QRUV )(min
R
deflate R
min
210
0ˆ
R
RGG
pT EQRV
= +
EUSV T
Dominant(signal)A Perturbation
(noise)
= +
USV+E decomposition
)( 2nO )(nO
Information retrieval
Latent Semantic Indexing method (LSI)
Library database
Webpage search engine (Google)
…
rank, revealing, updating, downdating, application
1
0
1
1
0
0
0
0
0
1
0
1
||
q12x8 term by document matrix
2222
,cos
j
jT
j
j
jAeq
Aeq
Aeq
Aeq
EUSVA T
= +
222222
~cos
j
jT
jT
jTT
jT
jTT
jWeq
WeUq
eSVq
eSVUq
eUSVq
eUSVq
TSVW
Image processing
Saving storage of photographs
FBI Fingerprint Image Database
Face Image Database
…
A 480x640 monochrome (baseball picture)
Grey levels: 0 => 1
black white
j
j
Rank 20 approximation imageRank 480 image
20)(%35.1 1 kArank
7.477.920.735.38 : 1510.8%
2.014.945.140.388.07 : 1341.3%
k SVD
2.87
lulv
3.02
lurvlarank
0.17
Running time (seconds)
15.2 : 1
Compression ratio
18
Approxi-rank
2.1%
Threshold
1
1
1
1:)( knm
mn
http://www.msu.edu/~leetsung/Software.htm
HighRankRev and LowRankRev Package
Thank you