Upload
others
View
3
Download
0
Embed Size (px)
Citation preview
§9.2 Orthogonal Matrices and Similarity Transformations
Def: A matrix Q ∈ Rn×n is said to be orthogonal if its columns{q(1),q(2), · · · ,q(n)
}form an orthonormal set in Rn.
Thm: Suppose matrix Q ∈ Rn×n is orthogonal. ThenI Q is invertible with Q−1 = QT .I For any x, y ∈ Rn, (Q x)T (Q y) = xTy.I For any x ∈ Rn, ‖Q x‖2 = ‖x‖2.
Ex
H =
12
12
12
12
−12
12 −1
212
−12 −1
212
12
12 −1
2 −12
12
, HT H = I .
§9.2 Orthogonal Matrices and Similarity Transformations
Def: A matrix Q ∈ Rn×n is said to be orthogonal if its columns{q(1),q(2), · · · ,q(n)
}form an orthonormal set in Rn.
Thm: Suppose matrix Q ∈ Rn×n is orthogonal. ThenI Q is invertible with Q−1 = QT .I For any x, y ∈ Rn, (Q x)T (Q y) = xTy.I For any x ∈ Rn, ‖Q x‖2 = ‖x‖2.
Ex
H =
12
12
12
12
−12
12 −1
212
−12 −1
212
12
12 −1
2 −12
12
, HT H = I .
§9.2 Orthogonal Matrices and Similarity Transformations
Def: A matrix Q ∈ Rn×n is said to be orthogonal if its columns{q(1),q(2), · · · ,q(n)
}form an orthonormal set in Rn.
Thm: Suppose matrix Q ∈ Rn×n is orthogonal. ThenI Q is invertible with Q−1 = QT .I For any x, y ∈ Rn, (Q x)T (Q y) = xTy.I For any x ∈ Rn, ‖Q x‖2 = ‖x‖2.
Ex
H =
12
12
12
12
−12
12 −1
212
−12 −1
212
12
12 −1
2 −12
12
, HT H = I .
Def: Two matrices A and B are similar if a nonsingular matrix Sexists with A = S−1 B S .
Thm: Suppose A and B are similar matrices with A = S−1 B S andλ is an eigenvalue of A with associated eigenvector x. Then λis an eigenvalue of B with associated eigenvector S x.
Proof: Let x 6= 0 be such that
A x =(S−1 B S
)x = λ x.
It follows that B (S x) = λ (S x)
Def: Two matrices A and B are similar if a nonsingular matrix Sexists with A = S−1 B S .
Thm: Suppose A and B are similar matrices with A = S−1 B S andλ is an eigenvalue of A with associated eigenvector x. Then λis an eigenvalue of B with associated eigenvector S x.
Proof: Let x 6= 0 be such that
A x =(S−1 B S
)x = λ x.
It follows that B (S x) = λ (S x)
Def: Two matrices A and B are similar if a nonsingular matrix Sexists with A = S−1 B S .
Thm: Suppose A and B are similar matrices with A = S−1 B S andλ is an eigenvalue of A with associated eigenvector x. Then λis an eigenvalue of B with associated eigenvector S x.
Proof: Let x 6= 0 be such that
A x =(S−1 B S
)x = λ x.
It follows that B (S x) = λ (S x)
Thm: A ∈ Rn is similar to a diagonal matrix D if and only if A has nlinearly independent eigenvectors.
Proof:
A = S D S−1,(S =
(v(1), v(2), · · · , v(n)
), D = diag (λ1, λ2, · · · , λn)
)⇐⇒ A
(v(1), v(2), · · · , v(n)
)=(v(1), v(2), · · · , v(n)
)diag (λ1, λ2, · · · , λn) ,
⇐⇒ A v(j) = λj v(j), j = 1, 2, · · · , n. v(1), v(2), · · · , v(n) L.I.D.
Cor: A ∈ Rn with n distinct eigenvalues is similar to diagonalmatrix.
Thm: A ∈ Rn is similar to a diagonal matrix D if and only if A has nlinearly independent eigenvectors.
Proof:
A = S D S−1,(S =
(v(1), v(2), · · · , v(n)
), D = diag (λ1, λ2, · · · , λn)
)⇐⇒ A
(v(1), v(2), · · · , v(n)
)=(v(1), v(2), · · · , v(n)
)diag (λ1, λ2, · · · , λn) ,
⇐⇒ A v(j) = λj v(j), j = 1, 2, · · · , n. v(1), v(2), · · · , v(n) L.I.D.
Cor: A ∈ Rn with n distinct eigenvalues is similar to diagonalmatrix.
Thm: A ∈ Rn is similar to a diagonal matrix D if and only if A has nlinearly independent eigenvectors.
Proof:
A = S D S−1,(S =
(v(1), v(2), · · · , v(n)
), D = diag (λ1, λ2, · · · , λn)
)⇐⇒ A
(v(1), v(2), · · · , v(n)
)=(v(1), v(2), · · · , v(n)
)diag (λ1, λ2, · · · , λn) ,
⇐⇒ A v(j) = λj v(j), j = 1, 2, · · · , n. v(1), v(2), · · · , v(n) L.I.D.
Cor: A ∈ Rn with n distinct eigenvalues is similar to diagonalmatrix.
Def: A matrix U ∈ Cn×n is unitary if ‖U x‖2 = ‖x‖2 forany vector x.
Schur Thm: Let A ∈ Rn. A unitary matrix U exists such that
T = U−1 AU =
is upper-triangular.
The diagonal entries of T are the eigenvalues of A.
Def: A matrix U ∈ Cn×n is unitary if ‖U x‖2 = ‖x‖2 forany vector x.
Schur Thm: Let A ∈ Rn. A unitary matrix U exists such that
T = U−1 AU =
is upper-triangular.
The diagonal entries of T are the eigenvalues of A.
Def: The complex conjugate of a complex vectoru = a +
√−1b ∈ Cn is u = a−
√−1b.
Def: ‖u‖2 =√aTa + bTb =
√uTu.
Thm: Let A = AT ∈ Rn be symmetric. Then all eigenvalues of Aare real.
Proof: Let λ be an eigenvalue of A with eigenvector u. Then λ iseigenvalue,
Au = λu, −→ Au = λu.
λ(uT u
)= uT (Au)
= (Au)T u = λ(uT u
).
Therefore λ = λ ∈ R.
Def: The complex conjugate of a complex vectoru = a +
√−1b ∈ Cn is u = a−
√−1b.
Def: ‖u‖2 =√aTa + bTb =
√uTu.
Thm: Let A = AT ∈ Rn be symmetric. Then all eigenvalues of Aare real.
Proof: Let λ be an eigenvalue of A with eigenvector u. Then λ iseigenvalue,
Au = λu, −→ Au = λu.
λ(uT u
)= uT (Au)
= (Au)T u = λ(uT u
).
Therefore λ = λ ∈ R.
Def: The complex conjugate of a complex vectoru = a +
√−1b ∈ Cn is u = a−
√−1b.
Def: ‖u‖2 =√aTa + bTb =
√uTu.
Thm: Let A = AT ∈ Rn be symmetric. Then all eigenvalues of Aare real.
Proof: Let λ be an eigenvalue of A with eigenvector u. Then λ iseigenvalue,
Au = λu, −→ Au = λu.
λ(uT u
)= uT (Au)
= (Au)T u = λ(uT u
).
Therefore λ = λ ∈ R.
Def: The complex conjugate of a complex vectoru = a +
√−1b ∈ Cn is u = a−
√−1b.
Def: ‖u‖2 =√aTa + bTb =
√uTu.
Thm: Let A = AT ∈ Rn be symmetric. Then all eigenvalues of Aare real.
Proof: Let λ be an eigenvalue of A with eigenvector u. Then λ iseigenvalue,
Au = λu, −→ Au = λu.
λ(uT u
)= uT (Au)
= (Au)T u = λ(uT u
).
Therefore λ = λ ∈ R.
Thm: A matrix A ∈ Rn is symmetric if and only if there exists adiagonal matrix D ∈ Rn and an orthogonal matrix Q so that
A = Q D QT = Q
QT .
Proof: I By induction on n. Assume theorem true for n − 1.I Let λ be eigenvalue of A with unit eigenvector u: Au = λu.I We extend u into an orthonormal basis for Rn: u,u2, · · · ,un
are unit, mutually orthogonal vectors.
Udef= (u,u2, · · · ,un)
def=(u, U
)∈ Rn×n is orthogonal.I
UT AU =
(uT
UT
)(Au,A U
)=
uT (Au) uT(A U
)UT (Au) UT
(A U
) =
(λ 0T
0 UT(A U
) ).
I Matrix UT(A U
)∈ R(n−1)×(n−1) is symmetric.
Thm: A matrix A ∈ Rn is symmetric if and only if there exists adiagonal matrix D ∈ Rn and an orthogonal matrix Q so that
A = Q D QT = Q
QT .
Proof: I By induction on n. Assume theorem true for n − 1.I Let λ be eigenvalue of A with unit eigenvector u: Au = λu.I We extend u into an orthonormal basis for Rn: u,u2, · · · ,un
are unit, mutually orthogonal vectors.
Udef= (u,u2, · · · ,un)
def=(u, U
)∈ Rn×n is orthogonal.I
UT AU =
(uT
UT
)(Au,A U
)=
uT (Au) uT(A U
)UT (Au) UT
(A U
) =
(λ 0T
0 UT(A U
) ).
I Matrix UT(A U
)∈ R(n−1)×(n−1) is symmetric.
Thm: A matrix A ∈ Rn×n is symmetric if and only if there exists adiagonal matrix D ∈ Rn×n and an orthogonal matrix Q so
that A = Q D QT = Q
QT .
Proof: I
UT AU =
(λ 0T
0 UT(A U
) ).
I By induction, there exist diagonal matrix D and orthogonalmatrix Q ∈ R(n−1)×(n−1),
UT(A U
)= Q D QT .
I therefore
UT AU =
(λ 0T
0 Q D QT
).
A =
(U
(1
Q
))(λ
D
)(U
(1
Q
))Tdef= Q D QT .
Thm: A matrix A ∈ Rn×n is symmetric if and only if there exists adiagonal matrix D ∈ Rn×n and an orthogonal matrix Q so
that A = Q D QT = Q
QT .
Proof: I
UT AU =
(λ 0T
0 UT(A U
) ).
I By induction, there exist diagonal matrix D and orthogonalmatrix Q ∈ R(n−1)×(n−1),
UT(A U
)= Q D QT .
I therefore
UT AU =
(λ 0T
0 Q D QT
).
A =
(U
(1
Q
))(λ
D
)(U
(1
Q
))Tdef= Q D QT .
Thm: Let matrix A ∈ Rn×n be symmetric. Then A is positivedefinite if and only if all eigenvalues of A are positive.
Proof: I Let the diagonal matrix D ∈ Rn×n and an orthogonal matrix Qbe so that A = Q D QT .
I D = diag (λ1, λ2, · · · , λn). λ1, λ2, · · · , λn eigenvalues of A.
A is positive definite
⇐⇒ xT A x > 0 for any non-zero x
⇐⇒(QT x
)TD(QT x
)> 0 for any non-zero x
⇐⇒ yT D y > 0 for any non-zero y
⇐⇒ diagonal entries of D are positive.
Thm: Let matrix A ∈ Rn×n be symmetric. Then A is positivedefinite if and only if all eigenvalues of A are positive.
Proof: I Let the diagonal matrix D ∈ Rn×n and an orthogonal matrix Qbe so that A = Q D QT .
I D = diag (λ1, λ2, · · · , λn). λ1, λ2, · · · , λn eigenvalues of A.
A is positive definite
⇐⇒ xT A x > 0 for any non-zero x
⇐⇒(QT x
)TD(QT x
)> 0 for any non-zero x
⇐⇒ yT D y > 0 for any non-zero y
⇐⇒ diagonal entries of D are positive.
Thm: Let matrix A ∈ Rn×n be symmetric. Then A is positivedefinite if and only if all eigenvalues of A are positive.
Proof: I Let the diagonal matrix D ∈ Rn×n and an orthogonal matrix Qbe so that A = Q D QT .
I D = diag (λ1, λ2, · · · , λn). λ1, λ2, · · · , λn eigenvalues of A.
A is positive definite
⇐⇒ xT A x > 0 for any non-zero x
⇐⇒(QT x
)TD(QT x
)> 0 for any non-zero x
⇐⇒ yT D y > 0 for any non-zero y
⇐⇒ diagonal entries of D are positive.
Thm: Let matrix A ∈ Rn×n be symmetric. Then A is positivedefinite if and only if all eigenvalues of A are positive.
Proof: I Let the diagonal matrix D ∈ Rn×n and an orthogonal matrix Qbe so that A = Q D QT .
I D = diag (λ1, λ2, · · · , λn). λ1, λ2, · · · , λn eigenvalues of A.
A is positive definite
⇐⇒ xT A x > 0 for any non-zero x
⇐⇒(QT x
)TD(QT x
)> 0 for any non-zero x
⇐⇒ yT D y > 0 for any non-zero y
⇐⇒ diagonal entries of D are positive.
Thm: Let matrix A ∈ Rn×n be symmetric. Then A is positivedefinite if and only if all eigenvalues of A are positive.
Proof: I Let the diagonal matrix D ∈ Rn×n and an orthogonal matrix Qbe so that A = Q D QT .
I D = diag (λ1, λ2, · · · , λn). λ1, λ2, · · · , λn eigenvalues of A.
A is positive definite
⇐⇒ xT A x > 0 for any non-zero x
⇐⇒(QT x
)TD(QT x
)> 0 for any non-zero x
⇐⇒ yT D y > 0 for any non-zero y
⇐⇒ diagonal entries of D are positive.
Thm: Let matrix A ∈ Rn×n be symmetric. Then A is positivedefinite if and only if all eigenvalues of A are positive.
Proof: I Let the diagonal matrix D ∈ Rn×n and an orthogonal matrix Qbe so that A = Q D QT .
I D = diag (λ1, λ2, · · · , λn). λ1, λ2, · · · , λn eigenvalues of A.
A is positive definite
⇐⇒ xT A x > 0 for any non-zero x
⇐⇒(QT x
)TD(QT x
)> 0 for any non-zero x
⇐⇒ yT D y > 0 for any non-zero y
⇐⇒ diagonal entries of D are positive.
§9.3 The Power Method for Google PageRank (I)
I The PageRank Principle: The importance of each Webpageis proportional to the total size of the other Webpages whichare pointing to it.
§9.3 The Power Method for Google PageRank (II)
I random surf with jump: A Websurfer surfs the nextWebpage
I either jumping to a page chosen at random from the entireWeb at 15% likelihood,
I or choosing a random link from the Webpage at 85%likelihood.
§9.3 The Power Method for Google PageRank (III)
I Google Matrix G : each row/column represents awebpage, each G entry models Web connectivity and Webuser surf patterns,
I PageRank vector x is eigenvector for G :
G x = 1 · x,
where 1 is always a simple eigenvalue of G .
I Power Method for iteratively computing x, given x(0),
x(k+1) = G x(k), k = 0, 1, · · · ,
The Power Method, in general
Given: Matrix A ∈ Rn×n, with n eigenvalues
|λ1| > |λ2| ≥ |λ3| ≥ · · · ≥ |λn| .
(A has precisely one eigenvalue, λ1, that is largest in magnitude.)
Task: Compute λ1 and corresponding eigenvector v1.
Despite condition on λ1, PM usually first method to try.
The Power Method, in general
Given: Matrix A ∈ Rn×n, with n eigenvalues
|λ1| > |λ2| ≥ |λ3| ≥ · · · ≥ |λn| .
(A has precisely one eigenvalue, λ1, that is largest in magnitude.)
Task: Compute λ1 and corresponding eigenvector v1.
Despite condition on λ1, PM usually first method to try.
The Power Method, in general
Given: Matrix A ∈ Rn×n, with n eigenvalues
|λ1| > |λ2| ≥ |λ3| ≥ · · · ≥ |λn| .
(A has precisely one eigenvalue, λ1, that is largest in magnitude.)
Task: Compute λ1 and corresponding eigenvector v1.
Despite condition on λ1, PM usually first method to try.
The Power MethodI Assume v1, v2, · · · , vn are eigenvectors pertaining toλ1, λ2, · · · , λn.
I Given initial vector x 6= 0. Then
x =n∑
j=1
βj vj
for some coefficients β1, · · · , βn. Assume β1 6= 0.
I For any k > 0
Ak x =n∑
j=1
βj λkj vj
= β1λk1
v1 +n∑
j=2
(λjλ1
)k (βjβ1
vj
)= β1λ
k1
(v1 + O
((λ2λ1
)k))
Ak x points to the direction of v1 for large k .
The Power MethodI Assume v1, v2, · · · , vn are eigenvectors pertaining toλ1, λ2, · · · , λn.
I Given initial vector x 6= 0. Then
x =n∑
j=1
βj vj
for some coefficients β1, · · · , βn. Assume β1 6= 0.I For any k > 0
Ak x =n∑
j=1
βj λkj vj
= β1λk1
v1 +n∑
j=2
(λjλ1
)k (βjβ1
vj
)= β1λ
k1
(v1 + O
((λ2λ1
)k))
Ak x points to the direction of v1 for large k .
The Power MethodI Assume v1, v2, · · · , vn are eigenvectors pertaining toλ1, λ2, · · · , λn.
I Given initial vector x 6= 0. Then
x =n∑
j=1
βj vj
for some coefficients β1, · · · , βn. Assume β1 6= 0.I For any k > 0
Ak x =n∑
j=1
βj λkj vj
= β1λk1
v1 +n∑
j=2
(λjλ1
)k (βjβ1
vj
)= β1λ
k1
(v1 + O
((λ2λ1
)k))
Ak x points to the direction of v1 for large k .
Rayleigh quotient
Given: Approximate eigenvector x.
Task: Find approximate eigenvalue λ.LS for λ: Choose λ in LS sense
minλ ‖A x− λ x‖2 .
LS Solution:
λ =xT (A x)
xT x
Rayleigh quotient
Given: Approximate eigenvector x.Task: Find approximate eigenvalue λ.
LS for λ: Choose λ in LS sense
minλ ‖A x− λ x‖2 .
LS Solution:
λ =xT (A x)
xT x
Algorithm 1 The Power Method
Input: Matrix A ∈ Rn×n,initial guess x(0) ∈ Rn, and tolerance τ > 0.
Output: Approximate eigenvalue λ, eigenvector x.Algorithm:Normalize: x(0) = x(0)
/∥∥x(0)∥∥2, y(0) = A x(0), k = 0.
λ =(x(0))T
y(0).
while∥∥y(k) − λ x(k)∥∥
2≥ τ do
x(k+1) = y(k)/∥∥y(k)∥∥
2, y(k+1) = A x(k+1).
λ =(x(k+1)
)Ty(k+1).
k = k + 1.end while
Algorithm 2 The Symmetric Power Method
Input: Symmetric matrix A ∈ Rn×n,initial guess x(0) ∈ Rn, and tolerance τ > 0.
Output: Approximate eigenvalue λ, eigenvector x.Algorithm:Normalize: x(0) = x(0)
/∥∥x(0)∥∥2, y(0) = A x(0), k = 0.
λ =(x(0))T
y(0).
while∥∥y(k) − λ x(k)∥∥
2≥ τ do
x(k+1) = y(k)/∥∥y(k)∥∥
2, y(k+1) = A x(k+1).
λ =(x(k+1)
)Ty(k+1).
k = k + 1.end while
Same PM, but Symmetric PM converges much faster.
I Ex 1: A =
−4 14 0−5 13 0−1 0 2
with x0) =
111
for λ1 = 6.
I Ex 2: A =
−4 −1 1−1 3 −2
1 −2 3
with x0) =
100
for λ1 = 6.
Thm: Let A ∈ Rn×n is symmetric with eigenvalues λ1, λ2, · · · , λn. Ifwe have ‖A x− λ x‖2 ≤ τ for some real number λ and unitvector x, then
min1≤j≤n |λ− λj | ≤ τ.
Proof: Let v1, v2, · · · , vn form an orthonormal set of A eigenvectorsassociated with eigenvalues λ1, λ2, · · · , λn. Then the matrix
Qdef= (v1, v2, · · · , vn) is orthogonal, and
x = β1 v1 + β2 v2 + · · ·+ βn vn
with
β1...βn
def= QT x unit vector.
‖A x− λ x‖2 = ‖β1 (λ1 − λ) v1 + · · ·+ βn (λn − λ) vn‖2
=
√β21 (λ1 − λ)2 + · · ·+ β2n (λn − λ)2
≥ (min1≤j≤n |λ− λj |)√β21 + · · ·+ β2n = min1≤j≤n |λ− λj |
Thm: Let A ∈ Rn×n is symmetric with eigenvalues λ1, λ2, · · · , λn. Ifwe have ‖A x− λ x‖2 ≤ τ for some real number λ and unitvector x, then
min1≤j≤n |λ− λj | ≤ τ.
Proof: Let v1, v2, · · · , vn form an orthonormal set of A eigenvectorsassociated with eigenvalues λ1, λ2, · · · , λn. Then the matrix
Qdef= (v1, v2, · · · , vn) is orthogonal, and
x = β1 v1 + β2 v2 + · · ·+ βn vn
with
β1...βn
def= QT x unit vector.
‖A x− λ x‖2 = ‖β1 (λ1 − λ) v1 + · · ·+ βn (λn − λ) vn‖2
=
√β21 (λ1 − λ)2 + · · ·+ β2n (λn − λ)2
≥ (min1≤j≤n |λ− λj |)√β21 + · · ·+ β2n = min1≤j≤n |λ− λj |
Thm: Let A ∈ Rn×n is symmetric with eigenvalues λ1, λ2, · · · , λn. Ifwe have ‖A x− λ x‖2 ≤ τ for some real number λ and unitvector x, then
min1≤j≤n |λ− λj | ≤ τ.
Proof: Let v1, v2, · · · , vn form an orthonormal set of A eigenvectorsassociated with eigenvalues λ1, λ2, · · · , λn. Then the matrix
Qdef= (v1, v2, · · · , vn) is orthogonal, and
x = β1 v1 + β2 v2 + · · ·+ βn vn
with
β1...βn
def= QT x unit vector.
‖A x− λ x‖2 = ‖β1 (λ1 − λ) v1 + · · ·+ βn (λn − λ) vn‖2
=
√β21 (λ1 − λ)2 + · · ·+ β2n (λn − λ)2
≥ (min1≤j≤n |λ− λj |)√β21 + · · ·+ β2n
= min1≤j≤n |λ− λj |
Thm: Let A ∈ Rn×n is symmetric with eigenvalues λ1, λ2, · · · , λn. Ifwe have ‖A x− λ x‖2 ≤ τ for some real number λ and unitvector x, then
min1≤j≤n |λ− λj | ≤ τ.
Proof: Let v1, v2, · · · , vn form an orthonormal set of A eigenvectorsassociated with eigenvalues λ1, λ2, · · · , λn. Then the matrix
Qdef= (v1, v2, · · · , vn) is orthogonal, and
x = β1 v1 + β2 v2 + · · ·+ βn vn
with
β1...βn
def= QT x unit vector.
‖A x− λ x‖2 = ‖β1 (λ1 − λ) v1 + · · ·+ βn (λn − λ) vn‖2
=
√β21 (λ1 − λ)2 + · · ·+ β2n (λn − λ)2
≥ (min1≤j≤n |λ− λj |)√β21 + · · ·+ β2n = min1≤j≤n |λ− λj |
The Inverse Power Method (I)
Given: Matrix A ∈ Rn×n, with n eigenvalues λ1, λ2, · · · , λn; andgiven shift q.
Task: Compute λi that is closest to q, and correspondingeigenvector vi .
Apply Power Method to (A− q I )−1.
The Inverse Power Method (I)
Given: Matrix A ∈ Rn×n, with n eigenvalues λ1, λ2, · · · , λn; andgiven shift q.
Task: Compute λi that is closest to q, and correspondingeigenvector vi .
Apply Power Method to (A− q I )−1.
I Matrix (A− q I )−1 has eigenvalues
1
λ1 − q,
1
λ2 − q, · · · , 1
λn − q.
I Assume q closest to λi and λk , but closer to λi .
I IPM converges to λi at order
(λi − q
λk − q
)k
.
I Matrix (A− q I )−1 has eigenvalues
1
λ1 − q,
1
λ2 − q, · · · , 1
λn − q.
I Assume q closest to λi and λk , but closer to λi .
I IPM converges to λi at order
(λi − q
λk − q
)k
.
I Matrix (A− q I )−1 has eigenvalues
1
λ1 − q,
1
λ2 − q, · · · , 1
λn − q.
I Assume q closest to λi and λk , but closer to λi .
I IPM converges to λi at order
(λi − q
λk − q
)k
.
Algorithm 3 The Inverse Power Method
Input: Matrix A ∈ Rn×n, shift q,initial guess x(0) ∈ Rn, and tolerance τ > 0.
Output: Approximate eigenvalue λ, eigenvector x.Algorithm:Normalize: x(0) = x(0)
/∥∥x(0)∥∥2, y(0) = (A− q I )−1 x(0).
λ =(x(0))T
y(0), k = 0.
while∥∥y(k) − λ x(k)∥∥
2≥ τ do
x(k+1) = y(k)/∥∥y(k)∥∥
2, y(k+1) = (A− q I )−1 x(k+1).
λ =(x(k+1)
)Ty(k+1).
k = k + 1.end while
I Symmetric/Non-symmetric PM Errors
I Symmetric IPM Errors
ReviewThm: A matrix A ∈ Rn×n is symmetric if and only if there exists a
diagonal matrix D ∈ Rn and an orthogonal matrix Q so that
A = Q D QT = Q
QT .
Proof: I Let λ be eigenvalue of A with unit eigenvector u: Au = λu.I We extend u into an orthonormal basis for Rn: u,u2, · · · ,un
are unit, mutually orthogonal vectors.
Udef= (u,u2, · · · ,un)
def=(u, U
)∈ Rn×n is orthogonal.I
UT AU =
(uT
UT
)(Au,A U
)=
uT (Au) uT(A U
)UT (Au) UT
(A U
) =
(λ 0T
0 UT(A U
) ).
I Repeat on symmetric matrix UT(A U
)∈ R(n−1)×(n−1).
ReviewThm: A matrix A ∈ Rn×n is symmetric if and only if there exists a
diagonal matrix D ∈ Rn and an orthogonal matrix Q so that
A = Q D QT = Q
QT .
Proof: I Let λ be eigenvalue of A with unit eigenvector u: Au = λu.I We extend u into an orthonormal basis for Rn: u,u2, · · · ,un
are unit, mutually orthogonal vectors.
Udef= (u,u2, · · · ,un)
def=(u, U
)∈ Rn×n is orthogonal.I
UT AU =
(uT
UT
)(Au,A U
)=
uT (Au) uT(A U
)UT (Au) UT
(A U
) =
(λ 0T
0 UT(A U
) ).
I Repeat on symmetric matrix UT(A U
)∈ R(n−1)×(n−1).
Computing all eigenvalues of matrix A ∈ Rn×n
I Compute one approximate eigenvalue λ of A with uniteigenvector u: Au = λu.
I Extend u into an orthonormal basis for Rn: u,u2, · · · ,un areunit, mutually orthogonal vectors.
Udef= (u,u2, · · · ,un)
def=(u, U
)∈ Rn×n is orthogonal.
I
UT AU =
(uT
UT
)(Au,A U
)=
uT (Au) uT(A U
)UT (Au) UT
(A U
) =
λ uT(A U
)0 UT
(A U
) . (Deflation)
I Continue on matrix Adef= UT
(A U
)∈ R(n−1)×(n−1).
Computing all eigenvalues of matrix A ∈ Rn×n
I Compute one approximate eigenvalue λ of A with uniteigenvector u: Au = λu.
I Extend u into an orthonormal basis for Rn: u,u2, · · · ,un areunit, mutually orthogonal vectors.
Udef= (u,u2, · · · ,un)
def=(u, U
)∈ Rn×n is orthogonal.
I
UT AU =
(uT
UT
)(Au,A U
)=
uT (Au) uT(A U
)UT (Au) UT
(A U
) =
λ uT(A U
)0 UT
(A U
) . (Deflation)
I Continue on matrix Adef= UT
(A U
)∈ R(n−1)×(n−1).
Householder ReflectionLet v ∈ Rn be a unit vector. Define Householder Reflection matrix
H = I − 2v vT ∈ Rn×n.
I H is symmetric and orthogonal
H = HT , H2 = I − 4v vT + 4v vT = I .
I For any vector x ∈ Rn, x†def= H x = x− 2v vTx reflects x in
the direction v⊥:
Deflation with Householder Reflection (I)
I Given eigenvalue λ of A with unit eigenvector u: Au = λu.
I Extend u into an orthonormal basis with a Householderreflection
U = I − 2v vTdef= (u,u2, · · · ,un)
def=(u, U
)I
UT AU =
λ uT(A U
)0 UT
(A U
) .
Find unit vector v so first column of I − 2v vT is u.
Deflation with Householder Reflection (II)I Partition
u =
(µu
), v =
(νv
).
I First column of I − 2v vT is(µu
)=
(10
)− 2
(νv
)ν.
I If µ ≤ 0, then
ν =
√1− µ
2, v = − u
2 ν, U = I − 2v vT =
(u, U
). (1)
I If µ > 0, then −u is also unit eigenvector. Compute v withequation (1) on −u:
ν =
√1 + µ
2, v =
u
2 ν, U = I − 2v vT =
(−u, U
). (2)
Equations (1) and (2) ensure numerical stability
Householder Reflection (II)Let v ∈ Rn be a unit vector. Define Householder Reflection matrix
H = I − 2 v vT ∈ Rn×n.
I For any vector x ∈ Rn, choose v so that
H x =
(± ‖x‖2
0
), (sign to be chosen for numerical stability.)
I
Partition x =
(ξx
),
(± ‖x‖2
0
)= H x =
(ξx
)− 2 v vTx,
udef=
(± ‖x‖2 − ξ−x
)choose===== −
(sign (ξ) (‖x‖2 + |ξ|)
x
)and v = u /‖u‖2