The Pseudoinverse - Moore-Penrose Inverse and Least Squaresbuzzard.ups.edu/courses/2014spring/420projects/math420-UPS-spring... · The Pseudoinverse Construction Application The Pseudoinverse

The PseudoinverseConstruction

Application

The PseudoinverseMoore-Penrose Inverse and Least Squares

Ross MacAusland

University of Puget Sound

April 23, 2014

Ross MacAusland Pseudoinverse


Application

Outline

1 The PseudoinverseGeneralized inverseMoore-Penrose Inverse

2 ConstructionQR DecompositionSVD

3 ApplicationLeast Squares



Application

Generalized inverseMoore-Penrose Inverse

Table of Contents






Application


What is the Generalized Inverse?

Any inverse-like matrix

Satisfies AA−A = AGuaranteed existence, not uniqueness.

Example

ALA = A(LA) = AI = AARA = (AR)A = IA = A



Application



Any inverse-like matrixSatisfies AA−A = A

Guaranteed existence, not uniqueness.

Example




Application



Any inverse-like matrixSatisfies AA−A = AGuaranteed existence, not uniqueness.

Example




Application



Any inverse-like matrixSatisfies AA−A = AGuaranteed existence, not uniqueness.

Example




Application


Defining the Pseudoinverse

Definition

If A ∈Mn×m, then there exists a unique A+ ∈Mm×n thatsatisfies the four Penrose conditions:

1 AA+A = A2 A+AA+ = A+

3 A+A = (A+A)∗ Hermitian4 AA+ = (AA+)∗ Hermitian



Application


Properties of the Pseudoinverse

Guaranteed existence and uniqueness

If A is nonsingular A+ = A−1

The pseudoinverse of the pseudoinverse is the originalmatrix (A+)+ = A(AB)+ = B+A+



Application



Guaranteed existence and uniquenessIf A is nonsingular A+ = A−1




Application




The pseudoinverse of the pseudoinverse is the originalmatrix (A+)+ = A

(AB)+ = B+A+



Application







Application



For any A ∈ Cn×m there exists a A+ ∈ Cm×n

N(A∗) = N(A+) and R(A∗) = R(A+)

R(A)⊕ N(A+) = Cn and R(A+)⊕ N(A) = Cm

HH+ is an orthogonal projection onto R(A), and usingsimilar H+H is an orthogonal projection onto N(A).



Application




N(A∗) = N(A+) and R(A∗) = R(A+)





Application




N(A∗) = N(A+) and R(A∗) = R(A+)





Application

QR DecompositionSVD

Table of Contents






Application

QR DecompositionSVD

QR Decomposition

There are two ways of constructing the Pseudoinverse usingQR.

If A is n ×m and n > m with rank equal to m then

A = Q[R1O

]

Then the pseudoinverse can be found byA+ =

[R−1

1 O∗]Q∗.

This only works if rank is equal to the minimum of m andn



Application

QR DecompositionSVD

QR Decomposition



A = Q[R1O

]Then the pseudoinverse can be found byA+ =

[R−1

1 O∗]Q∗.




Application

QR DecompositionSVD

QR Decomposition



A = Q[R1O

]Then the pseudoinverse can be found byA+ =

[R−1

1 O∗]Q∗.




Application

QR DecompositionSVD

QR Decomposition

The second way finds the pseudoinverse for the singular case

If A is n × n and rank is less than n then A = Q[R1 00 0

]U∗

Then the pseudoinverse can be found by Then the

pseudoinverse can be found by A+ = U[R−1

1 00 0

]Q∗



Application

QR DecompositionSVD

QR Decomposition

The second way finds the pseudoinverse for the singular case

If A is n × n and rank is less than n then A = Q[R1 00 0

]U∗

Then the pseudoinverse can be found by Then the

pseudoinverse can be found by A+ = U[R−1

1 00 0

]Q∗



Application

QR DecompositionSVD

QR Example

Example

Let A =

1 −1 41 4 −21 4 21 −1 0

Q =

1/2 −1/2 1/21/2 1/2 −1/21/2 1/2 1/21/2 −1/2 −1/2

R =

2 3 20 5 −20 0 40 0 0

R−1

1 =

1/2 −3/10 −2/50 1/5 1/100 0 1/4



Application

QR DecompositionSVD

QR Example

Example

Let A =

1 −1 41 4 −21 4 21 −1 0

Q =

1/2 −1/2 1/21/2 1/2 −1/21/2 1/2 1/21/2 −1/2 −1/2

R =

2 3 20 5 −20 0 40 0 0

R−11 =

1/2 −3/10 −2/50 1/5 1/100 0 1/4



Application

QR DecompositionSVD

QR Example

Example

Let A =

1 −1 41 4 −21 4 21 −1 0

Q =

1/2 −1/2 1/21/2 1/2 −1/21/2 1/2 1/21/2 −1/2 −1/2

R =

2 3 20 5 −20 0 40 0 0

R−1

1 =

1/2 −3/10 −2/50 1/5 1/100 0 1/4



Application

QR DecompositionSVD

QR Example

Example

A+ =[R−1

1 O∗]Q∗

A+ =

1/5 3/10 −1/10 3/5−1/20 1/20 3/20 −3/20

1/8 −1/8 1/8 −1/8

A+A =

1 0 00 1 00 0 1



Application

QR DecompositionSVD

QR Example

Example

A+ =[R−1

1 O∗]Q∗

A+ =

1/5 3/10 −1/10 3/5−1/20 1/20 3/20 −3/20

1/8 −1/8 1/8 −1/8

A+A =

1 0 00 1 00 0 1



Application

QR DecompositionSVD

SVD Construction

SVD can be represented by A = UDV ∗

or by A =∑r

i=1 sixiyi∗

the pseudoinverse can be found by A+ = VD+U∗

or A+ =∑r

i=1 s−1i yixi

∗



Application

QR DecompositionSVD

SVD Construction

SVD can be represented by A = UDV ∗

or by A =∑r

i=1 sixiyi∗

the pseudoinverse can be found by A+ = VD+U∗

or A+ =∑r

i=1 s−1i yixi

∗



Application

QR DecompositionSVD

SVD Example

Example

A =

1 00 1√3 0

then D =

2 00 10 0

Solving for D+, D+ =

[1/2 0 00 1 0

]and

A+ =

[.25 00.4330127018920 1 0

]A+A =

[1 00 1

]



Application

QR DecompositionSVD

SVD Example

Example

A =

1 00 1√3 0

then D =

2 00 10 0

Solving for D+, D+ =

[1/2 0 00 1 0

]and

A+ =

[.25 00.4330127018920 1 0

]A+A =

[1 00 1

]



ApplicationLeast Squares

Table of Contents







Least Squares Review

Ax = bx0 = A−1b

When A is singular, A−1 does not existx0 = A+b = (A∗A)−1A∗b = A+bA has full column rank, n > m. A+ solves the least squaressolutionsimilarly A+ = A∗(AA∗)−1 when A has full row rank.





Ax = bx0 = A−1bWhen A is singular, A−1 does not exist

x0 = A+b = (A∗A)−1A∗b = A+bA has full column rank, n > m. A+ solves the least squaressolutionsimilarly A+ = A∗(AA∗)−1 when A has full row rank.





Ax = bx0 = A−1bWhen A is singular, A−1 does not existx0 = A+b = (A∗A)−1A∗b = A+bA has full column rank, n > m. A+ solves the least squaressolution

similarly A+ = A∗(AA∗)−1 when A has full row rank.





Ax = bx0 = A−1bWhen A is singular, A−1 does not existx0 = A+b = (A∗A)−1A∗b = A+bA has full column rank, n > m. A+ solves the least squaressolutionsimilarly A+ = A∗(AA∗)−1 when A has full row rank.




Digital Image Restoration

Recovery of a degraded images

Many algorithms are used for a variety of reasonsThe Improvement in signal-to-noise ratio (ISNR) andrequired computational time compare algorithmsMoore-Penrose inverse is one of the most efficientalgorithms





Recovery of a degraded imagesMany algorithms are used for a variety of reasons

The Improvement in signal-to-noise ratio (ISNR) andrequired computational time compare algorithmsMoore-Penrose inverse is one of the most efficientalgorithms





Recovery of a degraded imagesMany algorithms are used for a variety of reasonsThe Improvement in signal-to-noise ratio (ISNR) andrequired computational time compare algorithms

Moore-Penrose inverse is one of the most efficientalgorithms





Recovery of a degraded imagesMany algorithms are used for a variety of reasonsThe Improvement in signal-to-noise ratio (ISNR) andrequired computational time compare algorithmsMoore-Penrose inverse is one of the most efficientalgorithms




Digital Image Restoration Example

xin = H+xout

Where H is the matrix representation of how the imagewas degraded by a uniform linear motion














Appendix For Further Reading

References

Applications of the Moore-Penrose Inverse in Digital ImageRestorationSpiros Chountasis, Vasilios N. Katsikis, and DimitriosPappas,Mathematical Problems in Engineering, vol. 2009, Article ID170724

Generalized Inverses: Theory and ApplicationsAdi Ben-Israel, Thomas N.E.Greville; 2001

Matrix Algebra: Theory, Computations, and Applications inStatisticsJames E. Gentle;New York, NY: Springer, 2007.


Documents

The Pseudoinverse - Moore-Penrose Inverse and Least Squaresbuzzard.ups.edu/courses/2014spring/420projects/math420-UPS-spring... · The Pseudoinverse Construction Application The Pseudoinverse