Advanced Matrix Algebraand Applications

- Swan and Khare

Bootes :

Y fundamentals of matrix computations- David Watkins

og Matrix Algebra . . .

- S . Boyd

Lectures :

Inner product .

( Dot product )

IR"

I R2 -11123

a, Y c- Irn

u= ( !: ),

Y -- ( by:')

n

< a ,y > = .nTy = Eni Yi

c- = I

< se ,n7 = aTn= II.nil = Hoell?

Cosas LseHIM 11311

C s . in equability ⇒ - IECOSOE '

n

* is C- IR are such that

< re ,y7= UTS so

LU

Cholesky

o.si#a....s::auan

Mxn

A Elk

b C- C- IRM

Ql : Does there exist ' MEIR"

s .'t .

A n=b ? ?

Hain!-ai:] : . -

. I ?:)a . -

. la . -- - any:) = :?:&:

At span (A) = { a A,

ta Azt - - + Ln An )

A,

- - - INE IR and

As,

. . .

,An are columns

of A }- Very easy to prove that cot span CA )

is a subspace of IR'm

m

If b GIR is in the cot spanCA )

,

then b can be written as a

linear combination of Ai 's.

F Ri,

R2,

- -

,In EIR S . t -

n

E Aioli = b-

El

⇒ Ax=b

⇒ solution exists ? ?

T

¥'

#y Uniqueness

The columns of A are in fact

basis vectors for the column spaceof A -

E Relate the geometrical concepts with

algebraic constraints onrank CA

'

:b )

Solve:

A x=b

I '

DirectIterative

methodsmethods

i÷:÷÷÷t÷HA A

Use = b

Annan = bn

Nn =I bn

U'hnUn

- i ,n , Nnn t Um,

n kn = bn ,

Unna L [ bn . ,- Unum kn )

Unt,

n I

Efeoatingpoint operations ftops-

Floating Point Arithmetic

Similarly,

Lx = b where L is an

lower triangular system can also be

solved very easily by forward

substitution .

In general ,we will try to convert

Ax=b problem into an equivalent

upper triangular or lower triangular

system .

LU,

cholesky,

QR

II

symmetric positive definite matrices .

nxn

A EIR

A : symmetric .

UTAH > o Ha EIR"

Ax -b where symmetric positive

definite -

A = µJµ principal minors

Lecture 2

-

"

. Gaussian Elimination ( pivoting )

. i:÷÷÷÷: in tilI Aib ]

" it'

• LU

nxn

A GIR non . singular .

A -

-

LDU

C . :iIf÷.

. :V÷:3

Ax =b

L @x) =b

y

Ly --

b

Ux = y

-

Ax = b

l :#A#"sy#

A =LE

A is symmetric p . d .

A- = LDLT where D has

only positive

diagonal entries.

TA = GG

where G -

-LD

"

-

To solve Ax=b

In practice we generally solve

a perturbed system .

( At # ( xtrx ) = I btsb )

II =D

Suppose that one can solve IF =T

exactly far Te.

Then we declare Te to be the solution

to Ax=b .

Can we justify this ? ?

Answer : Not always i ! C Sensitivity- analysis )

Exi Ax=b

a- f :::499.3in .li:::)Note : x -

- f ! ) is the solution.

a Ii::::is i ::I÷:p .

Now solve :

~

A I = b

e.I:i:D .

a :c :::::L it :::::D

e. as

's =L.

t.gg:::X ::::

I =

f-998×1999.01 -1999×1997.01

⇐go.itwe observe here that a

' '

small"

perturbation

is b causes a

"

very large"

perturbationin x .

A

Q : Can we quantify this perturbationin x ? ?

Norris :

Vector norm .

11 .11 : 112

"

→ IR

I) Hall 70 An EIR"

and Halle o IH us o

ii ) Hazell = 121 11h11

iii ) until ) E HMH -11191142

c.) Halla = TE - ( Eani )t

C. ) use 'lp= ( II.initb)" b AZ '

=

is a vector norm .

n

Hall,

= I InitE- I

II Hallo =

miaxtail

Matrix norms :

nxn

A E R

Consider A as a vector . in ik ?

" AHF = ( §; ait )"

is in fact the 2- norm of the

vector in IRR.

Induced.

matrix norm

-

:

11 All,

=Max HANEO 11kHz

MAR=

a±oHA

aII. Hz

= Mak It Ayllz119112=1

in

Lecture 4 .

least squares problem .

.AE RN "N > 7h . } Given

b C- IRN

solve : A *= b

lattes :÷÷÷:ee?

. .

- Cl )

Geometry , N-

IR

←solution

to

son . . , .

'÷:

Lists! " tangespaalColumn space af A

b

Hit.

-

testimate,±s.ae

Find I such that

A- I I b -AI

⇒CAI

,b - AI > = O

⇒ IT At f b - AI ) = 0

⇒ItAtb = IT AT AT

Atb = antafI why ? ?

⇒← Normal Eq ?

ATA c- qznxn

Solving the normal eg? Atb EIR

" '

CATA) I = AT b

we get the LS solution to the

system Ax= b

In particular ,if ATA is invertible

men /i⇒b'

-

The matrix ( ATA)t AT is known as

pseudo - inverse of A and is denoted

as At.

Qi Why is this solution called

Least-squares CLS ) ? ?

min 1lb - Axllz←

MEIR"

Min 1lb - Aall ? c-⑧

ME IN

Hb - Axil ? =Cb - Ax

,

b - Ax )

= ( b -Axjt C b - At )

=fbIxTaT ) Cb - AH

=

BTK- XTAT b - TAX f-

-

AT At A x

= FAT Ax-

zxb+

BT b

min XTATAX - 2*TAtb t BT b

XGIR" #

cost f ?

pa @t.fr/--o2ATAx-2ATb--

0

⇒ ATA x -

-Atb

ss - Asb . * ,

" - ft:)a- f) " "

Hm,

NT

" b- *

* .

< b - Ax,

b - Ax ) = Sr,

-7n

= Eri?

it

-

Given Adb,

to solve the LS problem,

one needs to construct normal equations

and then solve them simultaneously .

ATA ,i= Atb- -

i ) computational complexity of computing

ATA.

Iii ) As

µg)

I1ATA -

-

fi, E)

µ , y )=L::*) ⑨

Numerical approach to solve LS problem .

A- x=b A- ERN 'm

NxtBER

To find X s - t . 1lb -Axtlz is

minimized.

Hb - Axil ! = Lb - Ax,

b- Ax >

= LQCB - Axl,

Qcb - Axl >

= ( b -ADT QTQ Cb - Ax )-

I#

If Q is such that QTQ -- I

MQT C b- Ax ) II = 1lb - Axliz

"

11 Qtb - QTAXH? = Kotb -Rall ?

-

R

=\