LU Decomposition - MATH FOR COLLEGEmathforcollege.com/nm/mws/gen/04sle/mws_gen_sle_ppt_ludecomp.pdf · LU Decomposition LU Decomposition is another method to solve a set of simultaneous

10/7/2015 http://numericalmethods.eng.usf.edu 1

LU Decomposition

Major: All Engineering Majors

Authors: Autar Kaw

http://numericalmethods.eng.usf.eduTransforming Numerical Methods Education for STEM

Undergraduates

http://numericalmethods.eng.usf.edu/

LU Decomposition

http://numericalmethods.eng.usf.edu



LU DecompositionLU Decomposition is another method to solve a set of

simultaneous linear equations

Which is better, Gauss Elimination or LU Decomposition?

To answer this, a closer look at LU decomposition is needed.

MethodFor most non-singular matrix [A] that one could conduct Naïve Gauss Elimination forward elimination steps, one can always write it as

[A] = [L][U]where

[L] = lower triangular matrix

[U] = upper triangular matrix


LU Decomposition


How does LU Decomposition work?If solving a set of linear equations

If [A] = [L][U] thenMultiply by

Which givesRemember [L]-1[L] = [I] which leads to

Now, if [I][U] = [U] thenNow, let

Which ends withand

[A][X] = [C][L][U][X] = [C][L]-1

[L]-1[L][U][X] = [L]-1[C][I][U][X] = [L]-1[C][U][X] = [L]-1[C][L]-1[C]=[Z][L][Z] = [C] (1)[U][X] = [Z] (2)


LU DecompositionHow can this be used?

Given [A][X] = [C]

1. Decompose [A] into [L] and [U]

2. Solve [L][Z] = [C] for [Z]

3. Solve [U][X] = [Z] for [X]


Is LU Decomposition better than Gaussian Elimination?

Solve [A][X] = [B]

−+=

3204

38| 2

3 nnnTCT DE

( )nnTCT FS 44| 2 −=( )nnTCT BS 124| 2 +=

−+=

3328

38| 2

3 nnnTCT FE

Forward Elimination Decomposition to LU

Back Substitution Forward Substitution

( )nnTCT BS 124| 2 +=

Back Substitution

T = clock cycle time and nxn = size of the matrix


Is LU Decomposition better than Gaussian Elimination?

To solve [A][X] = [B]Time taken by methods

T = clock cycle time and nxn = size of the matrix

So both methods are equally efficient.

Gaussian Elimination LU Decomposition

++

3412

38 2

3 nnnT

++

3412

38 2

3 nnnT


To find inverse of [A]

Time taken by Gaussian Elimination Time taken by LU Decomposition

( )

++=

+=

3412

38

||2

34 nnnT

CTCTn BSFE

−+=

×+×+=

32012

332

|||

23 nnnT

CTnCTnCT BSFSDE


To find inverse of [A]

Time taken by Gaussian Elimination Time taken by LU Decomposition

−+

32012

332 2

3 nnnT

n 10 100 1000 10000CT|inverse GE / CT|inverse LU 3.288 25.84 250.8 2501

Table 1 Comparing computational times of finding inverse of a matrix using LU decomposition and Gaussian elimination.

++

3412

38 2

34 nnnT

For large n, CT|inverse GE / CT|inverse LU ≈ n/4


Method: [A] Decomposes to [L] and [U]

[ ] [ ][ ]

==

33

2322

131211

3231

21

000

101001

uuuuuu

ULA

[U] is the same as the coefficient matrix at the end of the forward elimination step.

[L] is obtained using the multipliers that were used in the forward elimination process


Finding the [U] matrixUsing the Forward Elimination Procedure of Gauss Elimination

11214418641525

( ) 11214456.18.40

152556.212;56.2

2564

−−=−= RowRow

( ) 76.48.16056.18.40

152576.513;76.5

25144

−−−−=−= RowRow

Step 1:


Finding the [U] Matrix

Step 2:

−−−−

76.48.16056.18.40

1525

( ) 7.00056.18.40

15255.323;5.3

8.48.16

−−=−=

−− RowRow

[ ]

−−=

7.00056.18.40

1525U

Matrix after Step 1:


Finding the [L] matrix

Using the multipliers used during the Forward Elimination Procedure

101001

3231

21

56.22564

11

2121 ===

aa

76.525

144

11

3131 ===

aa

From the first step of forward elimination

11214418641525


Finding the [L] Matrix

[ ]

=

15.376.50156.2001

L

From the second step of forward elimination

−−−−

76.48.16056.18.40

15255.3

8.48.16

22

3232 =

−−

==aa


Does [L][U] = [A]?

[ ][ ] =

−−

=

7.00056.18.40

1525

15.376.50156.2001

UL ?


Using LU Decomposition to solve SLEs

Solve the following set of linear equations using LU Decomposition

=

227921778106

11214418641525

3

2

1

.

.

.

xxx

Using the procedure for finding the [L] and [U] matrices

[ ] [ ][ ]

−−

==

7.00056.18.40

1525

15.376.50156.2001

ULA


Example

Set [L][Z] = [C]

Solve for [Z]

=

2.2792.1778.106

15.376.50156.2001

3

2

1

zzz

2.2795.376.52.17756.2

10

321

21

1

=++=+

=

zzzzz

z


ExampleComplete the forward substitution to solve for [Z]

( )

( ) ( )735.0

21.965.38.10676.52.2795.376.52.279

2.968.10656.22.177

56.22.1778.106

213

12

1

=−−−=

−−=−=

−=−=

=

zzz

zzz

[ ]

−=

=

735.021.968.106

3

2

1

zzz

Z


ExampleSet [U][X] = [Z]

Solve for [X] The 3 equations become

−=

−−

735021968106

7.00056.18.40

1525

3

2

1

...

xxx

735.07.021.9656.18.48.106525

3

32

321

=−=−−

=++

aaaaaa


Example

From the 3rd equation

050170

7350735070

3

3

3

.a.

.a

.a.

=

=

=

Substituting in a3 and using the second equation

219656184 32 .a.a. −=−−

( )

701984

0501561219684

5612196

2

2

32

.a.

...a

.a..a

=−+−

=

−+−

=


ExampleSubstituting in a3 and a2 using the first equation

8106525 321 .aaa =++

Hence the Solution Vector is:

=

050.170.19

2900.0

3

2

1

aaa

( )

2900025

05017019581062558106 32

1

.

...

aa.a

=

−−=

−−=


Finding the inverse of a square matrix

The inverse [B] of a square matrix [A] is defined as

[A][B] = [I] = [B][A]


Finding the inverse of a square matrixHow can LU Decomposition be used to find the inverse?Assume the first column of [B] to be [b11 b12 … bn1]T

Using this and the definition of matrix multiplication

First column of [B] Second column of [B]

[ ]

=

0

01

1

21

11

nb

bb

A [ ]

=

0

10

2

22

12

nb

bb

A

The remaining columns in [B] can be found in the same manner


Example: Inverse of a MatrixFind the inverse of a square matrix [A]

[ ]

=

11214418641525

A

[ ] [ ][ ]

−−

==

7000561840

1525

15376501562001

.

.. ..

.ULA

Using the decomposition procedure, the [L] and [U] matrices are found to be


Example: Inverse of a MatrixSolving for the each column of [B] requires two steps

1) Solve [L] [Z] = [C] for [Z]

2) Solve [U] [X] = [Z] for [X]

Step 1: [ ][ ] [ ]

=

→=

001

15.376.50156.2001

3

2

1

zzz

CZL

This generates the equations:

05.376.5056.21

321

21

1

=++=+=

zzzzzz


Example: Inverse of a MatrixSolving for [Z]

( )

( ) ( )23

5625317650537650

56215620

56201

213

12

1

....

z.z.z.

.z. z

z

=−−−=

−−=−=−=−=

=

[ ]

−=

=

23562

1

3

2

1

..

zzz

Z


Example: Inverse of a Matrix

Solving [U][X] = [Z] for [X]

−=

−−

3.22.561

7.00056.18.40

1525

31

21

11

bbb

2.37.056.256.18.4

1525

31

3121

312111

=−=−−

=++

bbbbbb


Example: Inverse of a MatrixUsing Backward Substitution

( )

( ) 04762.025

571.49524.05125

51

9524.08.4

571.4560.156.28.4560.156.2

571.47.02.3

312111

3121

31

=−−−

=

−−=

−=−

+−=

−+−

=

==

bbb

bb

b So the first column of the inverse of [A] is:

−=

571.49524.0

04762.0

31

21

11

bbb


Example: Inverse of a MatrixRepeating for the second and third columns of the inverse

Second Column Third Column

=

010

11214418641525

32

22

12

bbb

−

−=

000.5417.108333.0

32

22

12

bbb

=

100

11214418641525

33

23

13

bbb

−=

429.14643.0

03571.0

33

23

13

bbb


Example: Inverse of a MatrixThe inverse of [A] is

[ ]

−−−

−=−

429.1000.5571.44643.0417.19524.0

03571.008333.004762.01A

To check your work do the following operation

[A][A]-1 = [I] = [A]-1[A]

Additional ResourcesFor all resources on this topic such as digital audiovisual lectures, primers, textbook chapters, multiple-choice tests, worksheets in MATLAB, MATHEMATICA, MathCad and MAPLE, blogs, related physical problems, please visit

http://numericalmethods.eng.usf.edu/topics/lu_decomposition.html

http://numericalmethods.eng.usf.edu/topics/lu_decomposition.html

THE END



Documents

LU Decomposition - MATH FOR COLLEGEmathforcollege.com/nm/mws/gen/04sle/mws_gen_sle_ppt_ludecomp.pdf · LU Decomposition LU Decomposition is another method to solve a set of simultaneous