3/1/2016 1 LU Decomposition Computer Engineering Majors Authors: Autar Kaw Transforming

05/08/23http://

numericalmethods.eng.usf.edu 1

LU Decomposition

Computer Engineering Majors

Authors: Autar Kaw

http://numericalmethods.eng.usf.eduTransforming Numerical Methods Education for STEM

Undergraduates

http://numericalmethods.eng.usf.edu/


LU Decomposition

http://numericalmethods.eng.usf.edu




LU DecompositionLU Decomposition is another method to solve a set of

simultaneous linear equations

Which is better, Gauss Elimination or LU Decomposition?

To answer this, a closer look at LU decomposition is needed.

MethodFor most non-singular matrix [A] that one could conduct Naïve Gauss Elimination forward elimination steps, one can always write it as

[A] = [L][U]where

[L] = lower triangular matrix

[U] = upper triangular matrix


LU Decomposition


How does LU Decomposition work?


LU DecompositionHow can this be used?

Given [A][X] = [C]

1. Decompose [A] into [L] and [U]

2. Solve [L][Z] = [C] for [Z]

3. Solve [U][X] = [Z] for [X]


When is LU Decomposition better than Gaussian Elimination?

To solve [A][X] = [B]Table. Time taken by methods

where T = clock cycle time and n = size of the matrix

So both methods are equally efficient.

Gaussian Elimination LU Decomposition

3412

38 2

3 nnnT

3412

38 2

3 nnnT


To find inverse of [A]Time taken by Gaussian Elimination Time taken by LU

Decomposition

3412

38

||2

34 nnnT

CTCTn BSFE

32012

332

|||

23 nnnT

CTnCTnCT BSFSLU

n 10 100 1000 10000CT|inverse GE / CT|inverse LU 3.28 25.83 250.8 2501

Table 1 Comparing computational times of finding inverse of a matrix using LU decomposition and Gaussian elimination.


Method: [A] Decompose to [L] and [U]

33

2322

131211

3231

21

000

101001

uuuuuu

ULA

[U] is the same as the coefficient matrix at the end of the forward elimination step.

[L] is obtained using the multipliers that were used in the forward elimination process


Finding the [U] matrixUsing the Forward Elimination Procedure of Gauss Elimination

11214418641525

11214456.18.40

152556.212;56.2

2564

RowRow

76.48.16056.18.40

152576.513;76.5

25144

RowRow

Step 1:


Finding the [U] Matrix

Step 2:

76.48.16056.18.40

1525

7.00056.18.40

15255.323;5.3

8.48.16

RowRow

7.00056.18.40

1525U

Matrix after Step 1:


Finding the [L] matrix

Using the multipliers used during the Forward Elimination Procedure

101001

3231

21

56.22564

11

2121

aa

76.525

144

11

3131

aa

From the first step of forward elimination

11214418641525


Finding the [L] Matrix

15.376.50156.2001

L

From the second step of forward elimination

76.48.16056.18.40

15255.3

8.48.16

22

3232

aa


Does [L][U] = [A]?

7.00056.18.40

1525

15.376.50156.2001

UL ?

Example: Surface Shape DetectionTo infer the surface shape of an object from images taken of a surface from three different directions, one needs to solve the following set of equations

239248247

9428.02357.02357.09701.02425.009701.002425.0

3

2

1

xxx

The right hand side are the light intensities from the middle of the images, while the coefficient matrix is dependent on the light source directions with respect to the camera. The unknowns are the incident intensities that will determine the shape of the object.


Example: Surface Shape DetectionThe solution for the unknowns x1, x2, and x3 is given by

Find the values of x1, x2,and x3 using LU Decomposition.

239248247

9428.02357.02357.09701.02425.009701.002425.0

3

2

1

xxx


Example: Surface Shape DetectionUse the Forward Elimination Procedure to find the [U] matrix.

9428.02357.02357.09701.02425.009701.002425.0

.......

RowRow.

9428023570235709701024250097010024250

012;0242500

......

RowRow..

885712357009701024250097010024250

97196.013;97196.02425023570

Step 1


Example: Surface Shape Detection

8857.12357.009701.02425.009701.002425.0

.....

RowRow..

82862009701024250097010024250

97196.022;97196.02425023570

8286.2009701.02425.009701.002425.0

U


This is the matrix after the 1st step:

Step 2

Example: Surface Shape DetectionUse the multipliers from Forward Elimination

02425.00

11

2121

aa

9719602425023570

11

3131 .

..

aa

From the 1st step of forward elimination

9428.02357.02357.09701.02425.009701.002425.0


101001

3231

21


197196.097196.0010001

101001

3231

21

L


From the 2nd step of forward elimination

9719602425023570

22

3232 .

..

aa

8857.12357.009701.02425.009701.002425.0


Does [L][U] = [A] ?

?8286.2009701.02425.009701.002425.0

197196.097196.0010001

UL



Set [L][Z] = [C]

239248247

zzz

197196.097196.0010001

3

2

1

2399719609719602480247

321

21

1

zz.z.zzz


Solve for [Z]

Example: Surface Shape DetectionSolve for [Z]

247z1

248z0248z 12

12.72024897196.024797196.0239

z972196.0z97196.0239z 213

12.720248247

zzz

Z

3

2

1


Example: Surface Shape DetectionSet [U][X] = [Z]

12.720248247

xxx

8286.2009701.02425.009701.002425.0

3

2

1

12.720x8286.2248x9701.0x2425.0247x9701.0x0x2425.0

3

32

321


Solve for [X]The 3 equations become


59.2548286.2

12.720x3

2328.42425.0

)59.254()9701.0(2482425.0

x)9701.0(248x 32


Solve for [X]


Hence the Solution Vector is:

10905.02425.0

)59.254()9701.0(2472425.0

x)9701.0(x0247 x 321

59.2542328.4

10905.0

3

2

1

xxx



Finding the inverse of a square matrix

The inverse [B] of a square matrix [A] is defined as

[A][B] = [I] = [B][A]


Finding the inverse of a square matrixHow can LU Decomposition be used to find the inverse?Assume the first column of [B] to be [b11 b12 … bn1]T

Using this and the definition of matrix multiplication

First column of [B] Second column of [B]

0

01

1

21

11

nb

bb

A

0

10

2

22

12

nb

bb

A

The remaining columns in [B] can be found in the same manner


Example: Inverse of a MatrixFind the inverse of a square matrix [A]

11214418641525

A

7000561840

1525

15376501562001

.

.. ..

.ULA

Using the decomposition procedure, the [L] and [U] matrices are found to be


Example: Inverse of a MatrixSolving for the each column of [B] requires two steps

1)Solve [L] [Z] = [C] for [Z]

2)Solve [U] [X] = [Z] for [X]

Step 1:

001

15.376.50156.2001

3

2

1

zzz

CZL

This generates the equations:

05.376.5056.21

321

21

1

zzzzzz


Example: Inverse of a MatrixSolving for [Z]

23

5625317650537650

56215620

56201

213

12

1

....

z.z.z.

.z. z

z

23562

1

3

2

1

..

zzz

Z


Example: Inverse of a Matrix

Solving [U][X] = [Z] for [X]

3.22.561

7.00056.18.40

1525

31

21

11

bbb

2.37.056.256.18.4

1525

31

3121

312111

bbbbbb


Example: Inverse of a MatrixUsing Backward Substitution

04762.025

571.49524.05125

51

9524.08.4

571.4560.156.28.4560.156.2

571.47.02.3

312111

3121

31

bbb

bb

b So the first column of the inverse of [A] is:

571.49524.0

04762.0

31

21

11

bbb


Example: Inverse of a MatrixRepeating for the second and third columns of the inverse

Second Column Third Column

010

11214418641525

32

22

12

bbb

000.5417.108333.0

32

22

12

bbb

100

11214418641525

33

23

13

bbb

429.14643.0

03571.0

33

23

13

bbb


Example: Inverse of a MatrixThe inverse of [A] is

429.1000.5571.44643.0417.19524.0

03571.008333.004762.01A

To check your work do the following operation

[A][A]-1 = [I] = [A]-1[A]

Additional ResourcesFor all resources on this topic such as digital audiovisual lectures, primers, textbook chapters, multiple-choice tests, worksheets in MATLAB, MATHEMATICA, MathCad and MAPLE, blogs, related physical problems, please visit http://numericalmethods.eng.usf.edu/topics/lu_decomposition.html

http://numericalmethods.eng.usf.edu/topics/gaussian_elimination.html

http://numericalmethods.eng.usf.edu/topics/lu_decomposition.html

http://numericalmethods.eng.usf.edu/topics/lu_decomposition.html

THE END



Documents

3/1/2016 1 LU Decomposition Computer Engineering Majors Authors: Autar Kaw Transforming