1/18/2016 1 LU Decomposition Industrial Engineering Majors Authors: Autar Kaw Transforming

05/03/23http://

numericalmethods.eng.usf.edu 1

LU Decomposition

Industrial Engineering Majors

Authors: Autar Kaw

http://numericalmethods.eng.usf.eduTransforming Numerical Methods Education for STEM

Undergraduates

LU Decomposition

http://numericalmethods.eng.usf.edu

http://numericalmethods.eng.usf.edu

LU DecompositionLU Decomposition is another method to solve a set of

simultaneous linear equations

Which is better, Gauss Elimination or LU Decomposition?

To answer this, a closer look at LU decomposition is needed.

MethodFor most non-singular matrix [A] that one could conduct Naïve Gauss Elimination forward elimination steps, one can always write it as

[A] = [L][U]where

[L] = lower triangular matrix

[U] = upper triangular matrix

http://numericalmethods.eng.usf.edu

LU Decomposition

http://numericalmethods.eng.usf.edu

How does LU Decomposition work?

http://numericalmethods.eng.usf.edu

LU DecompositionHow can this be used?

Given [A][X] = [C]

1. Decompose [A] into [L] and [U]

2. Solve [L][Z] = [C] for [Z]

3. Solve [U][X] = [Z] for [X]

http://numericalmethods.eng.usf.edu

When is LU Decomposition better than Gaussian Elimination?

To solve [A][X] = [B]Table. Time taken by methods

where T = clock cycle time and n = size of the matrix

So both methods are equally efficient.

Gaussian Elimination LU Decomposition

3412

38 2

3 nnnT

3412

38 2

3 nnnT

http://numericalmethods.eng.usf.edu

To find inverse of [A]Time taken by Gaussian Elimination Time taken by LU

Decomposition

3412

38

||2

34 nnnT

CTCTn BSFE

32012

332

|||

23 nnnT

CTnCTnCT BSFSLU

n 10 100 1000 10000CT|inverse GE / CT|inverse LU 3.28 25.83 250.8 2501

Table 1 Comparing computational times of finding inverse of a matrix using LU decomposition and Gaussian elimination.

http://numericalmethods.eng.usf.edu

Method: [A] Decompose to [L] and [U]

33

2322

131211

3231

21

000

101001

uuuuuu

ULA

[U] is the same as the coefficient matrix at the end of the forward elimination step.

[L] is obtained using the multipliers that were used in the forward elimination process

http://numericalmethods.eng.usf.edu

Finding the [U] matrixUsing the Forward Elimination Procedure of Gauss Elimination

11214418641525

11214456.18.40

152556.212;56.2

2564

RowRow

76.48.16056.18.40

152576.513;76.5

25144

RowRow

Step 1:

http://numericalmethods.eng.usf.edu

Finding the [U] Matrix

Step 2:

76.48.16056.18.40

1525

7.00056.18.40

15255.323;5.3

8.48.16

RowRow

7.00056.18.40

1525U

Matrix after Step 1:

http://numericalmethods.eng.usf.edu

Finding the [L] matrix

Using the multipliers used during the Forward Elimination Procedure

101001

3231

21

56.22564

11

2121

aa

76.525

144

11

3131

aa

From the first step of forward elimination

11214418641525

http://numericalmethods.eng.usf.edu

Finding the [L] Matrix

15.376.50156.2001

L

From the second step of forward elimination

76.48.16056.18.40

15255.3

8.48.16

22

3232

aa

http://numericalmethods.eng.usf.edu

Does [L][U] = [A]?

7.00056.18.40

1525

15.376.50156.2001

UL ?

Example: Production Optimization

To find the number of toys a company should manufacture per day to optimally use their injection-molding machine and the assembly line, one needs to solve the following set of equations. The unknowns are the number of toys for boys, x1, number of toys for girls, x2, and the number of unisexual toys, x3.

01260756

000001051333306667016670666701667033330

3

2

1

xxx

.........

Find the values of x1, x2,and x3 using LU Decomposition.

http://numericalmethods.eng.usf.edu

Example: Production Optimization

Use Forward Elimination to find the [U] matrix

00.000.105.13333.06667.01667.06667.01667.03333.0

00.000.105.1

00015002.058332.006667.01667.03333.0

50015.012;50015.03333.01667.0

RowRow

1003.25252.10

00015002.058332.006667.01667.03333.0

1503.313;1503.33333.005.1

RowRow

Step 1

http://numericalmethods.eng.usf.edu

Example: Production Optimization

1003.25252.10

00015002.058332.006667.01667.03333.0

1007.200

00015002.058332.006667.01667.03333.0

6146.223;6146.258332.0

5252.1

RowRow

1007.20000015002.058332.00

6667.01667.03333.0U

This is the matrix after the 1st step:

Step 2

http://numericalmethods.eng.usf.edu

Example: Production Optimization

Use the multipliers from Forward Elimination

101001

3231

21

50015.03333.01667.0

aa

11

2121

1503.33333.005.1

aa

11

3131

From the 1st step of forward elimination

00.000.105.13333.06667.01667.06667.01667.03333.0

http://numericalmethods.eng.usf.edu

Example: Production Optimization

16146.21503.30150015.0001

L

From the 2nd step of forward elimination

6146.258332.0

5252.1aa

22

3232

1003.252516.100002.05833.00

6667.01667.03333.0

http://numericalmethods.eng.usf.edu

Example: Production Optimization

Does [L][U] = [A] ?

?1007.200

00015002.058332.006667.01667.03333.0

16146.21503.30150015.0001

UL

http://numericalmethods.eng.usf.edu

Example: Production Optimization

Set [L][Z] = [C]

01260756

1614621503301500150001

3

2

1

zzz

...

06146215033126050015.0756

31

21

1

z.z.zzz

Solve for [Z]

http://numericalmethods.eng.usf.edu

Example: Production Optimization

7561 z

89.88175650015.01260

50015.01260 12

zz

864758988161462756150330

61462150330 213

....

z.z.z

864.7589.881

756

zzz

3

2

1

Z

Solve for [Z]

http://numericalmethods.eng.usf.edu

Example: Production Optimization

Set [U][X] = [Z]

8647589881

756

10072000001500205833200

666701667033330

3

2

1

..

xxx

.

.....

864.751007.2

8988100020583307566667.01667.03333.0

3

32

321

x.x.x.

xxx

Solve for [X]The 3 equations become

http://numericalmethods.eng.usf.edu

Example: Production Optimization

113.361007.2

864.753

x

8.1511583320

113.3600015002089881583320

00015002089881 32

...

.x..x

Solve for [X]

http://numericalmethods.eng.usf.edu

Example: Production Optimization

8.14393333.0

113.366667.08.15111667.07563333.0

6667.01667.0756 321

xxx

113369151181439

3

2

1

...

xxx

Solve for [X] cont.

http://numericalmethods.eng.usf.edu

Example: Production Optimization

The solution vector is

113369151181439

3

2

1

...

xxx

1440 toys for boys should be produced1512 toys for girls should be produced36 unisexual toys should be produced

http://numericalmethods.eng.usf.edu

http://numericalmethods.eng.usf.edu

Finding the inverse of a square matrix

The inverse [B] of a square matrix [A] is defined as

[A][B] = [I] = [B][A]

http://numericalmethods.eng.usf.edu

Finding the inverse of a square matrixHow can LU Decomposition be used to find the inverse?Assume the first column of [B] to be [b11 b12 … bn1]T

Using this and the definition of matrix multiplication

First column of [B] Second column of [B]

0

01

1

21

11

nb

bb

A

0

10

2

22

12

nb

bb

A

The remaining columns in [B] can be found in the same manner

http://numericalmethods.eng.usf.edu

Example: Inverse of a MatrixFind the inverse of a square matrix [A]

11214418641525

A

7000561840

1525

15376501562001

.

.. ..

.ULA

Using the decomposition procedure, the [L] and [U] matrices are found to be

http://numericalmethods.eng.usf.edu

Example: Inverse of a MatrixSolving for the each column of [B] requires two steps

1)Solve [L] [Z] = [C] for [Z]

2)Solve [U] [X] = [Z] for [X]

Step 1:

001

15.376.50156.2001

3

2

1

zzz

CZL

This generates the equations:

05.376.5056.21

321

21

1

zzzzzz

http://numericalmethods.eng.usf.edu

Example: Inverse of a MatrixSolving for [Z]

23

5625317650537650

56215620

56201

213

12

1

....

z.z.z.

.z. z

z

23562

1

3

2

1

..

zzz

Z

http://numericalmethods.eng.usf.edu

Example: Inverse of a Matrix

Solving [U][X] = [Z] for [X]

3.22.561

7.00056.18.40

1525

31

21

11

bbb

2.37.056.256.18.4

1525

31

3121

312111

bbbbbb

http://numericalmethods.eng.usf.edu

Example: Inverse of a MatrixUsing Backward Substitution

04762.025

571.49524.05125

51

9524.08.4

571.4560.156.28.4560.156.2

571.47.02.3

312111

3121

31

bbb

bb

b So the first column of the inverse of [A] is:

571.49524.0

04762.0

31

21

11

bbb

http://numericalmethods.eng.usf.edu

Example: Inverse of a MatrixRepeating for the second and third columns of the inverse

Second Column Third Column

010

11214418641525

32

22

12

bbb

000.5417.108333.0

32

22

12

bbb

100

11214418641525

33

23

13

bbb

429.14643.0

03571.0

33

23

13

bbb

http://numericalmethods.eng.usf.edu

Example: Inverse of a MatrixThe inverse of [A] is

429.1000.5571.44643.0417.19524.0

03571.008333.004762.01A

To check your work do the following operation

[A][A]-1 = [I] = [A]-1[A]

Additional ResourcesFor all resources on this topic such as digital audiovisual lectures, primers, textbook chapters, multiple-choice tests, worksheets in MATLAB, MATHEMATICA, MathCad and MAPLE, blogs, related physical problems, please visit http://numericalmethods.eng.usf.edu/topics/lu_decomposition.html

THE END

http://numericalmethods.eng.usf.edu