Lecture 10 - Nonlinear gradient techniques and LU Decomposition

Lecture 10 - Nonlinear gradient Lecture 10 - Nonlinear gradient techniques and LU Decompositiontechniques and LU Decomposition

CVEN 302

June 24, 2002

Lecture’s GoalsLecture’s Goals

• Nonlinear Gradient technique

• LU Decomposition– Crout’s technique – Doolittle’s technique– Cholesky’s technique

Nonlinear EquationsNonlinear Equations

• The nonlinear equations can be solved using a gradient technique.

• The minimization technique calculates a positive scalar value and use a gradient to find the zero of multiple functions.

Minimization algorithmMinimization algorithm

• Calculate the square function.– h(x) = [f(x))2]

• Calculate a scalar value – z0 = h(x)

• Calculate the gradient– dx = - dh/dx

Minimization algorithmMinimization algorithm

• Multiple loops to convergence

xnew = xold + dx ; z1 = h(xnew ); dif = z1 - z0;

if dif > 0dx = dx/2

xnew = xold + dx

elseend loop

endif

Program FFMINProgram FFMIN

• The program is adapted from the book to do a minimization of scalar and uses a gradient technique to find the roots.

Example of the 2-D ProblemExample of the 2-D Problem

f1(x,y) = x2 + y2 - 1

f2(x,y) = x2 - y

Example of the 2-D ProblemExample of the 2-D Problem

The gradient function:

h(x,y) =[( x2 + y2 - 1)2 +( x2 - y) 2]

The derivative of the function:

dh=[-(4(x2 + y2-1)x + 4( x2 - y)x)

-(4(x2 +y2 - 1)y - 2( x2 - y))]

Example of the 3-D ProblemExample of the 3-D Problem

f1(x,y,z) = x2 + 2y2 + 4z2 - 7

f2(x,y,z) = 2x2 + y3 + 6z2 - 10

f3(x,y,z) = xyz + 1

Example of the 3-D ProblemExample of the 3-D Problem

The gradient function:

h(x,y,z) = [ (x2 + 2y2 + 4z2 - 7)2 +

(2x2 + y3 + 6z2 - 10)2 +

(xyz + 1) 2]

End of material on Exam 1

Exam 1

Chapter 1 through 5

Monday July 3, 2002

open book and open notes

Chapter 6Chapter 6

LU Decomposition of Matrices

LU DecompositionLU Decomposition

A modification of the elimination method, called the LU decomposition. The technique will rewrite the matrix as the product of two matrices.

A = LU

LU DecompositionLU Decomposition

The technique breaks the matrix into a product of two matrices, L and U, L is a lower triangular matrix and U is an upper triangular matrix.

LU DecompositionLU Decomposition

– Crout’s reduction (U has ones on the diagonal)– Doolittle’s method( L has ones on the diagonal)– Cholesky’s method ( The diagonal terms are the

same value for the L and U matrices)

There are variation of the technique using different methods.

DecompositionDecomposition2 1 1 2 0 0 1 0.5 0.5

0 4 2 0 4 0 0 1 0.5

6 3 0 6 0 3 0 0 1

1 0 0 2 1 1

0 1 0 0 4 2

3 0 1 0 0 3

1 0 0 2 1 1

0 2 0 0 2 1

3 0 1 0 0 3

LU Decomposition SolvingLU Decomposition Solving

Using the LU decomposition

[A]{x} = [L][U]{x} = [L]{[U]{x}} = {b}

Solve

[L]{y} = {b}

and then solve

[U]{x} = {y}

LU DecompositionLU Decomposition

The matrices are represented by

11 12 13 14 11 11 12 13 14

21 22 23 24 21 22 22 23 24

31 32 33 34 31 32 33 33 34

41 42 43 44 41 42 43 44 44

a a a a 0 0 0

a a a a 0 0 0

a a a a 0 0 0

a a a a 0 0 0

Equation SolvingEquation Solving

What is the advantage of breaking up one linear set into two successive ones?

– The advantage is that the solution of triangular set of equations is trivial to solve.

Equation SolvingEquation Solving

• First step - forward substitution

N,2,i ,yb1

y

by

1i

1jjiji

iii

11

11

Equation SolvingEquation Solving

• Second step - back substitution

,11,N i ,xy1

x

yx

N

1ijjiji

iii

NN

NN

LU Decomposition (Crout’s reduction)LU Decomposition (Crout’s reduction)

Matrix decomposition

11 12 13 14 11 12 13 14

21 22 23 24 21 22 23 24

31 32 33 34 31 32 33 34

41 42 43 44 41 42 43 44

a a a a 0 0 0 1

a a a a 0 0 0 1

a a a a 0 0 0 1

a a a a 0 0 0 1

l u u u

l l u u

l l l u

l l l l

LU Decomposition (Doolittle’s method)LU Decomposition (Doolittle’s method)

Matrix decomposition

11 12 13 14 11 12 13 14

21 22 23 24 21 22 23 24

31 32 33 34 31 32 33 34

41 42 43 44 41 42 43 44

a a a a 1 0 0 0

a a a a 1 0 0 0

a a a a 1 0 0 0

a a a a 1 0 0 0

u u u u

l u u u

l l u u

l l l u

Cholesky’s methodCholesky’s method

Matrix is decomposed into:

where, lii = uii

11 12 13 14 11 11 12 13 14

21 22 23 24 21 22 22 23 24

31 32 33 34 31 32 33 33 34

41 42 43 44 41 42 43 44 44

a a a a 0 0 0

a a a a 0 0 0

a a a a 0 0 0

a a a a 0 0 0

l u u u u

l l u u u

l l l u u

l l l l u

LU Decomposition (Crout’s reduction)LU Decomposition (Crout’s reduction)

Matrix decomposition

11 12 13 14 11 12 13 14

21 22 23 24 21 22 23 24

31 32 33 34 31 32 33 34

41 42 43 44 41 42 43 44

a a a a 0 0 0 1

a a a a 0 0 0 1

a a a a 0 0 0 1

a a a a 0 0 0 1

l u u u

l l u u

l l l u

l l l l

Crout’s ReductionCrout’s Reduction

11 11

21 21

31 31

41 41

a

a

a

a

l

l

l

l

The method alternates from solving from the lower triangular to the upper triangular

1211 12 12 12

11

1311 13 13 13

11

1411 14 14 14

11

aa

aa

aa

l u ul

l u ul

l u ul

Crout’s ReductionCrout’s Reduction

21 12 22 22 22 22 21 12

31 12 32 32 32 32 31 12

41 12 42 42 42 42 41 12

1 a a

1 a a

1 a a

l u l l l u

l u l l l u

l u l l l u

23 21 1321 13 22 23 23 23

22

24 21 1421 14 22 24 24 24

22

aa

aa

l ul u l u u

l

l ul u l u u

l

General formulation of Crout’sGeneral formulation of Crout’s

n,2,j j,i l

ulau

N,1,i i,j ulal

ii

1i

1kkjikij

ij

1j

1kkjikijij

These are the general equations for the component of the two matrices

ExampleExample

100

110

3/23/11

13/42

03/71

003

122

321

213

The matrix is broken into a lower and upper triangular matrices.

LU Decomposition (Doolittle’s method)LU Decomposition (Doolittle’s method)

Matrix decomposition

11 12 13 14 11 12 13 14

21 22 23 24 21 22 23 24

31 32 33 34 31 32 33 34

41 42 43 44 41 42 43 44

a a a a 1 0 0 0

a a a a 1 0 0 0

a a a a 1 0 0 0

a a a a 1 0 0 0

u u u u

l u u u

l l u u

l l l u

Doolitte’s methodDoolitte’s method

1414

1313

1212

1111

au

au

au

au

11

4141411141

11

3131311131

11

2121211121

u

alaul

u

alaul

u

alaul

The method alternates from solving from the upper triangular to the lower triangular

General formulation of Doolittle’sGeneral formulation of Doolittle’s

n,2,i i,j u

ulal

N,1,j j,i ulau

ii

1j

1kkjikij

ij

1i

1kkjikijij

The problem is reverse of the Crout’s reduction, starting with the upper triangular matrix and going to the lower triangular matrix.

LU ProgramsLU Programs

• There are two programs– LU_factorLU_factor - the program does a Doolittle

decomposition of a matrix and returns the L and U matrices

– LU_solveLU_solve uses an L and U matrix combination to solve the system of equations.

ExampleExample

• The matrix is broken into a lower and upper triangular matrices.

3 1 2 1 0 0 3 1 2

1 2 3 1 3 1 0 0 7 3 7 3

2 2 1 2 3 4 7 1 0 0 1

SummarySummary

• Nonlinear scalar gradient method uses a simple step to find the crossing terms.

• Setup of the LU decomposition techniques.

HomeworkHomework

• Check the Homework webpage