Chem 302 - Math 252

Chapter 2Solutions of Systems of Linear Equations / Matrix

Inversion

Solutions of Systems of Linear Equations

• n linear equations, n unknowns

• Three possibilities– Unique solution– No solution– Infinite solutions

• Numerically systems that are almost singular cause problems– Range of solutions– Ill-conditioned problem

Singular Systems

Linear Equations 1 (Unique Solution)

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

0 0.5 1 1.5 2 2.5 3 3.5

x

x-y=-1

2x+y=4

x=1, y=2

Linear Equations 2 (No Solution)

0

0.5

1

1.5

2

2.5

0 0.5 1 1.5 2 2.5 3 3.5

x

2x+3y=6

4x+6y=10

Linear Equations 3 (Infinite Solutions)

0

0.5

1

1.5

2

2.5

0 0.5 1 1.5 2 2.5 3 3.5

x

2x+3y=6

4x+6y=12

Linear Equations 4 (Almost Singular)

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

0 0.5 1 1.5 2 2.5

x

7x+10y=175x+7y=12

Solutions of Systems of Linear Equations

• Direct Methods– Determine solution in finite number of steps– Usually preferred– Round-off error can cause problems

• Indirect Methods– Use iteration scheme– Require infinite operations to determine exact solution– Useful when Direct Methods fail

Direct Methods

• Cramer’s Rule

• Gaussian Elimination

• Gauss-Jordan Elimination– Maximum Pivot Strategy

Cramer’s Rule

1. Write coefficient matrix (A)

2. Evaluate |A|– If |A|=0 then singular

3. Form A1

– Replace column 1 of A with answer column

4. Compute x1 = |A1|/|A|

5. Repeat 3 and 4 for other variables

Cramer’s Rule1 2 3

1 2 3

1 2 3

2 7 4 9

9 6 1

3 8 5 6

x x x

x x x

x x x

2 7 4

1 9 6

3 8 5

A

2 7 4

1 9 6 235

3 8 5

A Not singular: System has unique

solution

1

7 4

9 6

8 5

9

1

6

A 1

9 7 4

1 9 6 940

6 8 5

A 1 1 / 940 / 235 4x A A

Cramer’s Rule

2

9

1

6

2 4

1 6

3 5

A 2 2 / 235/ 235 1x A A

3 3 / 470 / 235 2x A A

2

2 9 4

1 1 6 235

3 6 5

A

3

2 7

1 9

3 8

9

1

6

A 3

2 7 9

1 9 1 470

3 8 6

A

Cramer’s Rule

• Good for small systems

• Good if only one or two variables are needed

• Very slow and inefficient for large systems– n order system requires (n+1)! × & (n+1)! Additions

• 2nd order 6 ×, 6 +

• 10th order 3628800 ×, 3628800 +

• 600th order 1.27×101408 ×, 1.27×101408 +

Gaussian Elimination

1. Form augmented matrix

2. Use elementary row operations to transform the augmented matrix so that the A portion is in upper triangular form

• Switch rows• Multiply row by constant• Linear combination of rows

3. Use back substitution to find solutions

• Requires n3+n2- n ×, n3+½n2- n +

Gaussian Elimination1 2 3

1 2 3

1 2 3

2 7 4 9

9 6 1

3 8 5 6

x x x

x x x

x x x

2 7 4 9

| 1 9 6 1

3 8 5 6

A b

2 7 4 9

0 12.5 8 3.5

0 2.5 11 19.5

2 7 4 9

0 12.5 8 3.5

0 0 9.4 18.8

3

2 3

1 3 2

18.8/9.4 2

( 3.5 8 ) /12.5 1

(9 4 7 ) / 2 4

x

x x

x x x

Gauss-Jordan Elimination

1. Form augmented matrix2. Normalize 1st row3. Use elementary row operations to transform the augmented

matrix so that the A portion is the identity matrix• Switch rows• Multiply row by constant• Linear combination of rows

• Requires ½n3+n2- 2½n+2 ×, ½n3-1½n+1 +• Can also be used to find matrix inverse

Gauss-Jordan Elimination1 2 3

1 2 3

1 2 3

2 7 4 9

9 6 1

3 8 5 6

x x x

x x x

x x x

2 7 4 9

| 1 9 6 1

3 8 5 6

A b

7 92 21 2

1 9 6 1

3 8 5 6

1

2

3

4

1

2

x

x

x

7 92 2

25 72 2

5 392 2

1 2

0 8

0 11

7 92 2

16 725 25

5 392 2

1 2

0 1

0 11

6 8825 25

16 725 25

47 945 5

1 0

0 1

0 0

6 8825 25

16 725 25

1 0

0 1

0 0 1 2

1 0 0 4

0 1 0 1

0 0 1 2

Maximum Pivot Strategy

• Elimination methods can run into difficulties if one or more of diagonal elements is close to (or exactly) zero

• Normalize row with largest (magnitude) element.

Gauss-Jordan Elimination1 2 3

1 2 3

1 2 3

9 6 1

2 7 4 9

3 8 5 6

x x x

x x x

x x x

1 9 6 1

| 2 7 4 9

3 8 5 6

A b

1 2 19 3 91

2 7 4 9

3 8 5 6

2

1

3

1

4

2

x

x

x

1 2 19 3 9

25 8829 3 9

35 31 469 3 9

1

0

0

1 2 19 3 9

25 8829 3 9

35 4693 93

1

0

0 1

13 4193 93

235 94093 93

35 4693 93

1 0

0 0

0 1

13 4193 93

35 4693 93

1 0

1 0 0 4

0 1

0 1 0 1

1 0 0 4

0 0 1 2

Comparison of Direct Methods

• Small systems (n<10) not a big deal

• Large systems criticalNumber of floating point operations

n Cramer’s Gaussian Elimination Gauss-Jordan Elimination

2 12 9 7

3 48 28 27

4 240 62 67

5 1440 115 133

10 79833600 805 1063

20 1.0×1020 5910 8323

100 1.9×10160 681550 1009603

1000 4.0×102570 6.7×108 1.0×109

Comparison of Direct MethodsTime required on a 300 MFLOP computer (500 TFLOP)

n Cramer’s Gaussian Elimination Gauss-Jordan Elimination

2 2.4×10-8s 1.8×10-8s 1.4×10-8s

3 9.6×10-8s 5.6×10-8s 5.4×10-8s

4 4.8×10-7s 1.2×10-7s 1.3×10-7s

5 2.9×10-6s 2.3×10-7s 2.7×10-7s

10 0.16s 1.6×10-6s 2.1×10-6s

20 6475 years (2.4 days) 1.2×10-5s 1.7×10-5s

100 1×10144 (1×10138) years 1.4×10-3s 2.0×10-3s

1000 102554 (102548) years 1.3 2

Indirect Methods

• Jacobi Method

• Gauss-Seidel Method

• Use iterations– Guess solution– Iterate to self consistent

• Can be combined with Direct Methods

Jacobi Method

• Rearrange system of equations to isolate the diagonal elements• Guess solution• Iterate until self-consistent

11 1 12 2 1 1

21 1 22 2 2 2

1 1 2 2

n n

n n

n n nn n n

a x a x a x b

a x a x a x b

a x a x a x b

1 1 12 2 13 3 111

2 2 21 1 23 3 122

1 1 2 2 1 1

1

1

1

n n

n n

n n n n nn nnn

x b a x a x a xa

x b a x a x a xa

x b a x a x a xa

Jacobi Method

1 2 3

1 2 3

1 2 3

8 8

7 2 4

2 1 9 12

x x x

x x x

x x x

1 11 2 38 8

12 1 37

13 1 29

1

4 2

12 2

x x x

x x x

x x x

iteration x1 x2 x3

0 0 0 0

1 1 0.571429 1.333333

2 1.095238 1.095238 1.047619

3 0.994048 1.027211 0.968254

4 0.99263 0.990079 0.998299

5 1.001027 0.998461 1.00274

6 1.000535 1.00093 0.999943

7 0.999877 1.00006 0.999778

8 0.999965 0.999919 1.000021

9 1.000013 1.000001 1.000017

10 1.000002 1.000007 0.999997

11 0.999999 0.999999 0.999999

12 1 0.999999 1

13 1 1 1

Gauss-Seidel Method

• Same as Jacobi method, but use updated values as soon as they are calculated.

Jacobi Method

1 11 2 38 8

12 1 37

13 1 29

1

4 2

12 2

x x x

x x x

x x x

iteration x1 x2 x3

0 0 0 0

1 1 0.571429 1.333333

2 1.095238 1.095238 1.047619

3 0.994048 1.027211 0.968254

4 0.99263 0.990079 0.998299

5 1.001027 0.998461 1.00274

6 1.000535 1.00093 0.999943

7 0.999877 1.00006 0.999778

8 0.999965 0.999919 1.000021

9 1.000013 1.000001 1.000017

10 1.000002 1.000007 0.999997

11 0.999999 0.999999 0.999999

12 1 0.999999 1

13 1 1 1

Gauss-Seidel Method

iteration x1 x2 x3

0 0 0 0

1 1 0.714286 1.031746

2 1.039683 1.014739 0.989544

3 0.996851 0.996563 1.001082

4 1.000565 1.00039 0.999831

5 0.99993 0.999942 1.000022

6 1.00001 1.000008 0.999997

7 0.999999 0.999999 1

8 1 1 1

9 1 1 1

Indirect Methods

• Sufficient condition– Diagonally dominant

• Large problems

• Sparse matrix (many zeros)