57
Iterative Techniques in Matrix Algebra Jacobi & Gauss-Seidel Iterative Techniques I Numerical Analysis (9th Edition) R L Burden & J D Faires Beamer Presentation Slides prepared by John Carroll Dublin City University c 2011 Brooks/Cole, Cengage Learning

Gauss & Jordan Iterative Techniques

  • Upload
    vrsafe

  • View
    28

  • Download
    0

Embed Size (px)

DESCRIPTION

Iterative Techniques - Matrix Algebra

Citation preview

Page 1: Gauss & Jordan Iterative Techniques

Iterative Techniques in Matrix Algebra

Jacobi & Gauss-Seidel Iterative Techniques I

Numerical Analysis (9th Edition)R L Burden & J D Faires

Beamer Presentation Slidesprepared byJohn Carroll

Dublin City University

c© 2011 Brooks/Cole, Cengage Learning

Page 2: Gauss & Jordan Iterative Techniques

Introduction Jacobi’s Method Equivalent System Jacobi Algorithm

Outline

1 Introducing Iterative Techniques for Linear Systems

2 The Jacobi Iterative Method

3 Converting Ax = b into an Equivalent System

4 The Jacobi Iterative Algorithm

Numerical Analysis (Chapter 7) Jacobi & Gauss-Seidel Methods I R L Burden & J D Faires 2 / 26

Page 3: Gauss & Jordan Iterative Techniques

Introduction Jacobi’s Method Equivalent System Jacobi Algorithm

Outline

1 Introducing Iterative Techniques for Linear Systems

2 The Jacobi Iterative Method

3 Converting Ax = b into an Equivalent System

4 The Jacobi Iterative Algorithm

Numerical Analysis (Chapter 7) Jacobi & Gauss-Seidel Methods I R L Burden & J D Faires 2 / 26

Page 4: Gauss & Jordan Iterative Techniques

Introduction Jacobi’s Method Equivalent System Jacobi Algorithm

Outline

1 Introducing Iterative Techniques for Linear Systems

2 The Jacobi Iterative Method

3 Converting Ax = b into an Equivalent System

4 The Jacobi Iterative Algorithm

Numerical Analysis (Chapter 7) Jacobi & Gauss-Seidel Methods I R L Burden & J D Faires 2 / 26

Page 5: Gauss & Jordan Iterative Techniques

Introduction Jacobi’s Method Equivalent System Jacobi Algorithm

Outline

1 Introducing Iterative Techniques for Linear Systems

2 The Jacobi Iterative Method

3 Converting Ax = b into an Equivalent System

4 The Jacobi Iterative Algorithm

Numerical Analysis (Chapter 7) Jacobi & Gauss-Seidel Methods I R L Burden & J D Faires 2 / 26

Page 6: Gauss & Jordan Iterative Techniques

Introduction Jacobi’s Method Equivalent System Jacobi Algorithm

Outline

1 Introducing Iterative Techniques for Linear Systems

2 The Jacobi Iterative Method

3 Converting Ax = b into an Equivalent System

4 The Jacobi Iterative Algorithm

Numerical Analysis (Chapter 7) Jacobi & Gauss-Seidel Methods I R L Burden & J D Faires 3 / 26

Page 7: Gauss & Jordan Iterative Techniques

Introduction Jacobi’s Method Equivalent System Jacobi Algorithm

The Jacobi & Gauss-Seidel Methods

IntyroductionWe will now describe the Jacobi and the Gauss-Seidel iterativemethods, classic methods that date to the late eighteenth century.

Iterative techniques are seldom used for solving linear systems ofsmall dimension since the time required for sufficient accuracyexceeds that required for direct techniques such as Gaussianelimination.For large systems with a high percentage of 0 entries, however,these techniques are efficient in terms of both computer storageand computation.

Numerical Analysis (Chapter 7) Jacobi & Gauss-Seidel Methods I R L Burden & J D Faires 4 / 26

Page 8: Gauss & Jordan Iterative Techniques

Introduction Jacobi’s Method Equivalent System Jacobi Algorithm

The Jacobi & Gauss-Seidel Methods

IntyroductionWe will now describe the Jacobi and the Gauss-Seidel iterativemethods, classic methods that date to the late eighteenth century.Iterative techniques are seldom used for solving linear systems ofsmall dimension since the time required for sufficient accuracyexceeds that required for direct techniques such as Gaussianelimination.

For large systems with a high percentage of 0 entries, however,these techniques are efficient in terms of both computer storageand computation.

Numerical Analysis (Chapter 7) Jacobi & Gauss-Seidel Methods I R L Burden & J D Faires 4 / 26

Page 9: Gauss & Jordan Iterative Techniques

Introduction Jacobi’s Method Equivalent System Jacobi Algorithm

The Jacobi & Gauss-Seidel Methods

IntyroductionWe will now describe the Jacobi and the Gauss-Seidel iterativemethods, classic methods that date to the late eighteenth century.Iterative techniques are seldom used for solving linear systems ofsmall dimension since the time required for sufficient accuracyexceeds that required for direct techniques such as Gaussianelimination.For large systems with a high percentage of 0 entries, however,these techniques are efficient in terms of both computer storageand computation.

Numerical Analysis (Chapter 7) Jacobi & Gauss-Seidel Methods I R L Burden & J D Faires 4 / 26

Page 10: Gauss & Jordan Iterative Techniques

Introduction Jacobi’s Method Equivalent System Jacobi Algorithm

The Jacobi & Gauss-Seidel Methods

Iterative TechniqueAn iterative technique to solve the n × n linear system

Ax = b

starts with an initial approximation

x(0)

to the solution x and generates a sequence of vectors

{x(k)}∞k=0

that converges to x.

Numerical Analysis (Chapter 7) Jacobi & Gauss-Seidel Methods I R L Burden & J D Faires 5 / 26

Page 11: Gauss & Jordan Iterative Techniques

Introduction Jacobi’s Method Equivalent System Jacobi Algorithm

The Jacobi & Gauss-Seidel Methods

Iterative TechniqueAn iterative technique to solve the n × n linear system

Ax = b

starts with an initial approximation

x(0)

to the solution x

and generates a sequence of vectors

{x(k)}∞k=0

that converges to x.

Numerical Analysis (Chapter 7) Jacobi & Gauss-Seidel Methods I R L Burden & J D Faires 5 / 26

Page 12: Gauss & Jordan Iterative Techniques

Introduction Jacobi’s Method Equivalent System Jacobi Algorithm

The Jacobi & Gauss-Seidel Methods

Iterative TechniqueAn iterative technique to solve the n × n linear system

Ax = b

starts with an initial approximation

x(0)

to the solution x and generates a sequence of vectors

{x(k)}∞k=0

that converges to x.

Numerical Analysis (Chapter 7) Jacobi & Gauss-Seidel Methods I R L Burden & J D Faires 5 / 26

Page 13: Gauss & Jordan Iterative Techniques

Introduction Jacobi’s Method Equivalent System Jacobi Algorithm

Outline

1 Introducing Iterative Techniques for Linear Systems

2 The Jacobi Iterative Method

3 Converting Ax = b into an Equivalent System

4 The Jacobi Iterative Algorithm

Numerical Analysis (Chapter 7) Jacobi & Gauss-Seidel Methods I R L Burden & J D Faires 6 / 26

Page 14: Gauss & Jordan Iterative Techniques

Introduction Jacobi’s Method Equivalent System Jacobi Algorithm

Jacobi’s Method

The Jacobi iterative method is obtained by solving the i th equation inAx = b for xi to obtain (provided aii 6= 0)

xi =n∑

j=1j 6=i

(−

aijxj

aii

)+

bi

aii, for i = 1, 2, . . . , n

For each k ≥ 1, generate the components x (k)i of x(k) from the

components of x(k−1) by

x (k)i =

1aii

n∑j=1j 6=i

(−aijx

(k−1)j

)+ bi

, for i = 1, 2, . . . , n

Numerical Analysis (Chapter 7) Jacobi & Gauss-Seidel Methods I R L Burden & J D Faires 7 / 26

Page 15: Gauss & Jordan Iterative Techniques

Introduction Jacobi’s Method Equivalent System Jacobi Algorithm

Jacobi’s Method

The Jacobi iterative method is obtained by solving the i th equation inAx = b for xi to obtain (provided aii 6= 0)

xi =n∑

j=1j 6=i

(−

aijxj

aii

)+

bi

aii, for i = 1, 2, . . . , n

For each k ≥ 1, generate the components x (k)i of x(k) from the

components of x(k−1) by

x (k)i =

1aii

n∑j=1j 6=i

(−aijx

(k−1)j

)+ bi

, for i = 1, 2, . . . , n

Numerical Analysis (Chapter 7) Jacobi & Gauss-Seidel Methods I R L Burden & J D Faires 7 / 26

Page 16: Gauss & Jordan Iterative Techniques

Introduction Jacobi’s Method Equivalent System Jacobi Algorithm

Jacobi’s Method

ExampleThe linear system Ax = b given by

E1 : 10x1 − x2 + 2x3 = 6E2 : −x1 + 11x2 − x3 + 3x4 = 25E3 : 2x1 − x2 + 10x3 − x4 = −11,

E4 : 3x2 − x3 + 8x4 = 15

has the unique solution x = (1, 2,−1, 1)t .

Use Jacobi’s iterativetechnique to find approximations x(k) to x starting withx(0) = (0, 0, 0, 0)t until

‖x(k) − x(k−1)‖∞‖x(k)‖∞

< 10−3

Numerical Analysis (Chapter 7) Jacobi & Gauss-Seidel Methods I R L Burden & J D Faires 8 / 26

Page 17: Gauss & Jordan Iterative Techniques

Introduction Jacobi’s Method Equivalent System Jacobi Algorithm

Jacobi’s Method

ExampleThe linear system Ax = b given by

E1 : 10x1 − x2 + 2x3 = 6E2 : −x1 + 11x2 − x3 + 3x4 = 25E3 : 2x1 − x2 + 10x3 − x4 = −11,

E4 : 3x2 − x3 + 8x4 = 15

has the unique solution x = (1, 2,−1, 1)t . Use Jacobi’s iterativetechnique to find approximations x(k) to x starting withx(0) = (0, 0, 0, 0)t until

‖x(k) − x(k−1)‖∞‖x(k)‖∞

< 10−3

Numerical Analysis (Chapter 7) Jacobi & Gauss-Seidel Methods I R L Burden & J D Faires 8 / 26

Page 18: Gauss & Jordan Iterative Techniques

Introduction Jacobi’s Method Equivalent System Jacobi Algorithm

Jacobi’s Method: Example

Solution (1/4)We first solve equation Ei for xi , for each i = 1, 2, 3, 4, to obtain

x1 =1

10x2 −

15

x3 +35

x2 =1

11x1 +

111

x3 −3

11x4 +

2511

x3 = −15

x1 +1

10x2 +

110

x4 −1110

x4 = − 38

x2 +18

x3 +158

Numerical Analysis (Chapter 7) Jacobi & Gauss-Seidel Methods I R L Burden & J D Faires 9 / 26

Page 19: Gauss & Jordan Iterative Techniques

Introduction Jacobi’s Method Equivalent System Jacobi Algorithm

Jacobi’s Method: Example

Solution (2/4)

From the initial approximation x(0) = (0, 0, 0, 0)t we have x(1) given by

x (1)1 =

110

x (0)2 − 1

5x (0)

3 +35

= 0.6000

x (1)2 =

111

x (0)1 +

111

x (0)3 − 3

11x (0)

4 +2511

= 2.2727

x (1)3 = −1

5x (0)

1 +1

10x (0)

2 +1

10x (0)

4 − 1110

= −1.1000

x (1)4 = − 3

8x (0)

2 +18

x (0)3 +

158

= 1.8750

Numerical Analysis (Chapter 7) Jacobi & Gauss-Seidel Methods I R L Burden & J D Faires 10 / 26

Page 20: Gauss & Jordan Iterative Techniques

Introduction Jacobi’s Method Equivalent System Jacobi Algorithm

Jacobi’s Method: Example

Solution (3/4)

Additional iterates, x(k) = (x (k)1 , x (k)

2 , x (k)3 , x (k)

4 )t , are generated in asimilar manner and are summarized as follows:

k 0 1 2 3 4 10

x (k)1 0.0 0.6000 1.0473 0.9326 1.0152 · · · 1.0001

x (k)2 0.0 2.2727 1.7159 2.053 1.9537 · · · 1.9998

x (k)3 0.0 −1.1000 −0.8052 −1.0493 −0.9681 · · · −0.9998

x (k)4 0.0 1.8750 0.8852 1.1309 0.9739 · · · 0.9998

Numerical Analysis (Chapter 7) Jacobi & Gauss-Seidel Methods I R L Burden & J D Faires 11 / 26

Page 21: Gauss & Jordan Iterative Techniques

Introduction Jacobi’s Method Equivalent System Jacobi Algorithm

Jacobi’s Method: Example

Solution (4/4)The process was stopped after 10 iterations because

‖x(10) − x(9)‖∞‖x(10)‖∞

=8.0× 10−4

1.9998< 10−3

In fact, ‖x(10) − x‖∞ = 0.0002.

Numerical Analysis (Chapter 7) Jacobi & Gauss-Seidel Methods I R L Burden & J D Faires 12 / 26

Page 22: Gauss & Jordan Iterative Techniques

Introduction Jacobi’s Method Equivalent System Jacobi Algorithm

Outline

1 Introducing Iterative Techniques for Linear Systems

2 The Jacobi Iterative Method

3 Converting Ax = b into an Equivalent System

4 The Jacobi Iterative Algorithm

Numerical Analysis (Chapter 7) Jacobi & Gauss-Seidel Methods I R L Burden & J D Faires 13 / 26

Page 23: Gauss & Jordan Iterative Techniques

Introduction Jacobi’s Method Equivalent System Jacobi Algorithm

Jacobi’s Method in the form x(k) = T x(k−1) + c

A More General RepresentationIn general, iterative techniques for solving linear systems involve aprocess that converts the system Ax = b into an equivalentsystem of the form

x = T x + c

for some fixed matrix T and vector c.

After the initial vector x(0) is selected, the sequence ofapproximate solution vectors is generated by computing

x(k) = T x(k−1) + c

for each k = 1, 2, 3, . . . (reminiscent of the fixed-point iteration forsolving nonlinear equations).

Numerical Analysis (Chapter 7) Jacobi & Gauss-Seidel Methods I R L Burden & J D Faires 14 / 26

Page 24: Gauss & Jordan Iterative Techniques

Introduction Jacobi’s Method Equivalent System Jacobi Algorithm

Jacobi’s Method in the form x(k) = T x(k−1) + c

A More General RepresentationIn general, iterative techniques for solving linear systems involve aprocess that converts the system Ax = b into an equivalentsystem of the form

x = T x + c

for some fixed matrix T and vector c.After the initial vector x(0) is selected, the sequence ofapproximate solution vectors is generated by computing

x(k) = T x(k−1) + c

for each k = 1, 2, 3, . . . (reminiscent of the fixed-point iteration forsolving nonlinear equations).

Numerical Analysis (Chapter 7) Jacobi & Gauss-Seidel Methods I R L Burden & J D Faires 14 / 26

Page 25: Gauss & Jordan Iterative Techniques

Introduction Jacobi’s Method Equivalent System Jacobi Algorithm

Jacobi’s Method in the form x(k) = T x(k−1) + c

A More General Representation (Cont’d)The Jacobi method can be written in the form

x(k) = T x(k−1) + c

by splitting A into its diagonal and off-diagonal parts.

To see this, let D be the diagonal matrix whose diagonal entriesare those of A, −L be the strictly lower-triangular part of A, and−U be the strictly upper-triangular part of A where

A =

a11 a12 · · · a1na21 a22 · · · a2n

......

...an1 an2 · · · ann

Numerical Analysis (Chapter 7) Jacobi & Gauss-Seidel Methods I R L Burden & J D Faires 15 / 26

Page 26: Gauss & Jordan Iterative Techniques

Introduction Jacobi’s Method Equivalent System Jacobi Algorithm

Jacobi’s Method in the form x(k) = T x(k−1) + c

A More General Representation (Cont’d)The Jacobi method can be written in the form

x(k) = T x(k−1) + c

by splitting A into its diagonal and off-diagonal parts.To see this, let D be the diagonal matrix whose diagonal entriesare those of A, −L be the strictly lower-triangular part of A, and−U be the strictly upper-triangular part of A where

A =

a11 a12 · · · a1na21 a22 · · · a2n

......

...an1 an2 · · · ann

Numerical Analysis (Chapter 7) Jacobi & Gauss-Seidel Methods I R L Burden & J D Faires 15 / 26

Page 27: Gauss & Jordan Iterative Techniques

Introduction Jacobi’s Method Equivalent System Jacobi Algorithm

Jacobi’s Method in the form x(k) = T x(k−1) + c

A More General Representation (Cont’d)We then write A = D − L− U

where

D =

a11 0 · · · 0

0 a22. . .

......

. . . . . . 00 · · · 0 ann

L =

0 · · · · · · 0

−a21. . .

......

. . . . . ....

−an1 · · · −an,n−1 0

and

U =

0 −a12 · · · −a1n...

. . . . . ....

.... . . −an−1,n

0 · · · · · · 0

Numerical Analysis (Chapter 7) Jacobi & Gauss-Seidel Methods I R L Burden & J D Faires 16 / 26

Page 28: Gauss & Jordan Iterative Techniques

Introduction Jacobi’s Method Equivalent System Jacobi Algorithm

Jacobi’s Method in the form x(k) = T x(k−1) + c

A More General Representation (Cont’d)We then write A = D − L− U where

D =

a11 0 · · · 0

0 a22. . .

......

. . . . . . 00 · · · 0 ann

L =

0 · · · · · · 0

−a21. . .

......

. . . . . ....

−an1 · · · −an,n−1 0

and

U =

0 −a12 · · · −a1n...

. . . . . ....

.... . . −an−1,n

0 · · · · · · 0

Numerical Analysis (Chapter 7) Jacobi & Gauss-Seidel Methods I R L Burden & J D Faires 16 / 26

Page 29: Gauss & Jordan Iterative Techniques

Introduction Jacobi’s Method Equivalent System Jacobi Algorithm

Jacobi’s Method in the form x(k) = T x(k−1) + c

A More General Representation (Cont’d)We then write A = D − L− U where

D =

a11 0 · · · 0

0 a22. . .

......

. . . . . . 00 · · · 0 ann

L =

0 · · · · · · 0

−a21. . .

......

. . . . . ....

−an1 · · · −an,n−1 0

and

U =

0 −a12 · · · −a1n...

. . . . . ....

.... . . −an−1,n

0 · · · · · · 0

Numerical Analysis (Chapter 7) Jacobi & Gauss-Seidel Methods I R L Burden & J D Faires 16 / 26

Page 30: Gauss & Jordan Iterative Techniques

Introduction Jacobi’s Method Equivalent System Jacobi Algorithm

Jacobi’s Method in the form x(k) = T x(k−1) + c

A More General Representation (Cont’d)We then write A = D − L− U where

D =

a11 0 · · · 0

0 a22. . .

......

. . . . . . 00 · · · 0 ann

L =

0 · · · · · · 0

−a21. . .

......

. . . . . ....

−an1 · · · −an,n−1 0

and

U =

0 −a12 · · · −a1n...

. . . . . ....

.... . . −an−1,n

0 · · · · · · 0

Numerical Analysis (Chapter 7) Jacobi & Gauss-Seidel Methods I R L Burden & J D Faires 16 / 26

Page 31: Gauss & Jordan Iterative Techniques

Introduction Jacobi’s Method Equivalent System Jacobi Algorithm

Jacobi’s Method in the form x(k) = T x(k−1) + c

A More General Representation (Cont’d)The equation Ax = b, or (D − L− U)x = b, is then transformed into

Dx = (L + U)x + b

and, if D−1 exists, that is, if aii 6= 0 for each i , then

x = D−1(L + U)x + D−1b

This results in the matrix form of the Jacobi iterative technique:

x(k) = D−1(L + U)x(k−1) + D−1b, k = 1, 2, . . .

Numerical Analysis (Chapter 7) Jacobi & Gauss-Seidel Methods I R L Burden & J D Faires 17 / 26

Page 32: Gauss & Jordan Iterative Techniques

Introduction Jacobi’s Method Equivalent System Jacobi Algorithm

Jacobi’s Method in the form x(k) = T x(k−1) + c

A More General Representation (Cont’d)The equation Ax = b, or (D − L− U)x = b, is then transformed into

Dx = (L + U)x + b

and, if D−1 exists, that is, if aii 6= 0 for each i , then

x = D−1(L + U)x + D−1b

This results in the matrix form of the Jacobi iterative technique:

x(k) = D−1(L + U)x(k−1) + D−1b, k = 1, 2, . . .

Numerical Analysis (Chapter 7) Jacobi & Gauss-Seidel Methods I R L Burden & J D Faires 17 / 26

Page 33: Gauss & Jordan Iterative Techniques

Introduction Jacobi’s Method Equivalent System Jacobi Algorithm

Jacobi’s Method in the form x(k) = T x(k−1) + c

A More General Representation (Cont’d)The equation Ax = b, or (D − L− U)x = b, is then transformed into

Dx = (L + U)x + b

and, if D−1 exists, that is, if aii 6= 0 for each i , then

x = D−1(L + U)x + D−1b

This results in the matrix form of the Jacobi iterative technique:

x(k) = D−1(L + U)x(k−1) + D−1b, k = 1, 2, . . .

Numerical Analysis (Chapter 7) Jacobi & Gauss-Seidel Methods I R L Burden & J D Faires 17 / 26

Page 34: Gauss & Jordan Iterative Techniques

Introduction Jacobi’s Method Equivalent System Jacobi Algorithm

Jacobi’s Method in the form x(k) = T x(k−1) + c

A More General Representation (Cont’d)

Introducing the notation Tj = D−1(L + U) and cj = D−1b

gives theJacobi technique the form

x(k) = Tjx(k−1) + cj

In practice, this form is only used for theoretical purposes while

x (k)i =

1aii

n∑j=1j 6=i

(−aijx

(k−1)j

)+ bi

, for i = 1, 2, . . . , n

is used in computation.

Numerical Analysis (Chapter 7) Jacobi & Gauss-Seidel Methods I R L Burden & J D Faires 18 / 26

Page 35: Gauss & Jordan Iterative Techniques

Introduction Jacobi’s Method Equivalent System Jacobi Algorithm

Jacobi’s Method in the form x(k) = T x(k−1) + c

A More General Representation (Cont’d)

Introducing the notation Tj = D−1(L + U) and cj = D−1b gives theJacobi technique the form

x(k) = Tjx(k−1) + cj

In practice, this form is only used for theoretical purposes while

x (k)i =

1aii

n∑j=1j 6=i

(−aijx

(k−1)j

)+ bi

, for i = 1, 2, . . . , n

is used in computation.

Numerical Analysis (Chapter 7) Jacobi & Gauss-Seidel Methods I R L Burden & J D Faires 18 / 26

Page 36: Gauss & Jordan Iterative Techniques

Introduction Jacobi’s Method Equivalent System Jacobi Algorithm

Jacobi’s Method in the form x(k) = T x(k−1) + c

A More General Representation (Cont’d)

Introducing the notation Tj = D−1(L + U) and cj = D−1b gives theJacobi technique the form

x(k) = Tjx(k−1) + cj

In practice, this form is only used for theoretical purposes while

x (k)i =

1aii

n∑j=1j 6=i

(−aijx

(k−1)j

)+ bi

, for i = 1, 2, . . . , n

is used in computation.

Numerical Analysis (Chapter 7) Jacobi & Gauss-Seidel Methods I R L Burden & J D Faires 18 / 26

Page 37: Gauss & Jordan Iterative Techniques

Introduction Jacobi’s Method Equivalent System Jacobi Algorithm

Jacobi’s Method in the form x(k) = T x(k−1) + c

ExampleExpress the Jacobi iteration method for the linear system Ax = b givenby

E1 : 10x1 − x2 + 2x3 = 6E2 : −x1 + 11x2 − x3 + 3x4 = 25E3 : 2x1 − x2 + 10x3 − x4 = −11E4 : 3x2 − x3 + 8x4 = 15

in the form x(k) = T x(k−1) + c.

Numerical Analysis (Chapter 7) Jacobi & Gauss-Seidel Methods I R L Burden & J D Faires 19 / 26

Page 38: Gauss & Jordan Iterative Techniques

Introduction Jacobi’s Method Equivalent System Jacobi Algorithm

Jacobi’s Method in the form x(k) = T x(k−1) + c

Solution (1/2)We saw earlier that the Jacobi method for this system has the form

x1 =1

10x2 −

15

x3 +35

x2 =1

11x1 +

111

x3 −3

11x4 +

2511

x3 = −15

x1 +1

10x2 +

110

x4 −1110

x4 = − 38

x2 +18

x3 +158

Numerical Analysis (Chapter 7) Jacobi & Gauss-Seidel Methods I R L Burden & J D Faires 20 / 26

Page 39: Gauss & Jordan Iterative Techniques

Introduction Jacobi’s Method Equivalent System Jacobi Algorithm

Jacobi’s Method in the form x(k) = T x(k−1) + c

Solution (2/2)Hence, we have

T =

0 1

10 −15 0

111 0 1

11 − 311

−15

110 0 1

10

0 −38

18 0

and c =

35

2511

−1110

158

Numerical Analysis (Chapter 7) Jacobi & Gauss-Seidel Methods I R L Burden & J D Faires 21 / 26

Page 40: Gauss & Jordan Iterative Techniques

Introduction Jacobi’s Method Equivalent System Jacobi Algorithm

Outline

1 Introducing Iterative Techniques for Linear Systems

2 The Jacobi Iterative Method

3 Converting Ax = b into an Equivalent System

4 The Jacobi Iterative Algorithm

Numerical Analysis (Chapter 7) Jacobi & Gauss-Seidel Methods I R L Burden & J D Faires 22 / 26

Page 41: Gauss & Jordan Iterative Techniques

Introduction Jacobi’s Method Equivalent System Jacobi Algorithm

The Jacobi Iterative Algorithm (1/2)

To solve Ax = b given an initial approximation x(0):

INPUT the number of equations and unknowns n;the entries aij , 1 ≤ i , j ≤ n of the matrix A;the entries bi , 1 ≤ i ≤ n of b;the entries XOi , 1 ≤ i ≤ n of XO = x(0);tolerance TOL;maximum number of iterations N.

OUTPUT the approximate solution x1, . . . , xn or a message thatthe number of iterations was exceeded.

Numerical Analysis (Chapter 7) Jacobi & Gauss-Seidel Methods I R L Burden & J D Faires 23 / 26

Page 42: Gauss & Jordan Iterative Techniques

Introduction Jacobi’s Method Equivalent System Jacobi Algorithm

The Jacobi Iterative Algorithm (1/2)

To solve Ax = b given an initial approximation x(0):

INPUT the number of equations and unknowns n;the entries aij , 1 ≤ i , j ≤ n of the matrix A;the entries bi , 1 ≤ i ≤ n of b;the entries XOi , 1 ≤ i ≤ n of XO = x(0);tolerance TOL;maximum number of iterations N.

OUTPUT the approximate solution x1, . . . , xn or a message thatthe number of iterations was exceeded.

Numerical Analysis (Chapter 7) Jacobi & Gauss-Seidel Methods I R L Burden & J D Faires 23 / 26

Page 43: Gauss & Jordan Iterative Techniques

Introduction Jacobi’s Method Equivalent System Jacobi Algorithm

The Jacobi Iterative Algorithm (1/2)

To solve Ax = b given an initial approximation x(0):

INPUT the number of equations and unknowns n;the entries aij , 1 ≤ i , j ≤ n of the matrix A;the entries bi , 1 ≤ i ≤ n of b;the entries XOi , 1 ≤ i ≤ n of XO = x(0);tolerance TOL;maximum number of iterations N.

OUTPUT the approximate solution x1, . . . , xn or a message thatthe number of iterations was exceeded.

Numerical Analysis (Chapter 7) Jacobi & Gauss-Seidel Methods I R L Burden & J D Faires 23 / 26

Page 44: Gauss & Jordan Iterative Techniques

Introduction Jacobi’s Method Equivalent System Jacobi Algorithm

The Jacobi Iterative Algorithm (2/2)

Step 1 Set k = 1Step 2 While (k ≤ N) do Steps 3–6

Step 3 For i = 1, . . . , n

set xi =1aii

[−

∑nj=1j 6=i

(aijXOj) + bi

]Step 4 If ||x− XO|| < TOL then OUTPUT (x1, . . . , xn)

(The procedure was successful)STOP

Step 5 Set k = k + 1Step 6 For i = 1, . . . , n set XOi = xi

Step 7 OUTPUT (‘Maximum number of iterations exceeded’)(The procedure was successful)STOP

Numerical Analysis (Chapter 7) Jacobi & Gauss-Seidel Methods I R L Burden & J D Faires 24 / 26

Page 45: Gauss & Jordan Iterative Techniques

Introduction Jacobi’s Method Equivalent System Jacobi Algorithm

The Jacobi Iterative Algorithm (2/2)

Step 1 Set k = 1Step 2 While (k ≤ N) do Steps 3–6

Step 3 For i = 1, . . . , n

set xi =1aii

[−

∑nj=1j 6=i

(aijXOj) + bi

]Step 4 If ||x− XO|| < TOL then OUTPUT (x1, . . . , xn)

(The procedure was successful)STOP

Step 5 Set k = k + 1Step 6 For i = 1, . . . , n set XOi = xi

Step 7 OUTPUT (‘Maximum number of iterations exceeded’)(The procedure was successful)STOP

Numerical Analysis (Chapter 7) Jacobi & Gauss-Seidel Methods I R L Burden & J D Faires 24 / 26

Page 46: Gauss & Jordan Iterative Techniques

Introduction Jacobi’s Method Equivalent System Jacobi Algorithm

The Jacobi Iterative Algorithm (2/2)

Step 1 Set k = 1Step 2 While (k ≤ N) do Steps 3–6

Step 3 For i = 1, . . . , n

set xi =1aii

[−

∑nj=1j 6=i

(aijXOj) + bi

]

Step 4 If ||x− XO|| < TOL then OUTPUT (x1, . . . , xn)(The procedure was successful)

STOPStep 5 Set k = k + 1Step 6 For i = 1, . . . , n set XOi = xi

Step 7 OUTPUT (‘Maximum number of iterations exceeded’)(The procedure was successful)STOP

Numerical Analysis (Chapter 7) Jacobi & Gauss-Seidel Methods I R L Burden & J D Faires 24 / 26

Page 47: Gauss & Jordan Iterative Techniques

Introduction Jacobi’s Method Equivalent System Jacobi Algorithm

The Jacobi Iterative Algorithm (2/2)

Step 1 Set k = 1Step 2 While (k ≤ N) do Steps 3–6

Step 3 For i = 1, . . . , n

set xi =1aii

[−

∑nj=1j 6=i

(aijXOj) + bi

]Step 4 If ||x− XO|| < TOL then OUTPUT (x1, . . . , xn)

(The procedure was successful)STOP

Step 5 Set k = k + 1Step 6 For i = 1, . . . , n set XOi = xi

Step 7 OUTPUT (‘Maximum number of iterations exceeded’)(The procedure was successful)STOP

Numerical Analysis (Chapter 7) Jacobi & Gauss-Seidel Methods I R L Burden & J D Faires 24 / 26

Page 48: Gauss & Jordan Iterative Techniques

Introduction Jacobi’s Method Equivalent System Jacobi Algorithm

The Jacobi Iterative Algorithm (2/2)

Step 1 Set k = 1Step 2 While (k ≤ N) do Steps 3–6

Step 3 For i = 1, . . . , n

set xi =1aii

[−

∑nj=1j 6=i

(aijXOj) + bi

]Step 4 If ||x− XO|| < TOL then OUTPUT (x1, . . . , xn)

(The procedure was successful)STOP

Step 5 Set k = k + 1Step 6 For i = 1, . . . , n set XOi = xi

Step 7 OUTPUT (‘Maximum number of iterations exceeded’)(The procedure was successful)STOP

Numerical Analysis (Chapter 7) Jacobi & Gauss-Seidel Methods I R L Burden & J D Faires 24 / 26

Page 49: Gauss & Jordan Iterative Techniques

Introduction Jacobi’s Method Equivalent System Jacobi Algorithm

The Jacobi Iterative Algorithm (2/2)

Step 1 Set k = 1Step 2 While (k ≤ N) do Steps 3–6

Step 3 For i = 1, . . . , n

set xi =1aii

[−

∑nj=1j 6=i

(aijXOj) + bi

]Step 4 If ||x− XO|| < TOL then OUTPUT (x1, . . . , xn)

(The procedure was successful)STOP

Step 5 Set k = k + 1

Step 6 For i = 1, . . . , n set XOi = xi

Step 7 OUTPUT (‘Maximum number of iterations exceeded’)(The procedure was successful)STOP

Numerical Analysis (Chapter 7) Jacobi & Gauss-Seidel Methods I R L Burden & J D Faires 24 / 26

Page 50: Gauss & Jordan Iterative Techniques

Introduction Jacobi’s Method Equivalent System Jacobi Algorithm

The Jacobi Iterative Algorithm (2/2)

Step 1 Set k = 1Step 2 While (k ≤ N) do Steps 3–6

Step 3 For i = 1, . . . , n

set xi =1aii

[−

∑nj=1j 6=i

(aijXOj) + bi

]Step 4 If ||x− XO|| < TOL then OUTPUT (x1, . . . , xn)

(The procedure was successful)STOP

Step 5 Set k = k + 1Step 6 For i = 1, . . . , n set XOi = xi

Step 7 OUTPUT (‘Maximum number of iterations exceeded’)(The procedure was successful)STOP

Numerical Analysis (Chapter 7) Jacobi & Gauss-Seidel Methods I R L Burden & J D Faires 24 / 26

Page 51: Gauss & Jordan Iterative Techniques

Introduction Jacobi’s Method Equivalent System Jacobi Algorithm

The Jacobi Iterative Algorithm (2/2)

Step 1 Set k = 1Step 2 While (k ≤ N) do Steps 3–6

Step 3 For i = 1, . . . , n

set xi =1aii

[−

∑nj=1j 6=i

(aijXOj) + bi

]Step 4 If ||x− XO|| < TOL then OUTPUT (x1, . . . , xn)

(The procedure was successful)STOP

Step 5 Set k = k + 1Step 6 For i = 1, . . . , n set XOi = xi

Step 7 OUTPUT (‘Maximum number of iterations exceeded’)(The procedure was successful)STOP

Numerical Analysis (Chapter 7) Jacobi & Gauss-Seidel Methods I R L Burden & J D Faires 24 / 26

Page 52: Gauss & Jordan Iterative Techniques

Introduction Jacobi’s Method Equivalent System Jacobi Algorithm

Jacobi’s MethodComments on the Algorithm

Step 3 of the algorithm requires that aii 6= 0, for eachi = 1, 2, . . . , n.

If one of the aii entries is 0 and the system isnonsingular, a reordering of the equations can be performed sothat no aii = 0.To speed convergence, the equations should be arranged so thataii is as large as possible.Another possible stopping criterion in Step 4 is to iterate until

‖x(k) − x(k−1)‖‖x(k)‖

is smaller than some prescribed tolerance.For this purpose, any convenient norm can be used, the usualbeing the l∞ norm.

Numerical Analysis (Chapter 7) Jacobi & Gauss-Seidel Methods I R L Burden & J D Faires 25 / 26

Page 53: Gauss & Jordan Iterative Techniques

Introduction Jacobi’s Method Equivalent System Jacobi Algorithm

Jacobi’s MethodComments on the Algorithm

Step 3 of the algorithm requires that aii 6= 0, for eachi = 1, 2, . . . , n. If one of the aii entries is 0 and the system isnonsingular, a reordering of the equations can be performed sothat no aii = 0.

To speed convergence, the equations should be arranged so thataii is as large as possible.Another possible stopping criterion in Step 4 is to iterate until

‖x(k) − x(k−1)‖‖x(k)‖

is smaller than some prescribed tolerance.For this purpose, any convenient norm can be used, the usualbeing the l∞ norm.

Numerical Analysis (Chapter 7) Jacobi & Gauss-Seidel Methods I R L Burden & J D Faires 25 / 26

Page 54: Gauss & Jordan Iterative Techniques

Introduction Jacobi’s Method Equivalent System Jacobi Algorithm

Jacobi’s MethodComments on the Algorithm

Step 3 of the algorithm requires that aii 6= 0, for eachi = 1, 2, . . . , n. If one of the aii entries is 0 and the system isnonsingular, a reordering of the equations can be performed sothat no aii = 0.To speed convergence, the equations should be arranged so thataii is as large as possible.

Another possible stopping criterion in Step 4 is to iterate until

‖x(k) − x(k−1)‖‖x(k)‖

is smaller than some prescribed tolerance.For this purpose, any convenient norm can be used, the usualbeing the l∞ norm.

Numerical Analysis (Chapter 7) Jacobi & Gauss-Seidel Methods I R L Burden & J D Faires 25 / 26

Page 55: Gauss & Jordan Iterative Techniques

Introduction Jacobi’s Method Equivalent System Jacobi Algorithm

Jacobi’s MethodComments on the Algorithm

Step 3 of the algorithm requires that aii 6= 0, for eachi = 1, 2, . . . , n. If one of the aii entries is 0 and the system isnonsingular, a reordering of the equations can be performed sothat no aii = 0.To speed convergence, the equations should be arranged so thataii is as large as possible.Another possible stopping criterion in Step 4 is to iterate until

‖x(k) − x(k−1)‖‖x(k)‖

is smaller than some prescribed tolerance.

For this purpose, any convenient norm can be used, the usualbeing the l∞ norm.

Numerical Analysis (Chapter 7) Jacobi & Gauss-Seidel Methods I R L Burden & J D Faires 25 / 26

Page 56: Gauss & Jordan Iterative Techniques

Introduction Jacobi’s Method Equivalent System Jacobi Algorithm

Jacobi’s MethodComments on the Algorithm

Step 3 of the algorithm requires that aii 6= 0, for eachi = 1, 2, . . . , n. If one of the aii entries is 0 and the system isnonsingular, a reordering of the equations can be performed sothat no aii = 0.To speed convergence, the equations should be arranged so thataii is as large as possible.Another possible stopping criterion in Step 4 is to iterate until

‖x(k) − x(k−1)‖‖x(k)‖

is smaller than some prescribed tolerance.For this purpose, any convenient norm can be used, the usualbeing the l∞ norm.

Numerical Analysis (Chapter 7) Jacobi & Gauss-Seidel Methods I R L Burden & J D Faires 25 / 26

Page 57: Gauss & Jordan Iterative Techniques

Questions?