Upload
na2ry
View
235
Download
0
Embed Size (px)
Citation preview
8/9/2019 Lecture03 - Iterative Methods
1/21
Solution of Linear System of Equations
Lecture 3:
Iterative Methods
MTH2212 Computational Methods and Statistics
8/9/2019 Lecture03 - Iterative Methods
2/21
Dr. M. HrairiDr. M. Hrairi MTH2212MTH2212 -- Computational Methods and StatisticsComputational Methods and Statistics 22
Objectives
Introduction
Jacobi Method
Gauss-Seidel Method
8/9/2019 Lecture03 - Iterative Methods
3/21
Dr. M. HrairiDr. M. Hrairi MTH2212MTH2212 -- Computational Methods and StatisticsComputational Methods and Statistics 33
Introduction
To solve the linear system Ax = b we may use either:
Direct Methods
- Gaussianelimination
- PLU decomposition Iterative Methods
- Jacobi Method
- Gauss-Seidel Method
8/9/2019 Lecture03 - Iterative Methods
4/21
Dr. M. HrairiDr. M. Hrairi MTH2212MTH2212 -- Computational Methods and StatisticsComputational Methods and Statistics 44
Iterative Methods
Suppose we solve Ax = b for a given matrix A by finding the
PLU decomposition
Ifwe change the vectorb,we may continue to use the PLU
Ifwe change A,we now have to re-compute the PLUdecomposition: expensive
8/9/2019 Lecture03 - Iterative Methods
5/21
Dr. M. HrairiDr. M. Hrairi MTHMTH22122212 -- Computational Methods and StatisticsComputational Methods and Statistics 55
Iterative Methods
Instead, suppose we have solved the system
Ax = b
for a given matrix A
Suppose we change A slightly, e.g., modify a single resistor
in a circuit
Ifwe call that new matrix Amod
, is it possible to use the
solution to Ax = b to solve Amod
x = b?
8/9/2019 Lecture03 - Iterative Methods
6/21
Dr. M. HrairiDr. M. Hrairi MTHMTH22122212 -- Computational Methods and StatisticsComputational Methods and Statistics 66
Iterative Methods
They provide an alternative to the elimination method.
Let Ax = b be the set ofequations to be solved.
The system Ax = b is reshaped by solving the first equation
for x1, the second equation for x2, and the third for x3, and
nth equation for xn.
8/9/2019 Lecture03 - Iterative Methods
7/21
Dr. M. HrairiDr. M. Hrairi MTHMTH22122212 -- Computational Methods and StatisticsComputational Methods and Statistics 77
Iterative Methods
For ease ofcomputation, lets assume we have a 3x3
system ofequations to solve.
If the diagonal elements are all non-zero then:
!!
!
3333232131
2323222121
1313212111
bxaxaxa
bxaxaxa
bxaxaxa
33
23213133
22
32312122
11
31321211
a
xaxabx
a
xaxabx
a
xaxabx
!
!
!
8/9/2019 Lecture03 - Iterative Methods
8/21
Dr. M. HrairiDr. M. Hrairi MTHMTH22122212 -- Computational Methods and StatisticsComputational Methods and Statistics 88
Jacobi Iteration Method
1. Assume all the xs are zero
2. Substitute the zeros into the three equations to get:
3. Repeat the procedure until the error criterion is satisfied:
11
1
1 a
bx !
22
2
2 a
bx !
33
3
3 a
bx !
si
j
i
j
i
jja
xxx Iev
!I
%00,
33
23233
3
22
32322
2
3322
a
xaxabx
a
xaxabx
a
xaxabx
ii
i
iii
ii
i
!
!
!
8/9/2019 Lecture03 - Iterative Methods
9/21
Dr. M. HrairiDr. M. Hrairi MTHMTH22122212 -- Computational Methods and StatisticsComputational Methods and Statistics 99
Gauss-Seidel Method
It is the most commonly used iterative method.
8/9/2019 Lecture03 - Iterative Methods
10/21
Dr. M. HrairiDr. M. Hrairi MTHMTH22122212 -- Computational Methods and StatisticsComputational Methods and Statistics 1010
Gauss-Seidel Procedure
1. Assume all the xs are zero
2. Substitute the zeros into the first equation i.e. equation (1)to give:
3. Substitute the new value ofx1 and x3 = 0 into equation (2)to compute x2
4. Substitute the value ofx1 and the new value ofx2 inequation (3) to estimate x3
11
11
a
bx !
8/9/2019 Lecture03 - Iterative Methods
11/21
Dr. M. HrairiDr. M. Hrairi MTHMTH22122212 -- Computational Methods and StatisticsComputational Methods and Statistics 1111
Gauss-Seidel Procedure
5. Return to equation (1) and repeat the entire procedure untilthe error criterion is satisfied:
33
1
232
1
13131
3
22
323
1
12121
2
11
31321211
1
a
aab
a
aab
a
aab
ii
i
ii
i
ii
i
si
j
i
j
i
jja
xxx Iev
I
%00,
8/9/2019 Lecture03 - Iterative Methods
12/21
Dr. M. HrairiDr. M. Hrairi MTHMTH22122212 -- Computational Methods and StatisticsComputational Methods and Statistics 1212
Example 1
Use Gauss-Seidel method to solve the following set of
linear equations:
3x1 0.1x2 0.2x3 = 7.85 (1)
0.1x1 + 7x2 0.3x3 = -19.3 (2)
0.3x1 0.2x2 + 10x3 = 71.4 (3)
8/9/2019 Lecture03 - Iterative Methods
13/21
Dr. M. HrairiDr. M. Hrairi MTHMTH22122212 -- Computational Methods and StatisticsComputational Methods and Statistics 1313
Example 1 - Solution
First we have:
10
2.03.04.71
7
3.01.03.19
3
2.01.085.7
213
31
2
321
xx
x
xx
x
xx
x
8/9/2019 Lecture03 - Iterative Methods
14/21
Dr. M. HrairiDr. M. Hrairi MTHMTH22122212 -- Computational Methods and StatisticsComputational Methods and Statistics 1414
Example 1 - Solution
1st iteration
Assumethat x2 = 0 and x3 = 0, weobtain
Substitute x1 = 2.616667 and x3 = 0 intoequation (2)
Substitute x1
= 2.616667 and x2
= -2.794524 intoequation (3)
This completes the first iteration
6 6667.23
85.7!!x
794524.27
0)616667.2(1.03.192
x
0056 0.70
)794524.2(2.0)6 6667.2(3.04.73 !
!x
8/9/2019 Lecture03 - Iterative Methods
15/21
Dr. M. HrairiDr. M. Hrairi MTHMTH22122212 -- Computational Methods and StatisticsComputational Methods and Statistics 1515
Example 1 - Solution
2nd iteration
990557.)005610.7(.0)79454.(1.085.7
1 !
!x
499625.27
)005610.7(3.0)990557.2(1.03.192 !
!x
000291.710
)499625.2(2.0)990557.2(3.04.713 !
!x
8/9/2019 Lecture03 - Iterative Methods
16/21
Dr. M. HrairiDr. M. Hrairi MTHMTH22122212 -- Computational Methods and StatisticsComputational Methods and Statistics 1616
Example 1 - Solution
Error estimate
Forx1
Forx2
Forx3
%12%12
1221 !
! aI
%8.11%100499625.2
)794524.2(499625.22, !
! aI
%% !
! aI
8/9/2019 Lecture03 - Iterative Methods
17/21
Dr. M. HrairiDr. M. Hrairi MTHMTH22122212 -- Computational Methods and StatisticsComputational Methods and Statistics 1717
Convergence
Gauss-Seidel is similar in spirit to the simple fixed-pointiteration.
Gauss-Seidel will converge iffor every equation of the
system,w
e have:
Such system is said to be diagonally dominant.
This criterion is sufficient but not necessary forconvergence.
{!
"n
ijj
ijii aa1
8/9/2019 Lecture03 - Iterative Methods
18/21
Dr. M. HrairiDr. M. Hrairi MTHMTH22122212 -- Computational Methods and StatisticsComputational Methods and Statistics 1818
Relaxation
Designed to Enhance convergence.
After each new value ofx is computed, that value ismodified using:
Where is a weighting factor.
The choice of is problem-specific and is often determinedempirically.
oldi
new
i
new
ixxx PP! 1
20 ePe
P
8/9/2019 Lecture03 - Iterative Methods
19/21
Dr. M. HrairiDr. M. Hrairi MTHMTH22122212 -- Computational Methods and StatisticsComputational Methods and Statistics 1919
Gauss-Seidel/Jacobi Iteration Methods
8/9/2019 Lecture03 - Iterative Methods
20/21
Dr. M. HrairiDr. M. Hrairi MTHMTH22122212 -- Computational Methods and StatisticsComputational Methods and Statistics 2020
Gauss-Seidel iteration converges more rapidly than the
Jacobi iteration does; since, it uses the latest updates.
But there are some cases that Jacobi iteration does
converge but Gauss-Seidel does not.
Gauss-Seidel/Jacobi Iteration Methods
8/9/2019 Lecture03 - Iterative Methods
21/21
Dr. M. HrairiDr. M. Hrairi MTHMTH22122212 -- Computational Methods and StatisticsComputational Methods and Statistics 2121
Assignment #1
Computational Methods
12.11, 12.30, 12.33
Statistics
2.2, 2.14, 2.22, 2.26, 2.28, 2.37, 2.45, 2.52, 2.65, 2.74