Relaxation Method

Introduction

• Relaxation method is an iterative approach

solution to systems of linear equations.

• Basic idea behind this method is to improve

the solution vector successively by reducing the largest residual at a particular iteration.

What is a residual?

• Suppose x(i) € R is an approximation to the solution of the linear system defined by

Ax=b

• Residual vector for x(i) with respect to this system is

R(i) =b-A x(i) in ith iteration

• The error: E(I )= x-x(i)

• R(i) = b –Ax(i) = Ax –Ax(i) = A(x –x(i)) = AE(i):

• Residual equation:– AE(i)=R(i)

Let x(p) =( x1(p),x2

(p) … xn(p))T

be the solution vector obtained after pth

iteration. If Ri (p) denotes residual,

ai1x1 + ai2x2 + … + ainxn = bi

Define by,

Ri (p) = bi- (ai1x1 + ai2x2 + … + ainxn)

Applying relaxation method

• Transfer all the terms to the right hand side of the equation

• Reorder the equations in a way such that largest co-efficient in the equations appear on the diagonal

• Select the largest residual and give an increment

dx=-r(i)/aii

• Change x(i) to x(i) +dx(i) to relax R(i) that is to reduce R(i) to zero

Example :

6x1-3x2+x3 = 11

2x1+x2-8x3 =-15

x1-7x2+x3 = 10

0= 11- 6x1 - 3x2 - x3 R1

0= 10- x1 + 7x2 - x3 R2

0= -15- 2x1 - x2 + 8x3 R3

• Start with initial guesses x1=x2=x3=0

• R1=11,

• R2=10,

• R3=-15

• Largest residual is R3

• So that dx3 = - R3 /a33

• dx3= -15/-8 = 1.875

New guesses: x1=0 x2=0 and x3=1.875

Continue the process until r 0

Final result would be like this

Iterationno

R1 R2 R3MaxRi dx(i) x1 x2 x3

0 11 10 -15 1.875 0 0 0

1 -9.125 8.125 0 9.125 1.5288 0 0 1.875

2 0.0478 6.5962

-3.0576

6.5962 -0.9423

1.5288 0 1.875

3 -2.8747

0.0001

-2.1153

-2.8747

-0.4791

1.5288 -0.9423

1.875

4 -0.0031

0.4792

-1.1571

-1.1571

0.1446 1.0497 -0.9423

1.875

5 0.1447 0.3346

0.0003 0.3346 -0.0478

1.0497 -0.9423

2.0196

6 0.2881 0.0000

0.0475 0.2881 0.0480 1.0497 -0.9901

2.0196

7 -0.0001

0.048 0.1435 0.1435 -0.0179

1.0017 -0.9901

2.0196

8 0.0178 0.0659

0.0003 - - 1.0017 -0.9901

2.0017

• At ith iteration we can see that R1,R2 and R3 are small enough,

• So xi values in this iteration x1 = 1.007,

x2 = -0.9901,

x3 = 2.0017

• Which are very close to the Exact solutionsx1 = 1.0

x2 = -1.0

x3 = 2.0

Convergence

• Converges slowly for large systems of equations (large n)

Special cases

• Simple to implement

• Not useful as a stand alone solution method

• Key ingredients to multi grid methods– Jacobi– Gauss seidel– red

Comparison with Other Methods

Methods available to find solutions

Direct

Elimination Gaussian elimination Gauss-Jordan

elimination

Decomposition Court's reduction

(Cholesky's reduction)

Iterative

Jacobi's method Gauss-Seidel method Relaxation method

Advantages and Disadvantages

Relaxation method is the core part of linear algebra.

This method provide preconditions for new methods.

Easily adoptable to computers.

Can solve more than 100s of linear equations

simultaneously.

Slower progress than the competing methods

Solve: 6x - 3y + z = 11

2x + y - 8z = -15 x - 7y + z = 10

Gaussian Elimination

Gauss- Jordan

Elimination

Courts Reduction

Relaxation method

X 1

1

1

1.0017

Y -1

-1

-1

-0 9901

Z 2

2

2

2.0017

Relaxation method is the best method for :

Relaxation method is highly used for image processing .

This method has been developed for analysis of hydraulic structures .

Solving linear equations relating to the radiosity problem.

Relaxation methods are iterative methods for solving systems of equations, including nonlinear systems.

Relaxation method used with other numerical methods in mono-tropic programs.

Completed Researches

Why relaxation methods?

• Direct methods are robust.

• Direct methods are less computational costly.

But

• They require high memory access.

• Slow in convergence.

Evolution of relaxation methods

• Gauss Siedel Iteration

Gauss’s letter to Gerling

Era of electronic computing

• Work of David Young Notions - “Consistent Ordering” and “Property A” Convergence of the methods

• Ostrowski (1937) Relevant properties for M-Matrices

• Theorem of Stein – Rosenburg (1948) Asymptotic rates

• Concept of Irreducibility Grid oriented matrices

• Concept of Cyclic Matrices

Convergence theory of SOR methods

• Varga’s Contribution

Generalization of Young’s results

Matrix Iterative Analysis (1962)

Notions – Regular Splittings

Theories -Stieltjes and M-Matrices

Semi Iterative Methods

Richard Varga

• 1960s and 1970s

Chaotic Relaxations

Chazan , Miranker , Miellou , Robert

• Multigrid Methods

Krylov subspace method

Use of Eugene values

ReferencesRao, K.S., Year. Numerical Methods for Scientists and

Engineers. 2nd ed. Delhi: Prentice-Hall of India.

Yousef Sadd and , Henk A. van der Vorst, Iterative Solution of Linear Systems in the 20th Century [pdf]. Available at: <www-users.cs.umn.edu/~saad/PDF/umsi-99-152.pdf> Accessed [12th July 2012]

Relaxation Methods for Iterative Solution to Linear Systems of Equations Gerald Recktenwald Portland State UniversityMechanical Engineering Department

Scientic Computing II Relaxation MethodsMichael BaderSummer term 2012

Working scenario

Demonstration