Fixed-Point Iteration Douglas Wilhelm Harder Department of Electrical and Computer Engineering University of Waterloo Copyright © 2007 by Douglas Wilhelm

Fixed-Point Iteration

Douglas Wilhelm Harder

Department of Electrical and Computer Engineering

University of Waterloo

Copyright © 2007 by Douglas Wilhelm Harder. All rights reserved.

ECE 204 Numerical Methods for Computer Engineers


• In this topic, we will look at:– how fixed-point iteration can be used to solve

certain problems numerically, and– some of the various limitations of iterative

numerical methods including:• determining convergence,• slow convergence, and• non-convergence


• Everyone has at some point punched in an arbitrary number on a scientific calculator and started pushing one of the buttons

• We will look at two examples: sin and cos


• Starting with x = 0, if I repeatedly hit the cos button, I get the sequence:x = 0

x = 1

x = 0.540302305868140

x = 0.857553215846393

x = 0.654289790497779

x = 0.793480358742566

x = 0.701368773622757

x = 0.763959682900654

x = 0.722102425026708

x = 0.750417761763761

x = 0.731404042422510

x = 0.744237354900557

x = 0.735604740436347

x = 0.741425086610109

x = 0.737506890513243

x = 0.740147335567876

x = 0.738369204122323

x = 0.739567202212256

x = 0.738760319874211


• Looking at the first three digits of the last three iterations, we note that they appear to be converging:0.738369204122323

0.739567202212256

0.738760319874211


• After applying cos 100 times using Matlab, I get the sequence:x = 0.739085133215166

x = 0.739085133215157

x = 0.739085133215163

x = 0.739085133215159

x = 0.739085133215162

x = 0.739085133215160

x = 0.739085133215161

x = 0.739085133215160

x = 0.739085133215161

x = 0.739085133215160

x = 0.739085133215161

x = 0.739085133215161

x = 0.739085133215161


• If, however, you set your calculator to degrees instead of radians, you would have found a different sequence:>> x = 0;

>> for i = 1:6

x = cos(x/180*pi)

end

x = 1

x = 0.999847695156391

x = 0.999847741545212

x = 0.999847741531084

x = 0.999847741531088

x = 0.999847741531088


• In this case, the sequence converged significantly faster, but to what?


• Given an equation of the form

x = f(x)

if |f(1)(x)| < 1 in a neighbourhood of a solution to this equation, then for any initial point x0 in this neighbourhood, the sequence xk + 1

= f(xk) will converge to the solution to this equation


• The examples we have been looking at were finding points such that:

x = cos(x)

• This will differ depending on whether you are using radians or degrees:


• We will now look at one of the longest known examples of fixed-point iteration

• Starting with any value and repeated apply the function

f(x) = x/2 + 1/x

• For centuries, it has been known that this converges to 2


• To see why, recall that this converges to a solution of

x = x/2 + 1/x

which is also a solution ofx2 = x2/2 + 1

orx2/2 = 1

x2 = 2

Newton’s Method

• We also get this sequence by applying Newton’s method which was covered in first year

• The functionf(x) = x2 – 2

has two roots, and we can find these roots using Newton’s method:

Newton’s Method

• Thus, we iterate

• Looking at the right hand side, it simplifies to:

n

nn

n

nnn

x

xx

x

xxx

2

2

)(f

)f(

2

)1(1

x

x

x

xx

1

22

22


• For example, consider the equation

x = 2x(x – 1)

• This has two solutions, namely

x = 0 and x = 0.5

• If we start with x0 = 0.1 and define

f(x) = 2x(x – 1), we get the sequence


• Sequence is

k xk |xk – xk – 1|

0 0.1 –

1 0.18 0.08

2 0.2952 0.1152

3 0.41611392 0.1209

4 0.4859262511644672 0.06981

5 0.4996038591874287 0.01368

6 0.4999996861449132 0.0003958

7 0.4999999999998030 0.0000003139

8 0.5 0.0000000000001970

9 0.5 0


• To solve any such problem, we could continue to iterate until the sequence no longer changes, however, this does not always work

• Consider the slight modification, solving

x = 2.4x(x – 1)

which has the solution x = 7/12 ≈ 0.58333⋅⋅


• Starting with x0 = 0.1, we converge much more slowly, and after 32 iterations:

k xk |xk – xk – 1|

32 0.5833333333333457 0.432 10 –11

33 0.5833333333333284 0.173 10 –11

34 0.5833333333333354 0.070 10 –11

35 0.5833333333333325 0.029 10 –11

36 0.5833333333333337 0.012 10 –11

37 0.5833333333333332 0.005 10 –11

38 0.5833333333333334 0.002 10 –11

39 0.5833333333333332 0.002 10 –11

40 0.5833333333333334 0.002 10 –11


• This sequence will never converge, and therefore we need a weaker halting condition

• We will continue iterating until:

|xk – xk – 1| < step

where step is a suitably chosen predetermined criteria


• Unfortunately, this may also lead to erroneous results


• Suppose we are trying to solve

x = sin(x)

which has only one solution x = 0

• After 1000 iterations, we still have:

x1000 = 0.04802218067056788

x1001 = 0.04800372523478609

x1002 = 0.04798529106705650

x1003 = 0.04796687812655410


• In this case, the rate of convergence is very, very slow

• The following table lists the number of iterations required to satisfy the terminating condition:


• The following table shows how many iterations are required

step Iterations xk

10 –4 .. 123 0.0841946299701288

10 –5 .. 1658 0.03913176037625081

10 –6 .. 8785 0.01816976184230423

10 –7 .. 41870 0.008434193162746495

10 –8 .. 195441 0.003914852569606909

10 –9.. 908257 0.001817119078995468

10 –11 4216859 0.0008434325248239295

10 –12 19574033 0.0003914867496521149


• Notice that a small step size does not suggest that we are anywhere near the solution?

• After 20 million iterations:– the step size is less than 10–12, but– the value 0.0003915867·· has only three zeros

after the decimal place


• Therefore, we must restrict the number of iterations required to some fixed amount, say N

• Thus, if we iterate N times and have not satisfied our halting conditions, we stop and indicate that the method failed to converge


• If we stop due to iterating N times, this may mean that either:– a solution does not exist,– we chose a poor initial condition, or– we are using an inappropriate method for

finding a solution


• Alternatively, to find a solution to x = sin(x),

we could define f(x) = sin(x) – x and then iterate

• Thus, we get:x0 = 0.1

x1 = – 0.0000333452420668

x2 = 0.00000000001074791837

1)cos(

)sin(31

k

kkkk x

xxxx


• In this topic, we have seen a number of examples of fixed-point iteration– it may be possible to solve equations of the form x

= f(x) by repeated applying f(x)– the form of the iteration will affect the convergence– it is necessary to define a halting condition

stopping if the difference is less than step

– the sequence may not converge so we may have to halt after some fixed number of iterations

Usage Notes

• These slides are made publicly available on the web for anyone to use

• If you choose to use them, or a part thereof, for a course at another institution, I ask only three things:– that you inform me that you are using the slides,

– that you acknowledge my work, and

– that you alert me of any mistakes which I made or changes which you make, and allow me the option of incorporating such changes (with an acknowledgment) in my set of slides

Sincerely,

Douglas Wilhelm Harder, MMath

[email protected]

Documents

Fixed-Point Iteration Douglas Wilhelm Harder Department of Electrical and Computer Engineering University of Waterloo Copyright © 2007 by Douglas Wilhelm