Numerical MethodsNumerical Methods

Root FindingRoot Finding

Secant MethodSecant Method

Modified SecantModified Secant

False Position MethodFalse Position Method

Newton MethodNewton Method

4

Secant method

Secant method

Secant method

Requires two initial estimates

f(x) is not required to change signs, therefore

this is not a bracketing method

ii

iiiii xfxf

xxxfxx

1

11

Secant method

{initial estimates

f(x)

x

Select two estimates.Note: f(xi) and f(xi+1)are not oppositesigns.

Secant method

initial estimates

slopebetweentwoestimates

f(x)

x{

Secant method

new estimate initial estimates

slopebetweentwoestimates

f(x)

x{

To get xn+1 from xn and xn-1 we write out the equation of the secant line and using the points xn and xn-1. We then plug in the point (xn+1,0) and solve for xn+1.

)()(

)(

)()(

)(

)()()(0

)()()(

1

11

11

1

11

1

1

1

nn

nnnnn

nnnn

nnn

nnnn

nnn

nnn

nnn

xfxf

xxxfxx

xxxfxf

xxxf

xxxx

xfxfxf

xxxx

xfxfxfy equation of secant

substitute (xn+1,0)

solve for xn+1

xn+1=h(xn-1,xn)

The equation above gives the recursively defined sequence for xn. This is what is used for the Secant Method. The halting condition is usually given by the Standard Cauchy Error.

Secant Illustration

-20

0

20

40

60

80

100

120

1 2 3 4 5 6 7 8 9 10 11 12

Series1

F(x) = x2 - 101 (a=1, fa=-9) (b=10, fb=90) int = 1.8, fint = -6.72 (a=10, fa=90) (b=1.8, fb= -6.7) int = 0.88, fint = -9.223 (a=1.8, fa=-6.7) (b=0.88, fb=-9.22) int = 4.25, fint = 84 (a=0.88, fa=-9.22) (b=4.25, fb=8) Int =2.68, fint = -2.8Etc…

1 24

3

Determine the root of f(x) = e-x - x using the secant method. Use the starting points x0 = 0 and x1 = 1.0.

-2

0

2

4

6

8

10

-2 -1 0 1 2

x

f(x)

Example

Choose two starting points x0 = 0 f(x0 ) =1

x1 = 1.0 f(x1) = -0.632

Calculate x2

x2 = 1 - (-0.632)(0 - 1)/(1+0.632) = 0.6127

Solution

Second iteration x1 = 1.0 f(x1) = -0.632 x2 = 0.613 f(x2) = -0.0708 NOTE: f(x) are the same sign. OK here.

x3 = 0.613 - (-0.0708)(1-0.613)/(-0.632+0.0708)

x3 = 0.564 f(x3) = 0.0052 a = abs[(0.564-0.613)/(0.564)] x 100 =

8.23%

Solution

Third iteration x2 = 0.613 f(x2) = -0.0708

x3 = 0.564 f(x3) = 0.0052

x4 = 0.567 f(x4) = -0.00004

a = 0.59% t = 0.0048%

Know the difference between these error terms

Solution

double secant(double c, int iterations, double tol)…

for ( int i = 0; i < iterations; i++){ double x = ( fa*b - fb*a ) / ( fa - fb ); double fx = func( x, c ); if ( fabs( fx ) < tol ) return x; a = b; fa = fb; b = x; fb = fx;}return -1;

SECANT METHODThe change from ordinaryregula is that the sign check is dropped and points are just “shifted over”

Example

16

Ex.

Use secant method to estimate the root of f(x) = ln x.Start the computation with value of

xl = xi-1 = 0.5 xu = xi = 5.0

Solution

The secant method

17

Example 1

You are making a bookshelf to carry books that range from 8 ½ ” to 11” in height and would take 29”of space along length. The material is wood having Young’s Modulus 3.667 Msi, thickness 3/8 ” and width 12”. You want to find the maximum vertical deflection of the bookshelf. The vertical deflection of the shelf is given by

xxxxxv 018507.010 x 66722.010 x 13533.010 x 42493.0)( 4-65-83-4

where x is the position where the deflection is maximum. Hence to find

the maximum deflection we need to find where and conduct the second derivative test.

0)( dx

dvxf

18

Example 1 Cont.

The equation that gives the position x where the deflection is maximum is

given by

Use the secant method of finding roots of equations to find the position where the deflection is maximum. Conduct three iterations to estimate the root of the above equation. Find the absolute relative approximate error at the end of each iteration and the number of significant digits at least correct at the end of each iteration.

0018507.010x 12748.010 x 26689.010x 67665.0)f( 2-33-54-8 x xx x

Books

Bookshelf

Figure 2 A loaded bookshelf.

19

Example 1 Cont.

0018507.010x 12748.010 x 26689.010x 67665.0)f( 2-33-54-8 x xx x

Figure 3 Graph of the function f(x).

0 5 10 15 20 25 300.02

0.01

0

0.01

0.02

f(x)

0.01883

0.01851

0f x( )

290 x

20

Example 1 Cont.

Let us take the initial guesses of the root of as and .

Iteration 1

The estimate of the root is

0xf 101 x 150 x

4

233548

20

330

540

80

102591.8

018507.0151012748.015106689.2151067665.0

018507.01012748.0106689.21067665.0

xxxxf

10

10001

xfxf

xxxfxx

3

233548

21

331

541

81

104956.8

018507.0101012748.010106689.2101067665.0

018507.01012748.0106689.21067665.0

xxxxf

Solution

21

Example 1 Cont.

557.14

104956.8102591.8

1015102591.815

34

4

1

x

0 5 10 15 20 25 300.03

0.02

0.01

0

0.01

0.02

0.03

f(x)x'1, (first guess)x0, (previous guess)Secant linex1, (new guess)

0.027

0.027

0

f x( )

f x( )

f x( )

secant x( )

f x( )

290 x x 0 x 1' x x 1

22

Example 1 Cont.

The absolute relative approximate error at the end of Iteration 1 isa

% 0433.3

100557.14

15557.14

1001

01

x

xxa

The number of significant digits at least correct is 1, because the absolute relative approximate error is less than 5%.

23

Example 1 Cont.

Iteration 2

The estimate of the root is

5

233548

21

331

541

81

109870.2

018507.0557.141012748.0557.14106689.2557.141067665.0

018507.01012748.0106689.21067665.0

xxxxf

01

01112 xfxf

xxxfxx

572.14

102591.8109870.2

15557.14109870.215

45

5

2

x

24

Example 1 Cont.

The absolute relative approximate error at the end of Iteration 2 isa

% 10611.0

100572.14

557.14572.14

1002

12

x

xxa

The number of significant digits at least correct is 2, because the absolute relative approximate error is less than 0.5%.

25

Example 1 Cont.

Iteration 3

The estimate of the root is

9

233548

22

332

542

82

100676.6

018507.0572.141012748.0572.14106689.2572.141067665.0

018507.01012748.0106689.21067665.0

xxxxf

12

12223 xfxf

xxxfxx

572.14

109870.2100676.6

557.14572.14100676.6572.14

59

9

2

x

26

Example 1 Cont.

The absolute relative approximate error at the end of Iteration 3 isa

%102.1559

100572.14

572.14572.14

100

5

2

12

x

xxa

The number of significant digits at least correct is 6, because the absolute relative approximate error is less than 0.00005%.

Problems With the Secant Method

The number of iterations required can not be determined before the algorithm begins.

The algorithm will halt (program termination by division by zero if not checked for) if a horizontal secant line is encountered.

The secant method will sometimes find an extraneous root.

Root FindingModified Secant Method

29

Modified Secant Method

)()()(

1

11:Secant Original

ii

iiiii xfxf

xxxfxx

)()()(

:formulaR -N original the in )()(

)('

compute to fraction onperturbati small a Use

:Secant Modifiediii

iiii

i

iiii

xfxxf

xxfxx

x

xfxxfxf

1

30

Modified Secant Method

)()(

))((

1

11

ii

iiiii xfxf

xxxfxx

)()(

)(1

iii

iiii xfxxf

xfxxx

ii

iii xx

xfxfxf

1

1 )()()(

i

iiii x

xfxxfxf

)()(

)(

Modified Secant Method

Original Secant Method

Ex.

Use the secant method to estimate the root of

f(x) = e-x – x.

Use a value of 0.01 for and start with x0 = 1.0.

Solution (true root = 0.56714329…)

First iteration

x0 = 1 f(x0) = -0.63212

x1+ x0 = 1.01 f(x1+ x0) = -0.64578

Calculate t = 5.3%

)()(

)(1

iii

iiii xfxxf

xfxxx

537263.0

)63212.0(64578.0

)63212.0(01.011

x

You buy a $20 K piece of equipment for nothing downand $5K per year for 5 years. What interest rate are you paying? The formula relating present worth (P), annualpayments (A), number of years (n) and the interest rate(i) is:

A Pi i

i

n

n

1

1 1

Applied Problem

Root FindingFalse Position Method

False Position Method

“Brute Force” of bisection method is inefficient

Join points by a straight line

Improves the estimate

Replacing the curve by a straight line gives

the “false position”

xl

xu

f(xl)

f(xu)

next estimate, xr

f x

x x

f x

x x

x xf x x x

f x f x

l

r l

u

r u

r uu l u

l u

Based on similar triangles

36

The False-Position Method (Regula-Falsi)

We can approximate the solution by doing a linear interpolation between f(xu) and f(xl)

Find xr such that l(xr)=0, where l(x) is the linear approximation of f(x) between xl and xu

Derive xr using similar triangles

lu

luulr ff

fxfxx

Determine the root of the following equation using the false position method starting with an initial estimate of xl=4.55 and xu=4.65

f(x) = x3 - 98

-40

-30

-20

-10

0

10

20

30

4 4.5 5

x

f(x)

Example

Pitfalls of False Position Method

f(x)=x10-1

-5

05

10

15

2025

30

0 0.5 1 1.5

x

f(x)

Comparison of False Position and Secant Method

x

f(x)

x

f(x)

1

1

2

new est.new est.

2

x

f(x)

x

f(x)

1

FALSE POSITION

SECANT METHOD

1

- select first estimate

f(x)

1

2

x

f(x)

1

2

FALSE POSITION

SECANT METHOD

- select second estimate x

f(x)

1

2

x

f(x)

1

2

FALSE POSITION

SECANT METHOD

Note the signof f(x) in each method.False position mustbracket the root.

x

f(x)

1

2

x

f(x)

1

2

FALSE POSITION

SECANT METHOD

- Connect the two points with a line x

f(x)

1

2

new est.

x

f(x)

1

new est.

2

FALSE POSITION

SECANT METHOD

The new estimateis selected from theintersection with thex-axis

x

Root FindingNewton Method

Newton’s Method

Open solution, that requires only one current

guess.

Root does not need to be bracketed.

Consider some point x0.If we approximate f(x) as a line about x0, then we can

again solve for the root of the line.

0 0 0( ) ( )( ) ( )l x f x x x f x

Newton Raphson

tangent

dy

dxf

f xf x

x x

rearrange

x xf x

f x

ii

i i

i ii

i

'

'

'

0

1

1

f(xi)

xi

tangent

xi+1

Newton’s Method

Solving, leads to the following iteration:

01 0

0

1

( ) 0

( )

( )

( )

( )i

i ii

l x

f xx x

f x

f xx x

f x

Secant method

ii

iiiii

i

iii

xfxf

xxxfxx

xf

xfxx

1

11

1 '

Substitute finite difference approximation for thefirst derivative into this equation for Newton Raphson

Newton’s method

f(x)

(x)

solution diverges

Newton RaphsonPitfalls

Finding a square-root

Example: 2 = 1.4142135623730950488016887242097

Let x0 be one and apply Newton’s method.

2

1

0

1

2

( ) 2

2 1 2

2 2

1

1 2 31 1.5000000000

2 1 2

1 3 4 171.4166666667

2 2 3 12

ii i i

i i

f x x

xx x x

x x

x

x

x

Finding a square-root

Example: 2 =

1.4142135623730950488016887242097

Note the rapid convergence

Note, this was done with the standard Microsoft

calculator to maximum precision.

3

4

5

6

1 17 24 5771.414215686

2 12 17 408

1.4142135623746

1.4142135623730950488016896

1.4142135623730950488016887242097

x

x

x

x

Newton’s method

Use the Newton Raphson method to determine the root off(x) = x2 - 11 using an initial guess of xi = 3

-20

0

20

40

60

80

100

0 2 4 6 8 10

x

f(x)

Example

f(x) = x2 - 11

f '(x) = 2x

initial guess xi = 3

f(3) = -2

f '(3) = 6

-20

0

20

40

60

80

100

0 2 4 6 8 10

xf(x)

Solution

x x

f x

f xi ii

i

a

1 32

63

1

33 33

3 33 3

3 33100 10%

'.

.

.

In this method, we begin to use a numbering system:

x0 = 3x1 = 3.33Continue to determine x2, x3 etc.

Solution

%05.0100317.3

315.3317.3

317.363.6

0108.0315.3

'

%55.0100315.3

33.3315.3

315.366.6

11.033.3

'

2

223

1

112

a

a

xf

xfxx

xf

xfxx

Solution

Newton’s Algorithm

Requires the derivative function to be evaluated,

hence more function evaluations per iteration.

A robust solution would check to see if the

iteration is stepping too far and limit the step.

Most uses of Newton’s method assume the

approximation is pretty close and apply one to

three iterations blindly.

Division by Multiplication

Newton’s method has many uses in

computing basic numbers.

For example, consider the equation:

Newton’s method gives the iteration:

10a

x

21

2

1

1

2

kk k k k k

k

k k

ax

x x x x ax

x

x ax

If the numbers xn become closer and

closer to r as n becomes large, then

we say that the sequence converges to r

and we write: lim nn

x r

CONVERGENCE

Applied Problem

The concentration of pollutant bacteria C in a lakedecreases according to:

Determine the time required for the bacteria to be reduced to 10 ppm.

C e et t 80 202 0 1.

Summary

Method Pros ConsBisection - Easy, Reliable, Convergent

- One function evaluation per iteration- No knowledge of derivative is needed

- Slow- Needs an interval [a,b] containing the root, i.e., f(a)f(b)<0

Newton - Fast (if near the root)- Two function evaluations per iteration

- May diverge- Needs derivative and an initial guess x0 such that f’(x0) is nonzero

Secant - Fast (slower than Newton)- One function evaluation per iteration- No knowledge of derivative is needed

- May diverge- Needs two initial points guess x0, x1 such that f(x0)- f(x1) is nonzero