Upload
suyashagarwal
View
224
Download
2
Tags:
Embed Size (px)
Citation preview
Solving Nonlinear Equations
CHAPTER 2
2
Introduction:
In scientific and engineering studies, a frequently occurring problem is to find the roots or zeros of equations of the form f(x) = 0. (Root Finding Problem)
f(x) may be algebraic or transcendental or a combination of both.
Algebraic functions of the form P(x)= a0 + a1 x + … + an xn , are called polynomials.
A non-algebraic function is called a transcendental function.
3
Polynomials up to degree 4 can be solved exactly. Since finding the root of f(x)=0 is not always possible by analytical means, we have to go for some other techniques or methods.
Solution Methods are either Bracketing:Bisection Method, Regula Falsi Method
or Iterative:Secant Method, Newton's Method, Graffe’s Root Squaring Method,
Rate of Convergence. Advantages/Disadvantages.
Solution Methods
4
Multiplicity
A root p of the equation f(x) = 0 is said to be a Root of Multiplicity m if f can be written in the form
A root of multiplicity one is said to be a Simple Root. Theorem: Let f be a continuous function with m continuous
derivatives. The equation f(x) = 0 has a root of multiplicity m at x = p if and only if
( ) ( ) ( ), where lim ( ) 0.m
x pf x x p q x q x
( 1) ( )( ) ( ) ( ) ... ( ) 0 but ( ) 0.m mf p f p f p f p f p
5
The Bisection Method
The Bisection method is based on the Intermediate Value Theorem and Cantor's Intersection Theorem.
These methods essentially work by first finding an interval which is guaranteed to contain a root and then systematically shrinking the size of that interval.
Intermediate Value Theorem: Let f(x) be continuous function in [a,b] and let k be any number between f(a) and f(b). Then, there exists a number c in (a, b) such that f(c) = k.
Hence, an equation f(x) = 0 where f(x) is a real continuous function, has at least one root between a and b if f(a) f(b) < 0.
6
Bisection method continued …
Cantor's Intersection Theorem:
If a sequence of closed intervals [an, bn] is such that
[an+1, bn+1] [an, bn] for all n, and lim(bn - an)=0 as n → ∞,
then consists of exactly one point.
Algorithm for Bisection Method:
Suppose f(x) is continuous and we have two numbers a1 and b1
such that f(a1)f(b1)<0, then by Intermediate Value Theorem we
know that there is a root for f(x) between a1 and b1.
nnn
ba ,0
7
Bisection method continued …
Then divide the interval into two parts and take the middle point suppose that is c,
if f(c ) =0 then we got the root,
if not so then by IVT the root lies within a and c or c and b, depending on whether f(a1)f(c) <0 or f(c)f(b1)<0.
Then we have to take the next interval in which the root lies and continue our process.
Since we are dividing the interval by half each time, we are approaching to zero, hence this algorithm gives us the approximate solution. Now we have to put some accuracy criteria to stop the process.
8
Bisection method continued …
Algorithm for Bisection Method:
Step 1: Choose a and b as two initial guesses for the root such that f(a) f(b) < 0.
Step 2: Set x= (a+b)/2.
Step 3: If f(a) f(x) < 0, the root lies in the interval (a, x).
Then, set b = x
else set a = x
end if (|a – b| ≤ ε1 or f(x) ≤ ε2)
Remark: The method may produce a false root if f(x) is discontinuous on [a, b].
9
Bisection method continued …
For the given values of 1 and 2, the iteration continues until
|bn - an| < 1 and/or |f(x)| < 2.
The root correct up to desired accuracy is (an - bn)/2.
Advantage: This method definitely converges to the root of any type of non-linear equation.
Disadvantages:
• This method is too slow (linear) i.e. where c <1.
• We need two initial guesses for this method.
1 ,n ne c e
10
Example 1: Find a real root of the equation x3 - 2x - 5 = 0, correct up-to 3-significant digits.
11
Bisection method continued … Example 1:
Find a real root of the equation x3-2x-5 = 0, correct up-to 3-significant digits.
Solution: Let f(x)=x3-2x-5. Now f(2)=-1 and f(3)=16. Hence, by IVT, there exist a root between 2 and 3.
Iteration 1. Now we set a0=2, b0=3, and hence x= (a0+b0)/2=2.5. Since f(2.5)= 5.625, we have f(2) f(2.5) < 0, set a1=2, b1 = 2.5 for the next iteration.
Iteration 2. a1=2, b1=2.5, x=(a1+b1)/2= 2.25, f(2.25)= 1.890625. Since f(2)f(2.25)<0, set a2=2, b2 = 2.25.
12
Bisection method continued … Example 1:
Iteration 3. a2=2, b2=2.25, x=(a2+b2)/2= 2.125, f(2.125)=0.3457. Since f(2)f(2.125)<0, set a3=2, b3=2.125.
Iteration 4. a3=2, b3=2.125, x=(a3+b3)/2= 2.0625, f(2.0625)=-0.3513. Since f(2)f(2.0625)>0, set a4=2.0625, b4=2.125.
Iteration 5. a4=2.0625, b4=2.125, x=(a4+b4)/2 =2.09375, f(2.09375)=-0.0089.
Since f(2.0625)f(2.09375)>0, set a5=2.09375, b5=2.125.
13
Bisection method continued … Example 1:
Iteration 6. a5=2.09375, b5=2.125 Now, x=(a5+b5)/2=2.10938, f(2.10938)=0.1668. Since f(2.09375)f(2.10938)>0, set a6=2.09375, b6=2.10938.
Iteration 7. a6=2.09375, b6=2.10938, x=(a6+b6)/2 =2.10156.
Thus, a 3-significant digits root of the given equation is 2.10156.
Further iterations can be made to get better accuracy.
14
Example 1…
k a b f(a) f(b) x f(x) 1 2.00000 3.00000 -1.00000 16.00000 2.50000 5.62500 2 2.00000 2.50000 -1.00000 5.62500 2.25000 1.89063 3 2.00000 2.25000 -1.00000 1.89063 2.12500 0.34570 4 2.00000 2.12500 -1.00000 0.34570 2.06250 -0.35132 5 2.06250 2.12500 -0.35132 0.34570 2.09375 -0.00894 6 2.09375 2.12500 -0.00894 0.34570 2.10938 0.16684 7 2.09375 2.10938 -0.00894 0.16684 2.10156 0.07856 8 2.09375 2.10156 -0.00894 0.07856 2.09766 0.03471
15
Bisection method continued … Exercises:
Problem1: Perform 5 Iterations of the bisection method to obtain the smallest positive root of the equation
x3-5x+1 =0.
Problem2: Determine the smallest positive root correct up to two decimal places using bisection method for
tan x + tanh x=0.
16
17
The Secant Method
Let x0 and x1 be the two approximation of the root . If the function f is linear, the straight line passes through (x0, f(x0)) and (x1, f(x1)) would intersect the X-axis at the root (y=0). But since f is not linear, the intersection of the line with X-axis is not at x= but this should be close to it.
The equation of chord through (x0, f(x0)) and (x1,f(x1)) is given by
).()()(
)( 001
010 xx
xx
xfxfxfy
18
secant method continued …
At the intersection with X-axis, y=0. So,
On simplification we get
Now we take this x as our next approximation and denote it as x2
,)()(
))((
01
0100 xfxf
xxxfxx
,)()(
)()(
)()(
))((
01
0110
01
0100 xfxf
xfxxfx
xfxf
xxxfxx
19
secant method continued …
From x1 and x2 we can calculate our next approximation x3 using the previous technique and in general, we can write the iteration scheme as
We should start with x0 and x1 such that x1 is closer to the root than x0 to get better result.
,3,2,1,)()(
)()(
1
111
n
xfxf
xfxxfxx
nn
nnnnn
20
secant method continued …Geometrical Interpretation of Secant Method
The approximation points are a,b, x1,x2,x3,…
21
secant method continued … Example: Find the root of f(x)=x3+x2-3x-3 correct up to 3-significant digits with initial approximation x0=1 and x1=2 by secant method.
Solution: The iteration scheme of the secant method is,
Using this formula we get
,3,2,1,)()(
)()(
1
111
n
xfxf
xfxxfxx
nn
nnnnn
,571429.1)1()2(
)1(2)2(1
)()(
)()(
01
01102
ff
ff
xfxf
xfxxfxx
22
secant method continued …
Applying the formula again we get x3 as
Similarly computing further we get
Thus the root correct up to 3-significant digits of the given equation by secant method is 1.731996.
,735136.1)()(
)()(
23
23324
xfxf
xfxxfxx
,705411.1)571419.1()2(
)2(571419.1)571419.1(2
)()(
)()(
12
12213
ff
ff
xfxf
xfxxfxx
,731996.1)()(
)()(
34
34435
xfxf
xfxxfxx
23
secant method continued … Advantages and Disadvantages
Advantages:
• Converges fast (if it converges) in comparison with Bisection method.
• Requires two guesses that do not need to bracket the root.
• Only requires one function evaluation per iteration.
• Avoids calculation of derivatives.
24
secant method continued …
Advantages and DisadvantagesDisadvantages:
• Division by zero
• In this method, it is not necessary to have two initial guesses bracketing the root, but on the other hand, convergence is not guaranteed.
• In some cases, swapping the two initial guesses x0 and x1 will change the behavior of the method from convergent to divergent.
25
secant method continued … Order of Convergence:
Let xn be a sequence of approximations to the root by a given method. Let en = - xn be the error in the n-th approximation of xn.
If there exists a number c 0 (independent of n) such that
then p is the order of convergence of the method.
ce
ep
n
n
n
||
|| 1lim
26
secant method continued …
Order of Convergence of Secant Method:
Prove that the order of convergence of secant method is approximately 1.618.
Proof: We know that by Secant Method the sequence is generated by
Further let the sequence {x0, x1, x2, …} converges to the root, (i.e. f()=0).
2
51
,3,2,1,)()(
)()(
1
111
n
xfxf
xfxxfxx
nn
nnnnn
27
secant method continued … Order of Convergence of Secant Method:
Now we have en+1= - xn+1
Thus
,)()(
)()(
1
11
nn
nnnn
xfxf
xfxxfx
.,, 11 nnnnnn exexorxe
.)()(
)()()()(
1
111
nn
nnnnn efef
efeefee
(1)
28
secant method continued …
Order of Convergence of Secant Method:
Expanding the functions by Taylor’s series we get
Substituting these values in (1) and neglecting the higher degree terms and simplifying
)(!2
)()(!2
)()()(22
fe
fefe
fefef nn
nnn
)(!2
)(
)(!2
)()()(
21
1
21
11
fe
fe
fe
fefef
nn
nnn
29
secant method continued …
Order of Convergence of Secant Method:we get
Let (A constant)
Then we have Suppose then . On substituting these values in (2) we get
.)(
)(
2
111
f
feee nnn
.)(
)(
2
1
f
f
.11 nnn eee pnn ee 1
pnn ee 1
(2)
.1)/1(
/11
/1
1
p
npn
p
nnn ee
eee
30
secant method continued …
Order of Convergence of Secant Method:
Comparing
and equating the powers of en on right sides we get
Solving for p we get
Hence the order of convergence is 1.618.
Therefore the secant method is faster than linear and slower than Newton's quadratic formula.
and1p
nn ee 1)/1(
/11
pnpn ee
011)/1( 2 pporpp
618.12
51
p
31
Regula Falsi Method:
When the bracketing is done for the secant method, it is known as linear interpolation or the method of false position (in Latin, Regula Falsi).
This technique for Regula Falsi method is similar to bisection except the next iterate is taken at the intersection of a line between the pair of X-values and X-axis rather than at the midpoint.
This method gives faster convergence in comparison with bisection method, but at the expense of a more complicated algorithm.
32
Regula Falsi Method continued… Algorithm for Regula Falsi Method:
Step 1: Choose x0 and x1 as two guesses for the
root such that f(x0 ) f(x1)<0.
Step 2: Set
Step 3:
If f(x0)f(x2)<0, the root lies in the interval (x0, x2).
Then, set x1=x2 and go to Step 2.
If f(x0)f(x2)>0, the root lies in the interval (x0, x2).
Then, set x0=x2 and go to Step 2.
)()(
)()(
01
01102 xfxf
xfxxfxx
33
Regula Falsi Method continued… Algorithm for Regula Falsi Method:
If f(x0)f(x2)=0, x2 is a root of the equation and the computation may be terminated.
Continue till a root is obtained to desired accuracy
34
Regula Falsi Method continued… General remarks on Regula Falsi Method:
Asymptotically, the other end-point will remain fixed for all subsequent iterations while one end-point always being updated.
As a result, unlike the bisection method, the width of the bracket does not tend to zero. As a consequence, Regula Falsi approximation to f(x), which is used to pick the false position, does not improve in its quality.
If the initial end-points x0 and x1 are chosen such that f(x0) and f(x1) are of opposite signs, then one of the end-points will converge to a root of f.
35
Regula Falsi method continued … Example:
Let f(x)=2x3-4x2+3x. Let x0=-1 and x1=1. The left end, -1, is never replaced and thus the width of the bracket never falls below 1. Hence, the right endpoint approaches 0 at a linear rate (with a speed of convergence of 2/3).
To accelerate the rate of convergence in such cases, the Modified Regula Falsi Method is used. In this modified rule, Set
if successive x2 falls on right of .
2/)()(
2/)()(
01
01102 xfxf
xfxxfxx
36
Regula Falsi method continued … Example:
or Set
if successive x2 falls on left of .
)(2/)(
)(2/)(
01
01102 xfxf
xfxxfxx
37
Regula Falsi method continued … Example:
Find a real root of the equation x3-2x-5=0 correct up-to 3-significant digits by using Regula Falsi method.
Solution: Here, f(2)=-1 and f(3)=16 and hence a root lies in (2,3). Let x0=2 and x1=3. We get x2 as
Now, f(x2)=-0.390799917 and hence the root lies between 2.058823529 and 3.
058823592.2)()(
)()(
01
01102
xfxf
xfxxfxx
38
Regula Falsi method continued … Example:
Computing x3 we get
Now, f(x3)=-0.147204057 and hence the root lies between 2.08126366 and 3. Similarly
Thus, the root correct up to 3-significant digits of the given equation by secant method is 2.089639211.
08126366.2)()(
)()(
12
12213
xfxf
xfxxfxx
089639211.2)()(
)()(
23
23324
xfxf
xfxxfxx
39
Newton's Method: Choose a value that is reasonably close to
the root of the equation.
Replace the function by its tangent.
Computes the zero of this tangent.
The zero of this tangent will be a better approximation to root of the equation and the procedure can be iterated.
In Newton's Method, we replace the chord in the secant method by a tangent.
40
Geometrical Interpretation of Newton’s Method
41
Newton's Method Continued…Theorem:
Newton-Raphson Theorem: Assume that fC2[a,b] and there exists a number [a,b], where f()=0. If f’() 0, then there exists a >0 such that the sequence defined by the iteration
will converge to for any initial approximation
.
}{0nx
n
,3,2,1,0for ,)(
)(1
n
xf
xfxx
n
nnn
],[0 x
42
Newton-Raphson Theorem Continued…
Proof: Since fC2[a,b], using Taylor's series expansion of f around x0, we have
where c lies in between x and x0.
Substituting x= in the Taylor's series with the fact f()=0, we get
).(!2
)()()()()(
20
000 cfxx
xfxxxfxf
).(!2
)()()()(0
20
000 cfx
xfxxf
43
Newton-Raphson Theorem Continued…
If x0 is close enough to , then the last term on the right side will be small compared to the sum of the first two terms. Hence it can be neglected and we can use the approximation,
Solving for we get
).()()(0 000 xfxxf
).(/)( 000 xfxfx
44
Newton-Raphson Theorem Continued…
This is used to define the next approximation x1 to the root as
In general the Newton Method scheme is given by
,3,2,1,0for ,)(
)(1
n
xf
xfxx
n
nnn
.)(
)(
0
001 xf
xfxx
45
Newton's method continued … Example:
Find a real root of the equation f(x)=3x+sin x- ex
correct up-to 3-significant digits by Newton's method.
Solution: Here, f(x)=3x+sin x-exC2(). Hence, f'(x)=3+cos x-ex. The iterative scheme for Newton's Method is
Let x0=0. Thus, we have x1=0-(-1/3) = 0.33333
,2,1,0.cos3
sin3
)(
)(1
nex
exxx
xf
xfxx
n
n
xn
xnn
nn
nnn
46
Newton-Raphson Method
47
Convergence of Newton's Method:
In Newton's Method, fC2[a,b], f()=0 and f'()0, where [a,b]. Hence,
Where n, n lies between and xn .
1 1
( ) ( )
( ) ( )n n
n n n nn n
f x f xe x x e
f x f x
.)()(
)()2/()()(
)(
)( 2
nn
nnnn
n
nn fef
fefefe
ef
efe
48
Convergence of Newton's Method:
(by using the Binomial expansion)
.
)(
)(1)(
)()2/()( 2
f
fef
fefee
nn
nnnn
.)()(
)(1
)(
)()2/( 22
nnnn
nnn eOf
fe
f
feee
).()()(2
1
)(3
2
nnnn eOff
f
e
49
Convergence of Newton's Method:
As n, n and . Thus
Thus, Newton's method converges quadratically i.e. Order of convergence p=2.
.0|)(|2
|)(|
||
||lim
21
c
f
f
e
e
n
n
n
50
Advantages of Newton’ method
It is rapidly convergent in most cases since p=2.
It is simple in its formulation, and therefore relatively easy to apply and program.
It is intuitive in its construction. This means it is easier to understand its behavior, when it is likely to behave well and when it may behave poorly.
51
Disadvantages of Newton’ method:
It may not converge.
It is likely to have difficulty if f'(xn)=0. That means the X-axis is tangent to the graph of y=f(x) at x=xn.
The closer the derivative goes to zero, the worse Newton's Method behaves. Flat tangents may diverge the scheme.
This method can spin forever if there is no real root of the equation. (Example: x2 + 2 = 0).
It needs to know both f(x) and f'(x). Contrast this with the bisection method which requires only f(x).
52
Newton's Method for Multiple Roots:
A root is called a multiple root of order m, (m) if f(x) = (x- )m h(x) such that h() 0.
For m=1, is called simple root of f(x).
The Newton's method
converges quadratically, when is a simple root of f(x)=0, but it converges linearly when is a multiple root of f(x) = 0.
,)(
)(1
n
nnn xf
xfxx
53
Fixed Point Method (FPM) Continued…
Modified Newton's Method:
In the modified (accelerated) Newton's method, we take
if is a multiple root of order m of f(x)=0 and it converges quadratically.
,)(
)(1
n
nnn xf
xfmxx
54
Example:
Find the approximate value of the double root of x3 -2x2 - 0.75 x + 2.25 =0, that is correct up to 3 decimal places and close to 1 such that iterations converges quadratically. Take x0 =1.
Solution: Since the root of the equation is a double root, we will use modified Newton's method in order to get quadratic convergence.
55
The End of Chapter 2