Upload
punarbasu-roy
View
226
Download
2
Embed Size (px)
Citation preview
Introduction Order of Convergence Bisection Method Fixed-point Iterations Newton’s Method Secant Method
Error Analysis in Iterative MethodsUE201 : Seminar
Punarbasu RoySR Number : 10975
April 15, 2016
Introduction Order of Convergence Bisection Method Fixed-point Iterations Newton’s Method Secant Method
Outline
1 IntroductionLinear vs Non-linear equationsIterative Methods
2 Order of ConvergenceDefinitionExample
3 Bisection Method
4 Fixed-point Iterations
5 Newton’s Method
6 Secant Method
Introduction Order of Convergence Bisection Method Fixed-point Iterations Newton’s Method Secant Method
Linear vs Non-linear equations
Linear equations (for eg: 3x + 4y = 7; 8x + 9y = 2) can besolved by direct methods such as gaussian elimination.
Non linear equations (for eg: x = 2− e−2x , x = x3 + x2 + 1 )in general don’t have any direct method for solving.
They require iterative methods.
Introduction Order of Convergence Bisection Method Fixed-point Iterations Newton’s Method Secant Method
Table of Contents
1 IntroductionLinear vs Non-linear equationsIterative Methods
2 Order of ConvergenceDefinitionExample
3 Bisection Method
4 Fixed-point Iterations
5 Newton’s Method
6 Secant Method
Introduction Order of Convergence Bisection Method Fixed-point Iterations Newton’s Method Secant Method
Iterative Methods
Iterative methods generate a sequence of iterates which areapproximations to the solution.
The approximate solution is refined with each iteration anditerations are run unless certain order of accuracy has beenachieved.
Some of the iterative methods are
Bisection Method.
Fixed Point Method.
Newton’s Method.
Introduction Order of Convergence Bisection Method Fixed-point Iterations Newton’s Method Secant Method
Iterative Methods
Iterative methods generate a sequence of iterates which areapproximations to the solution.
The approximate solution is refined with each iteration anditerations are run unless certain order of accuracy has beenachieved.
Some of the iterative methods are
Bisection Method.
Fixed Point Method.
Newton’s Method.
Which is the best?
Introduction Order of Convergence Bisection Method Fixed-point Iterations Newton’s Method Secant Method
Things to consider:
Number of function evaluations and operations.
Number of iterations needed for reaching desired accuracy.
Introduction Order of Convergence Bisection Method Fixed-point Iterations Newton’s Method Secant Method
Things to consider:
Number of function evaluations and operations.
Number of iterations needed for reaching desired accuracy.
Introduction Order of Convergence Bisection Method Fixed-point Iterations Newton’s Method Secant Method
Table of Contents
1 IntroductionLinear vs Non-linear equationsIterative Methods
2 Order of ConvergenceDefinitionExample
3 Bisection Method
4 Fixed-point Iterations
5 Newton’s Method
6 Secant Method
Introduction Order of Convergence Bisection Method Fixed-point Iterations Newton’s Method Secant Method
Order of Convergence
Definition
Let {pn }∞n=0 be the sequence that converges to p with pn 6= p forall n. If positive constants λ and α exist such that,
limn→∞
|pn+1 − p||pn − p|α
= λ
then the sequence {pn }∞n=0 converges to p of order α withasymptotic error constant λ where α ≥ 1 and ∞ > λ > 0.
where error at kth iteration (ek) is |pk − p|In general the sequence with higher order of convergenceconverges more rapidly.
The asymptotic error constant also has a role in deciding thespeed of convergence of a sequence but not to the extent oforder.
Introduction Order of Convergence Bisection Method Fixed-point Iterations Newton’s Method Secant Method
When α = 1 , the sequence is linearly convergent.
When α = 2 , the sequence is quadratically convergent.
When α = 3 , the sequence is cubically convergent
and so on....
Introduction Order of Convergence Bisection Method Fixed-point Iterations Newton’s Method Secant Method
But we don’t know what the exact solution is.
So we can’t figure out what is ek (k = 0, 1, 2, 3, ...)
Instead we use relative change in successive iterates whichshould converge to zero.
Iterates are the approximation to x where f (x) = 0 so we cancheck whether f (xk) is converging to zero or not.
Introduction Order of Convergence Bisection Method Fixed-point Iterations Newton’s Method Secant Method
Table of Contents
1 IntroductionLinear vs Non-linear equationsIterative Methods
2 Order of ConvergenceDefinitionExample
3 Bisection Method
4 Fixed-point Iterations
5 Newton’s Method
6 Secant Method
Introduction Order of Convergence Bisection Method Fixed-point Iterations Newton’s Method Secant Method
Example
What will be the order of convergence of the sequence {xn }∞n=1
where xn = 1nk
and k > 1 ?
limn→∞
|xn+1 − x ||xn − x |α
Clearly the sequence converges to 0.
= limn→∞
|xn+1 − 0||xn − 0|α
= limn→∞
1(n+1)k
( 1nk
)α
= limn→∞
( nα
n+1)k
Introduction Order of Convergence Bisection Method Fixed-point Iterations Newton’s Method Secant Method
Example
What will be the order of convergence of the sequence {xn }∞n=1
where xn = 1nk
and k > 1 ?
limn→∞
|xn+1 − x ||xn − x |α
Clearly the sequence converges to 0.
= limn→∞
|xn+1 − 0||xn − 0|α
= limn→∞
1(n+1)k
( 1nk
)α
= limn→∞
( nα
n+1)k
Introduction Order of Convergence Bisection Method Fixed-point Iterations Newton’s Method Secant Method
Example
What will be the order of convergence of the sequence {xn }∞n=1
where xn = 1nk
and k > 1 ?
limn→∞
|xn+1 − x ||xn − x |α
Clearly the sequence converges to 0.
= limn→∞
|xn+1 − 0||xn − 0|α
= limn→∞
1(n+1)k
( 1nk
)α
= limn→∞
( nα
n+1)k
Introduction Order of Convergence Bisection Method Fixed-point Iterations Newton’s Method Secant Method
Example
What will be the order of convergence of the sequence {xn }∞n=1
where xn = 1nk
and k > 1 ?
limn→∞
|xn+1 − x ||xn − x |α
Clearly the sequence converges to 0.
= limn→∞
|xn+1 − 0||xn − 0|α
= limn→∞
1(n+1)k
( 1nk
)α
= limn→∞
( nα
n+1)k
Introduction Order of Convergence Bisection Method Fixed-point Iterations Newton’s Method Secant Method
Example
What will be the order of convergence of the sequence {xn }∞n=1
where xn = 1nk
and k > 1 ?
limn→∞
|xn+1 − x ||xn − x |α
Clearly the sequence converges to 0.
= limn→∞
|xn+1 − 0||xn − 0|α
= limn→∞
1(n+1)k
( 1nk
)α
= limn→∞
( nα
n+1)k
Introduction Order of Convergence Bisection Method Fixed-point Iterations Newton’s Method Secant Method
= limn→∞
( nα
n+1)k
For α = 1 limit exists. For α > 1 limit does not exist.
So the sequence has order of convergence = 1 that is thesequence converges linearly.
Introduction Order of Convergence Bisection Method Fixed-point Iterations Newton’s Method Secant Method
Table of Contents
1 IntroductionLinear vs Non-linear equationsIterative Methods
2 Order of ConvergenceDefinitionExample
3 Bisection Method
4 Fixed-point Iterations
5 Newton’s Method
6 Secant Method
Introduction Order of Convergence Bisection Method Fixed-point Iterations Newton’s Method Secant Method
Bisection Method
Theorem
If f ∈ C [a, b] and f (a).f (b) < 0 then Bisection method generatesa sequence {pn }∞n=0 approximating a zero p of f with
|pn − p| ≤ b − a
2nwhen n ≥ 1
When the initial interval is [a, b], the interval for kth iterationis [ak , bk ] where bk − ak = 2−k(b − a) .
So the upperbound of the error in each iteration is the intervallength of that step.
After each iteration the the interval length is reduced by halfand so is the upperbound of error.
Introduction Order of Convergence Bisection Method Fixed-point Iterations Newton’s Method Secant Method
So error in (n + 1)th and nth iteration can be written as
|en+1| = |pn+1 − p| ≤ 12(bn+1 − an+1)
|en| = |pn − p| ≤ 12(bn − an)
Dividing the (n + 1)th one by nth one we can get
|en+1||en|
≤ 12
here α = 1 and λ = 12
Hence the bisection method converges linearly withasymptotic error constant (λ) 1
2
Introduction Order of Convergence Bisection Method Fixed-point Iterations Newton’s Method Secant Method
Table of Contents
1 IntroductionLinear vs Non-linear equationsIterative Methods
2 Order of ConvergenceDefinitionExample
3 Bisection Method
4 Fixed-point Iterations
5 Newton’s Method
6 Secant Method
Introduction Order of Convergence Bisection Method Fixed-point Iterations Newton’s Method Secant Method
Rate of Convergence in Fixed Point Iterations
Theorem
(i) If g ∈ C [a, b] and g(x) ∈ [a, b] for all x ∈ [a, b] then g has atleast one fixed point in [a, b].(ii) If, in addition, g ′(x) exists on (a, b) and a positive constantk < 1 exists with |g ′(x)| ≤ k for all x ∈ (a, b) then there is exactlyone fixed point in [a, b]
We use the formula pn+1 = g(pn) to generate the iterates.
Error at nth iteration; en+1 = pn+1 − pen+1 = g(pn+1)− g(p) = g ′(ξn)(pn − p) = g ′(ξn)(en)where ξn lieas between pn and p. (Mean Value Theorem)
limn→∞
|en+1||en| = lim
n→∞|g ′(ξn)| = |g ′(p)|
Converges linearly (α = 1) with asymptotic error constant(λ)= |g ′(p)|
Introduction Order of Convergence Bisection Method Fixed-point Iterations Newton’s Method Secant Method
The order of convergence increases when extra conditions on g aremet. Assume g ′(p) = 0 and g ′′ exists on (a, b) .
en+1 = g(pn)− g(p) = g ′(p)(pn − p) + 12g′′(ξn)(pn − p)2
(Taylor Theorem)
en+1 = 12g′′(ξn)(en)2
Result : Quadratic convergence (α = 2) and λ = 12g′′(p)
This leads to a general result in the next theorem.
Introduction Order of Convergence Bisection Method Fixed-point Iterations Newton’s Method Secant Method
Theorem
Let g(x) be a function which is m times differentiable on aninterval [a, b]. If g(x) ∈ [a, b] for x ∈ [a, b] and that |g ′(x) ≤ k | on(a, b) for some k < 1. If the unique fixed point p in [a, b] satisfies
0 = g(p) = g ′(p) = g ′′(p) = ... = gm−1(p)
Then for any intial guess (x0) ∈ [a, b], the Fixed-point Iteration
converges to p with α = m and λ = |g(m)(p)
n!|
Introduction Order of Convergence Bisection Method Fixed-point Iterations Newton’s Method Secant Method
Table of Contents
1 IntroductionLinear vs Non-linear equationsIterative Methods
2 Order of ConvergenceDefinitionExample
3 Bisection Method
4 Fixed-point Iterations
5 Newton’s Method
6 Secant Method
Introduction Order of Convergence Bisection Method Fixed-point Iterations Newton’s Method Secant Method
Rate of Convergence in Newton’s Method
With initial guess p0 Newton’s Method generates sequence ofiterates approximating the exact solution p by the formula ,
pn = pn−1 −f (pn−1)
f ′(pn−1)
Error at n + 1th iteration is given by,
en+1 = pn+1 − p
= pn −f (pn)
f ′(pn)− p
= en −f (pn)
f ′(pn)
= en −1
f ′(pn)[f (p)− f ′(pn)(p − pn)− 1
2f ′′(ξk)(pk − p)2]
continued...
Introduction Order of Convergence Bisection Method Fixed-point Iterations Newton’s Method Secant Method
en+1 = en −1
f ′(pn)[f (p)− f ′(pn)(p − pn)− 1
2f ′′(ξn)(pn − p)2]
= en −1
f ′(pn)[−f ′(pn)en −
1
2f ′′(ξn)(en)2]
= en − en +f ′′(ξn)
2f ′(pn)e2n
en+1 =f ′′(ξn)
2f ′(pn)e2n
limn→∞
|en+1||en|2
= limn→∞
f ′′(ξn)
2f ′(pn)=
f ′′(p)
2f ′(p)
Result :
Newton’s Method converges quadratically (α = 2) and λ =f ′′(p)
2f ′(p)
Introduction Order of Convergence Bisection Method Fixed-point Iterations Newton’s Method Secant Method
Table of Contents
1 IntroductionLinear vs Non-linear equationsIterative Methods
2 Order of ConvergenceDefinitionExample
3 Bisection Method
4 Fixed-point Iterations
5 Newton’s Method
6 Secant Method
Introduction Order of Convergence Bisection Method Fixed-point Iterations Newton’s Method Secant Method
Rate of Convergence in Secant Method
In secant method, the formula which generates the sequence ofiterates approximating the solution is given by
pn = pn−1 −f (pn−1)(pn−1 − pn−2)
f (pn−1)− f (pn−2)
Which has order of convergence (α) = 1.618 1
1The derivation is complicated thus not showed here.
Introduction Order of Convergence Bisection Method Fixed-point Iterations Newton’s Method Secant Method
Comparison : Newton vs Secant
Theoritical derivations show that Newton’s method has higherorder of convergence than Secant method.
Newton’s method should take less number of iteration thansecant to converge to the exact solution.
Comparing speed of convergence of Newton’s method vs Secantmethod for the function f (x) = x + e−x
2cos(x)
MATLAB programme iterated both methods until relativeerror of order less than 10−6 was achieved.
logarithm of relative error vs number of iterations were plottedfor both methods.
Introduction Order of Convergence Bisection Method Fixed-point Iterations Newton’s Method Secant Method
Practically too, Newton’s method converges faster than SecantMethod (at the price of evaluating derivative of the function eachstep).
Introduction Order of Convergence Bisection Method Fixed-point Iterations Newton’s Method Secant Method
Thank You
Introduction Order of Convergence Bisection Method Fixed-point Iterations Newton’s Method Secant Method
Bibliography
Numerical Analysis 9th Edition Richard Burden J. D. FairesSection 2.4
banach.millersville.edu/ bob/math375/IterativeError/main.pdf
math.usm.edu/lambers/mat460/fall09/lecture12.pdf
macs.citadel.edu/chenm/343.dir/01.dir/lect2 4.pdf