Upload
vishnuvardhan-ranganathan
View
218
Download
0
Tags:
Embed Size (px)
DESCRIPTION
math
Citation preview
MA/CS 375Fall 2002Lecture 31
MA/CS375 Spring 2002
Root FindingGiven a real valued function f of one variable (say x), the idea is to find an x such that:
f(x) = 0
MA/CS375 Spring 2002
Newtons Method cont.Repeat the formula to create an algorithm:
If at each step the linear model is a good approximation to f then xn should get closer to a root of f as n increases.
MA/CS375 Spring 2002
Newton in Matlab
MA/CS375 Spring 2002
Newtons Method Without Knowing the DerivativeRecall: we can approximate the derivative to a function with:
MA/CS375 Spring 2002
Modification
MA/CS375 Spring 2002
Team Exercise10 minutesModify your script to use the approximate derivative (note you will require an extra parameter delta)Test it with some function you do not know the derivative of.
MA/CS375 Spring 2002
Convergence Rate For Newtons MethodTheorem 8 (van Loan p 285)Suppose f(x) and f(x) are defined on an interval where and positive constants rho and delta exist such that
If xc is in I, then is in I and
That is x+ is at least half the distance to x* that xc was. Also, the rate of convergence is quadratic.
MA/CS375 Spring 2002
Convergence Rate of Newtons Method contThe proof of this theorem works by using the fundamental theorem of calculus. All of the restrictions are important and can be fairly easily broken by a general functionThe restrictions amount to:f does not change sign in a neighbourhood of the root x*f is not too non-linear (Lipschitz condition)the Newtons iteration starts close enough to the root x* then convergence is guaranteed and the convergence rate is quadratic.
MA/CS375 Spring 2002
Finding A Root Of A Two-dimensional Function of Two VariablesSuppose:
Specifically:
MA/CS375 Spring 2002
Finding A Root Of A Two-dimensional Function of Two VariablesSuppose:
Specifically:
We can construct a Taylor series:
MA/CS375 Spring 2002
Constructing A Newton SchemeCreate a sequence by using this linear approximation to update from (xn,yn) to (xn+1,yn+1)
MA/CS375 Spring 2002
Constructing A Newton SchemeWe figure that if the linear approximation is good then the f(xn+1,yn+1) and g(xn+1,yn+1) should be small
MA/CS375 Spring 2002
Two-dimensional Newton Method
MA/CS375 Spring 2002
Jacobian Matrix
MA/CS375 Spring 2002
Jacobian Matrix
MA/CS375 Spring 2002
Team Exercise (Part 1)Code up the two-dimensional Newton solver. Use the approximation for small delta:
Use it to find x,y such that:
MA/CS375 Spring 2002
Team Exercise (Part 2)Generalize your Newton Solver to solve:
Construct a 3-vector function and find its roots.
Plot the position of (xn,yn,zn) using sphere
MA/CS375 Spring 2002
Problems with Multi-D NewtonThe method relies on inverting the Jacobian matrix.
Recall that matrix inversion is strongly dependent on the condition number of the matrix.
Clearly, if the root is near a region where the gradient is small then we will run into slow convergence when the search nears the root.
MA/CS375 Spring 2002
Team Exercise:Dodgy ConvergenceTry to find the (0,0) root of
Compare the rate of convergence for the (0,0) root of: For each iteration plot the condition number of the Jacobian matrix. Try several different starting positions.
MA/CS375 Spring 2002
Roots of a PolynomialSuppose we wish to find all the roots of a polynomial of order P
Then there are going to be at most P roots!.
We can use a variant of Newtons method.
MA/CS375 Spring 2002
Roots of a Polynomial cont.Suppose we have an initial guess for one of the roots of the polynomial function f Then we can use Newtons method, starting at this guess to solve for f(x)=0 Once we have found the first root x1 we apply polynomial deflation to remove this root and then repeat the process to find the next root.
MA/CS375 Spring 2002
Algorithm1) We are seeking the roots x1,x2,..xP of a polynomial f 2) We find x1 using Newtons method.3) We then use Newtons method to find the next root of f(x)/(x-x1)4) Then we find x3 as a root of f(x)/((x-x1)(x-x2))5) Repeat until all roots found
MA/CS375 Spring 2002
Details of Newtons AlgorithmAt the kth step, we need to find a root of For Newtons we need:
MA/CS375 Spring 2002
Product rule fordifferentiation
MA/CS375 Spring 2002
MA/CS375 Spring 2002
Newton Scheme For Multiple Root Finding
MA/CS375 Spring 2002
Multiple Root Finder(applied to find roots of Legendre polynomials)
MA/CS375 Spring 2002
Recall: recurrence relation of Legendre polynomials
MA/CS375 Spring 2002
Recall: Recurrence Relation for Gradient of Legendre Polynomials
MA/CS375 Spring 2002
Roots of the 10th Order Legendre PolynomialNotice how they cluster at the end points
MA/CS375 Spring 2002
Numerical QuadratureA numerical quadrature is a set of two vectors. The first vector is a list of x-coordinates for nodes where a function is to be evaluated. The second vector is a set of integration weights, used to calculate the integral of a function which is given at the nodes
MA/CS375 Spring 2002
Example of QuadratureSay we wish to calculate an approximation to the integral of f over [-1,1] : Suppose we know the value of f at a set of N points then we would like to find a set of weights w1,w2,..,wN so that:
MA/CS375 Spring 2002
Newton-Cotes FormulaThe first approach we are going to use is the well known Newton-Cotes quadrature.
Suppose we are given a set of points x1,x2,..,xN. Then we require that the constant is exactly integrated:
MA/CS375 Spring 2002
Now we require that 1,x,x2,..,xN-1 are integrated exactly
MA/CS375 Spring 2002
In Matrix Notation: Notice anything familiar?
MA/CS375 Spring 2002
Its the transpose of the Vandermonde matrix
MA/CS375 Spring 2002
Integration by InterpolationIn essence this approach uses the unique (N-1)th order interpolating polynomial If and integrates the area under the If instead of the area under f Clearly, we can estimate the approximation error using the estimates for the error in the interpolation we used before.
MA/CS375 Spring 2002
Newton-Cotes Weights
MA/CS375 Spring 2002
Using Newton-Cotes Weights
MA/CS375 Spring 2002
Using Newton-Cotes Weights(Interpretation)i.e. we calculate the coefficients of the interpolating polynomial expansion using the Vandermonde, then since we know the integral of each term we can sum up the integral of each term to get the total.
MA/CS375 Spring 2002
Matlab Function for Calculating Newton-Cotes Weights
MA/CS375 Spring 2002
Demo: Matlab Function for Calculating Newton-Cotes Weightsset N=5 pointsbuild equispaced nodescalculate NC weights
evaluate F=X^3 at nodesevaluate integral
F is anti-symmetric on [-1,1] so its integral is 0Answer correct
MA/CS375 Spring 2002
Team ExerciseGet the directory Lecture19m from the cd-rom make sure your matlab path points to the copy of this directory using a script figure out what order polynomial the weights can exactly integrate for a given set of N points (say N=6).
MA/CS375 Spring 2002
Gauss QuadratureThe construction of the Newton-Cotes weights does not utilize the ability to choose the distribution of nodes for greater accuracy. We can in fact choose the set of nodes to increase the order of polynomial that can be integrated exactly using just N points.
MA/CS375 Spring 2002
Suppose:
MA/CS375 Spring 2002
Suppose:Remainder term, which must have p roots located at the interpolating nodes
MA/CS375 Spring 2002
At this point we can choose the nodes {xi}. If we choose them so that they are the p+1 roots of the (p+1)th order Legendre function then s(x) is in fact the N=(p+1)th order Legendre function itself!.
MA/CS375 Spring 2002
But we also know that if r is a lower order polynomial than (p+1)th order, it can be expressed as a linear combination of Legendre polynomials and is in fact orthogonal to Lp+1
MA/CS375 Spring 2002
i.e. the quadrature is exact for all polynomials of order up to p=(2N-1)Hence:
MA/CS375 Spring 2002
Summary of Gauss QuadratureWe can use the multiple root finder to locate the roots of the Nth order Legendre polynomial. We can then use the Newton-Cotes formula with the roots of the Nth order Legendre polynomial to calculate a set of N weights.
We now have a quadrature !!! which will integrate polynomials of order 2N-1 with N points
MA/CS375 Spring 2002
Team ExerciseUse the root finder and Newton-Cotes routines to build a quadrature for N points (N arbitrary).
Test it on some functions you know the integral of (sin(x) or tan(x) or exp(x) or ) over the interval [-1,1]
MA/CS375 Spring 2002
SummaryWe have looked at two ways to find the root of a single valued, single parameter function We considered a robust, but slow bisection method and then a faster but less robust Newtons method which we generalized to vector functions of multiple arguments
We discussed the theory of convergence for Newtons method.
MA/CS375 Spring 2002