Polynomial basis functions - faculty.jacobs-university.de file120202: ESM4A - Numerical Methods 425 Visualization and Computer Graphics Lab Jacobs University Polynomial basis functions

120202: ESM4A - Numerical Methods 425

Visualization and Computer Graphics LabJacobs University

Polynomial basis functions• We can use any polynomial basis that we had seen for

interpolation purposes. • A good basis is one, where the basis functions differ a lot. • Obviously, monomials are not the best choice.• A good choice are Chebychev polynomials over [-1,1]:

• E.g.:



Polynomial regression

• Frequently, polynomial approximations give very good results, if the degree of the polynomial is known.

• Obviously, increasing the degree allows for a betterfit, but also becomes more sensitive to noise error.

• If we increase the degree until we reach degree m for m+1 measurements, we actually obtain polynomialinterpolation, again.

• How can we determine an appropriate degree forpolynomial approximation, if it is unknown?

• Statistics can help us.




• Let be the polynomial thatrepresents the measurements without noise.

• The measured values can be described as with noise ε.

• Then, the variance can be computed as

• From statistical theory, one knows that




This observation leads to the following strategy:• Compute variances for polynomials of increasing

degree starting with σ02.

• If we reach the degree N, where σN2 ≈ σN+1

2, we havefound the desired polynomial pN(x).




Algorithm:• Given: m+1 measurements.• For N=0,…,m:

Determine pN(x) using the least-squares method.Compute variance σN

2.If (σN

2 ≈ σN-12 )

Return pN-1(x).



Summary• If measurements are due to noise, approximation is to be

preferred over interpolation.• If the nature of the underlying function is known, the

least squares method can be used to find the optimal parameters to minimize the total squared error.

• Even if the functions are highly non-linear, the least-squares method leads to solving a linear equation system, i.e, the normal equations.

• Least-squares method can also be used to find best solutions to overdetermined systems of linear equations.

• If the nature of the underlying function is unknown, onecan use a polynomial basis and find the appropriatedegree by applying polynomial regression.



5.9 Simpson‘s Scheme



Observation• Consider the linear interpolant

of function f over interval [a,b]. • We determine its definite integral over [a,b] by

• Hence, the trapezoid rule is the integral of the linear interpolant.



Idea• Use quadratic interpolant instead of linear one, i.e.,

with m = (a+b)/2 using Lagrange interpolation.• Performing the derivation for the integral over the

quadratic interpolant over two intervals of equal size, which is analogous to the derivation on the previousslide (exercise!), we obtain:

• This is called Simpson‘s scheme or Simpson‘s rule(after Thomas Simpson).



Remark

• Applying the recursive trapezoid rule to evaluate theintegral using two intervals of equal length, we obtain:

• This is similar to Simpson‘s rule

• Simpson‘s rule puts more emphasis on the middlepoint.



Error theorem

• Simpson‘ scheme has error

for some ,i.e., the error term is O(h5) with h=b-a.

• Proof: Exercise!It can be shown by deriving the Simpson estimateusing Taylor expansions for f(a+h) and f(a+2h).



Recursive Simpson‘s rule

• To compute the estimate for , one can applySimpson‘s scheme to

and

and add the resulting terms.• Of course, this process can be iterated.• When shall we stop?




• Starting with

we derive

new Simpson‘s estimate S2 new error term E2

Simpson‘s estimate S1 error term E1




• Hence,

and

with

stop iterationwhen this error term gets small




Simpson (f,a,b,ε):// Input: function f, interval [a,b], and maximum error ε.

compute S1 = S(a,b)compute S2 = S(a,(a+b)/2) + S((a+b)/2,b)if (| S1 - S2 | < 15 ε)

return (S2 + (S2 - S1)/15)else

return (Simpson (f,a,(a+b)/2,ε/2) + Simpson (f,(a+b)/2,b,ε/2))



Composite Simpson‘s rule

• Generalizing Simpson‘s rule to any even number n of intervals, one gets the estimate

with .

• This is the composite Simpson‘s rule, which computesthe same as the recursive scheme without using therecursion.



Error term

• For the composite Simpson‘s rule, we get the errorterm

for some .• We observe that one factor (b-a) in the error term

for Simpson‘s scheme is actually describing theinterval.

• Hence, the actual error is O(h4) using the compositeSimpson‘s rule.



Example

• Compute using an equidistant partition

with n = 5. • For we compute the function values

• For the composite trapezoid rule, we obtained

• The real value was



Example

• As n is odd, we cannot apply the composite Simpson‘s rule.• Instead, we use n = 4.• We evaluate f(0.25) = 0.98962, f(0.5) = 0.95885, and

f(0.75) = 0.90885.• Using the composite Simpson‘s rule, we obtain

• Hence, even when using less samples (n=4 instead of 5), the composite Simpson‘s rule is producing better results.



Remark

• The error term using the composite Simpson‘s rulehas better behavior that the one using the compositetrapezoid rule.

• However, both schemes apply about the samecomputations (just weighted differently).

• As we did in the Romberg algorithm for the compositetrapezoid rule, one can also apply Richardsenextrapolation to the composite Simpson‘s rule to obtain even better error terms.



5.10 Gaussian Quadrature



Observation & Motivation• Most numerical integration schemes follow the

pattern

with a ≤ x0 < x1 < … < xn ≤ b.• We have seen that choosing the weights Ai, i = 0,…, n,

appropriately matters.• For equidistant knots xi, the Simpson weights produce

asymptotically better results than the trapezoidweights.

• Question: How can we find the best weights Ai forgiven knots xi, i = 0,…, n?



Idea

• Approximate f(x) by a polynomial p(x) of degree n that interpolates the points f(x0),…,f(xn) at the knotsx0,…,xn.

• Approach: Use Lagrangian interpolation.• Hence,

with



Approach

• If f(x) ≈ p(x) for x є [a,b], we obtain

with



Remark

• The weights Ai only depend on the knot sequencex0,…,xn, but are independent of the function f(x) wewant to integrate.

• Hence, for a given knot sequence, the Ai can beprecomputed and stored.

• During integration, the Ai are multiplied with thefunction values f(xi) and the products are summed up.



Remark

• This is a generalization of the trapezoid rule and theSimpson‘s rule, as we had seen that they can bederived as the integral over the linear and quadraticpolynomials, respectively, (also using Lagrangeinterpolation).



Example

• Let [a,b] = [-2,2] and (x0,x1,x2) = (-1,0,1).



Example

• For any function f(x), we obtain the estimate for itsintegral over [-2,2] as

• The weights 8/3, -4/3, and 8/3 are the optimal weights for knot sequence (-1,0,1) with respect to theinterpolation model.

• For f(x) = 1, we get

• For f(x) = x, we get



Remark

• If f is a polynomial of degree ≤ n, we have f(x) = p(x).• Consequently, the integration scheme is exact.• This is true for any knot sequence with n+1 knots.• How about if f is a polynomial of degree > n?



Gaussian quadrature

• Up to now, we assumed that the knots x0,…,xn aregiven.

• They can actually be arbitrary.• For practical reasons, it makes sense to postulate

that they belong to interval [a,b].• Karl Friedrich Gauß (1777-1855) discovered that wise

placement of the knots can significantly improve theaccuracy of the numerical integration scheme.

• He formulated the choice of the special placement in the Gaussian quadrature theorem, where quadratureis another term for integration.



Gaussian quadrature theorem

• Let q be a non-trivial polynomial of degree n+1 such that

(*)for k=0,…,n.

• Let x0,…,xn be the roots of polynomial q.• Then,

with

for all polynomials f of degree ≤ 2n+1.



Proof• Let f(x) be a polynomial of degree ≤ 2n+1.• Then, we can write f(x) = s(x) • q(x) + r(x)

for polynomials r(x) and s(x) of degree ≤ n.• Hence,

= 0 because of (*)

polynomial of degree ≤ n

= 0



Remark

• The n+1 equations (*) do not define the polynomialq(x) uniquely, as q(x) is of degree n+1, i.e., it has n+2 coefficients in a monomial representation.



Gaussian quadrature

• Following the Gaussian quadrature theorem, we fix the n+1 knots x0,…,xn.

• The knots are also referred to as Gaussian nodes.• The knots uniquely determine the polynomial p(x) that

is supposed to approximate function f(x).• Having determined the knots, we use Lagrangian

interpolation to obtain the weights A0,…,An.• Altogether, we have chosen 2(n+1) variables to find an

optimal estimate for the integral (when using n+1 knots).

• This integration scheme is called Gaussianquadrature.



Remark

• We had figured out that the choice of weightsA0,…,An does only depend on the knots x0,…,xn.

• Using Gaussian quadrature, also the choice of theknots x0,…,xn is independent of the function f(x) wewant to integrate.

• Both knots x0,…,xn and weights A0,…,An aredetermined by the degree n of polynomial p(x) and the integration interval [a,b].

• Hence, all 2(n+1) parameters x0,…,xn and A0,…,An canbe precomputed independently of function f(x).



Example

• Use Gaussian quadrature to determine the value of

using 3 knots and weights.• The degree of p(x) is n=2.• The degree of q(x) is n+1=3.• Hence, q(x) is of the form q(x) = c0 + c1 x + c2 x2 + c3 x3.• From (*), we derive the 3 conditions



Example

• We want to determine coefficients c0, c1, c2, and c3 such that the 3 conditions are met.

• Choosing c0 = c2 = 0, we are left with the odd functionq(x) = c1 x + c3 x3 and fulfill conditions

as the integral of an odd function over an intervalthat is symmetric with respect to the origin is 0.



Example• We still need to meet condition

• Thus, we can choose c1 = -3 and c3 = 5.• We obtain the polynomial

• The roots of q(x) are

• These are the sought knots x0, x1, and x2.



Example

• Having determined the knots, we can fix the weightsA0, A1, and A2 in

• The weights can be computed by integration of theLagrange polynomials as in the preceding example.

• However, we can also determine them easier.• We can make use of the fact the equation above is

exact for any polynomial function f of degree ≤ 2.• Therefore, we evaluate the equation for f(x) = 1, f(x)

= x, and f(x) = x2.



Example

• We derive the 3 linear equations

• The system has a unique solution, as f(x) = 1, f(x) = x, and f(x) = x2 are linearly independent.



Example

• Altogether, we obtain the Gaussian quadrature formula

• This estimate is exact for all polynomials of degree ≤ 5.

• For example, if we pick f(x) = x4, we get the exactestimate of the integral as



Interval transformation

• The only input to computing Gaussian nodes and weights was the interval [-1,1] and the number of Gaussian nodes.

• Question: Can we use this result to compute integralsover a different interval?

• Can we use the result to compute the integral overany interval [a,b]?




• Let x є [a,b].• Then, we use the linear transformation

to obtain thatє [a,b]

for t є [-1,1].• Using the transformation, we can use the

transformed Gaussian nodes and weights of theintegral over interval [-1,1] to compute the integral over [a,b].




• We get

• Hence, the Gaussian nodes are

and the respective weights are



Example

• Let‘s compute

using 3 Gaussian nodes, i.e., the approximatingpolynomial p(x) is of degree 2.

• The transformation is given by



Example

• We obtain

• The real value of the definite integral amounts to



Polynomial q(x)

• We have seen that polynomial q(x) was of degree n+1, where n was the degree of the approximatingpolynomial n.

• From (*), we derive n+1 conditions, which left us withone degree of freedom for choosing polynomial q(x).

• If we “normalize“ polynomial q(x) by postulating thatq(1) = 1, we obtain a unique representation of q(x).

• Considering the interval [-1,1], the resultingpolynomials are called Legendre polynomials.

• They are named after Adrien-Marie Legendre (1752-1833).



Legendre polynomials

• The first Legendre polynomials qn(x) of degree n aregiven by

• They obey the recursion formula

Documents

Polynomial basis functions - faculty.jacobs-university.de file120202: ESM4A - Numerical Methods 425 Visualization and Computer Graphics Lab Jacobs University Polynomial basis functions