Function Approximation - National Chiao Tungcaig.cs.nctu.edu.tw/course/NM07S/slides/chap4_1.pdf · Chebyshev series of ex ... Least Squares Approximation using Orthogonal Polynomials

Function Approximation

Pafnuty Chebyshev, 1821-1894

Numerical Methods © Wen-Chieh Lin 2

Administration

Assignment 3 is out; due on April 23 midnightChapter 3 Excises 30 (a)(b)(c)(d), 33, 48, 49,

69 (20% each) Extra credit (10%): App 8Mid-term exam: Apr 30


Function Approximation How does a computer approximate cos(x), exp(x),

and other non-polynomial functions? Efficient and accurate approximation is desired

Chebyshev polynomialsOrthogonal polynomialsConverge faster than a Taylor series

Rational functionsRatio of two polynomials

Fourier seriesSeries of sine and cosine termsCan approximate periodic or discontinuous functions


Truncation Error in Taylor Series Recall that Taylor series

If approximating a function using first n terms

Error grows rapidly as x-value departs from x = a!

2)(!2

)("))((')()( ax

afaxafafxf

1)1()(

)()!1(

)()(

!)(

)(

nn

nn

axn

fax

naf

af

1)1(

)()!1(

)(

nn

axn

fError


Chebyshev PolynomialsRecursive definition

xxTxTxTxxTxT nnn )(,1)(),()(2)( 1011

12)( 22 xxT

xxxT 34)( 33

188)( 244 xxxT


Chebyshev Polynomials (cont.)

Equivalent form

Proof

))acos(cos()( xnxTn )cos(x

xxTxTxTxxTxT nnn )(,1)(),()(2)( 1011

coscos2)1cos()1cos( nnn

)(2)()( 11 xxTxTxT nnn


Chebyshev Series

12)( 22 xxT

xxxT 34)( 33

188)( 244 xxxT

1)(0 xT

xxT )(1

)(21

022 TTx

)3(41

133 TTx

)34(81

0244 TTTx

1Tx

01 T


Chebyshev Series is More Economical

On the interval [-1, 1], approximating afunction with a series of Chebyshevpolynomials is better than with a Tayor seriesas it has a smaller maximum error

On an interval [a, b], we have to transform thefunction to translate the x-value into [-1, 1]

12~

abax

x)~()( xfxf


)34(81

0244 TTTx

Example: Economizing a Power SeriesMaclaurin series is Taylor series expansion of

a function about 0Maclaurin series of ex

Approximated by a Chebyshev series

7201202462

165432 xxxxx

xex

)34(81

241

)3(41

61

)(21

21

024130210 TTTTTTTTTex

)(21

022 TTx

)3(41

133 TTx

)1510(23040

1)510(

161

1201

2031 TTTT


Example (cont.)

Maclaurin series of ex truncated after 3rddegree

Chebyshev series of ex truncated after 3rddegree

621

32 xxx

3210 0443.02715.01302.12661.1 TTTTex

32 1772.05430.09973.09946.0 xxx

7201202462

165432 xxxxx

xex


Example (cont.) Error of Maclaurin series grows rapidly when

x-value departs from zero

Error of Maclaurin series

Error of Chebyshev series

x


Orthogonaliy of Chebyshev Polynomials

Two polynomials p(x), q(x) are orthogonal onan interval [a, b] if their inner product

Set of polynomials is orthogonal if eachpolynomial is orthogonal to each other

Chebyshev Polynomials are orthogonal oninterval [-1, 1] with

0)(,0)()()(, xwdxxwxqxpqpb

a

211)( xxw


Verify Orthogonaliy ofChebyshev Polynomials

Example: p(x) = T0(x), q(x) = T3(x)

Matlab code int('(4*x^3-x)/sqrt(1-x^2)', -1 ,1)

01

1)34(1)(),(

2

1

1

3

dxx

xxxqxp


Least Squares Approximation usingOrthogonal Polynomials

System of normal equations for a high-degreepolynomial fit is ill-conditioned

Fitting data with orthogonal polynomials suchas Chebyshev polynomials can reduce thecondition number to about 5

)()()()( 1100 xTaxTaxTaxf nn

mnmnmm

n

n

y

yy

a

aa

xTxTxT

xTxTxTxTxTxT

2

1

1

0

10

22120

11110

)()()(

)()()()()()(


Rational Function Approximation

Approximating a function with a Chebyshevseries has smaller maximum error than with aTaylor series

We can further improve the maximum errorusing rational function approximation

Padé approximation: n ≥m

mm

nn

N xbxbxbxaxaxaa

xRxf

221

2210

1)()(

1 mnN


Padé ApproximationRepresent a function in a Taylor series

expanded at x = 0Coefficients are determined by setting

and equating coefficients

)()( xRxf N

mm

nnN

N xbxbxbxaxaxaa

xcxcxcc

221

22102

210 1)(

!)0()(

if

ci

i

00 ca

1011 ccba

nnmnmn ccbcba 11011 mNmNN cbcbc

0111 mnmnn cbcbc


Example: Padé Approximation

Find arctan(x) ≈R10(x)Maclaurin series of arctan(x) through x10 is

f(x) = R10(x)

9753

91

71

51

31

)arctan( xxxxxx

55

44

33

221

55

44

33

22109753

191

71

51

31

xbxbxbxbxbxaxaxaxaxaa

xxxxx


Example: Padé Approximation (cont.) Equating coefficients

x0 through x5

55

44

33

221

55

44

33

22109753

191

71

51

31

xbxbxbxbxbxaxaxaxaxaa

xxxxx

00 a

11 a

12 ba

231

3 ba

3131

4 bba

4231

51

5 bba

0331

151 bb

0431

251

71 bb

0531

351

171 bbb

0451

271

91 bb

x6 through x10

0551

371

191 bbb


Padé Approximation vs. Maclaurin Series

Padé Approximation

Maclaurin series9753

91

71

51

31

)arctan( xxxxxx

42

53

215

910

1

94564

97

)arctan(xx

xxxx


Padé Approximation vs. Maclaurin Series9753

91

71

51

31

)arctan( xxxxxx

42

53

215

910

1

94564

97

)arctan(xx

xxxx

Maclaurin series

Pade approximation

True value

Pade approximation


Chebyshev-Padé Rational FunctionWe can get a rational function approximation

from a Chebyshev series expansion of afunction as well

Example:

We first form

Then expand the numerator and set thecoefficients of each degree of the T’s to zero

3210 0443.02715.01302.12661.1 TTTTex

11

221103210 1

0443.02715.01302.12661.1Tb

TaTaaTTTT


])cos()[cos(21

)cos()cos( mnmnmn

11

221103210 1

0443.02715.01302.12661.1Tb

TaTaaTTTT

Recall that

Apply the last equation to

The numerator becomes

)cos()( nxTn

)]()([21

)()( xTxTxTxT mnmnmn

)(2

2661.10443.02715.01302.12661.1 11

13210 TT

bTTTT

)(2

0443.0)(

22715.0

)(2

1302.124

113

102

1 TTb

TTb

TTb

22110 TaTaa


Equating the Coefficients of Each Degreeof T’s in Numerator

)(2

2661.10443.02715.01302.12661.1 11

13210 TT

bTTTT

)(2

0443.0)(

22715.0

)(2

1302.124

113

102

1 TTb

TTb

TTb

22110 TaTaa

21302.1

2661.1 10

ba 2

2715.02661.11302.1 1

11b

ba

20443.0

21302.1

2715.0 112

bba

22715.0

0443.00 1b

xxx

TTT

ex

3263.011598.06727.00018.1

3263.010799.06727.00817.1 2

1

21


Chebyshev vs. Chebyshev-Padé Rational


Error Plot

Pade-Chebyshevrational function

Chebyshev serires


Fourier SeriesRepresenting a function as a trigonometric

series

f(x) need be integrable so that the coefficientscan be computed

1

0 )sin()cos(2

)(n

nn nxBnxAA

xf


Properties of Orthogonality

0)sin( dxnx

0,20,0

)cos(nn

dxnx

0)cos()sin( dxmxnx

mnmn

dxmxnx,

,0)sin()sin(

mnmn

dxmxnx,

,0)cos()cos(


....2,1,)()cos(

1mdxxfmxAm

1

0 )cos()cos()cos(2

)()cos(n

n dxmxnxAdxmxA

dxxfmx

Computing Fourier Coefficientsusing Orthogonality

1

0 )sin()cos(2

)(n

nn nxBnxAA

xf

11

0 )sin()cos(2

)(n

nn

n dxnxBdxnxAdxA

dxxf

000 A

1

)cos()sin(n

n dxmxnxB

00 mA


Computing Fourier Coefficientsusing Orthogonality (cont.)

1

0 )sin()cos(2

)(n

nn nxBnxAA

xf

1

0 )sin()cos()sin(2

)()sin(n

n dxmxnxAdxmxA

dxxfmx

1

)sin()sin(n

n dxmxnxB

00 mB

....2,1,)sin()(

1mdxmxxfBm


Fourier Series for Periods other than 2π

Let the period of f(x) be PConsider that f(x) is periodic in [-P/2, P/2]Change of variable

2

2....2,1,)

2cos()(

2 P

Pm mdxP

xmxf

PA

2Px

y

2

2....2,1,)

2sin()(

2 P

Pm mdxP

xmxf

PB

....2,1,)()cos(

1mdxxfmxAm

....2,1,)sin()(

1mdxmxxfBm


Fourier Series for Periods other than 2π

1

0 )2

sin()2

cos(2

)(n

nn Pxn

BP

xnA

Axf

2

2....2,1,)

2cos()(

2 P

Pn ndxP

xnxf

PA

2

2....2,1,)

2sin()(

2 P

Pn ndxP

xnxf

PB


Example: f(x) = x, x in [-π,π]

0)sin()sin(1

)cos(1

dx

mmx

mmx

xxdxmx

....2,1,)()cos(

1mdxxfmxAm

....2,1,)sin()(

1mdxmxxfBm

dxm

mxm

mxxxdxmx

)cos()cos(1)sin(

1

mmm m 1)1(2)cos(2

1

1

)sin()1(2

m

m

mxm

x


Fourier Series for Nonperiodic Functions

Chop the interval of interest [0, L]

L



Solution 1: make an even function byreflecting the segment about y-axis, f(x)=f(-x)

L



Solution 2: make an odd function by reflectingthe segment about the origin, f(x) = -f(x)

L


Properties of Even and Odd Functions

Product of two even functions is an even function Product of two odd functions is an even function Product of an even function and an odd

function is an odd function

LL

Ldxxfdxxfxf

0)(2)(evenis)(if

L

Ldxxfxf 0)(oddis)(if


Fourier Series for Even Functions

2

2....2,1,)

2cos()(

2 P

Pm mdxP

xmxf

PA

L

Ldx

Lxm

xfL

)2

2cos()(

22

L

dxL

xmxf

L 0)cos()(2

1

2

2....2,1,)

2sin()(

2 P

Pm mdxP

xmxf

PB

0 odd function

even function


Fourier Series for Odd Functions

2

2....2,1,)

2cos()(

2 P

Pm mdxP

xmxf

PA

L

Ldx

Lxm

xfL

)2

2sin()(

22

L

dxL

xmxf

L 0)sin()(2

1

2

2....2,1,)

2sin()(

2 P

Pm mdxP

xmxf

PB

0 odd function

even function


Example: Fourier Series of NonperiodicFunctions—even extension

21,110,0

)(xx

xf

L

m dxL

xmxf

LA

0)cos()(

2

odd,)1(

2

even,02)1(

mm

mm

2

0)

2cos()(

22

dxxm

xf

1)(22 2

00 dxxfA

1 2-1-2x

y

1

1

0 )2

sin()2

cos(2

)(n

nn Pxn

BP

xnA

Axf


Example: Fourier Series of NonperiodicFunctions—odd extension

21,110,0

)(xx

xf

L

m dxL

xmxf

LB

0)sin()(

2

)

2cos()cos(

2

mm

m2

0)

2sin()(

22

dxxm

xf

1 2-1-2x

y

1

-1

)2

sin()

2cos()cos(2

1

xnn

nn

n

1

0 )2

sin()2

cos(2

)(n

nn Pxn

BP

xnA

Axf

Documents

Function Approximation - National Chiao Tungcaig.cs.nctu.edu.tw/course/NM07S/slides/chap4_1.pdf · Chebyshev series of ex ... Least Squares Approximation using Orthogonal Polynomials