EEGN403 – COMPUTER METHODS IN NUMERICAL ANALYSIS

CSUF

TEXTBOOK Numerical method for engineers

2

OUTLINE Motivation What Are Numerical Methods Why You Need to Learn Numerical Methods Course Topics Tools You Should Know History

Milestone Algorithms Top 10 Algorithms

Mathematical PreliminariesDerivatives Partial Derivatives 3

MATHEMATICAL PRELIMINARIES

Tangents and Gradients Functions as curves Review of Functions The Class of Polynomials Taylor’s Series Mean-Value Theorem Rolles Theorem Caveat

4

PRE-COMPUTER ERA computer era

Motivation

5

What Are Numerical Methods?

Techniques by which mathematical problems are formulated so that they can be solved with arithmetic operations {+,-,*,/} that can then be performed by a computer.

Why You Need to Learn Numerical Methods?

1. Numerical methods are extremely powerful problem-solving tools.

2. During your career, you may often need to use commercial computer programs (canned programs) that involve numerical methods. You need to know the basic theory of numerical methods in order to be a better user.

3. You will often encounter problems that cannot be solved by existing canned programs; you must write your own program of numerical methods.

4. Numerical methods are an efficient vehicle for learning to use computers.

5. Numerical methods provide a good opportunity for you to reinforce your understanding of mathematics.

You need that in your life as an engineer or a scientist.

COURSE TOPICS Mathematical Preliminaries Computer Representation of real numbers Algorithms for finding the roots of a function,

i.e., where f(x,y,z) = 0 Functional approximation via interpolation Numerical differentiation of functions.

8

COURSE TOPICS Numerical integration of functions

also termed quadrature Monte-Carlo and other randomized

techniques Numerical solution of linear systems

Motivation for algorithms Examples from EE related problems

9

TOOLS YOU SHOULD KNOW One of computer programs Matlab – a system for performing many

numerical tasks.

10

HISTORY Numerical analysis can be traced back to a

symposium with the title ``Problems for the Numerical Analysis of the Future, UCLA, July 29-31, 1948.

Volume 15 in Applied Mathematics Series, National Bureau of Standards

Boom of research and applications: Fluid flow, weather prediction, semiconductor, physics, ……

11

MILESTONE ALGORITHMS 1901: Runge-Kutta methods for ODEs 1903: Cholesky decomposition 1926: Aitken acceleration process

1946: Monte Carlo method 1947: The simplex algorithm 1955: Romberg method 1956: The finite element method

1limif2

)( 1

12

21

aSSSS

SSSSSST

n

n

nnnn

nnnn

12

MILESTONE ALGORITHMS1957: The Fortran optimizing compiler 1959: QR algorithm1960: Multigrid method1965: Fast Fourier transform (FFT)1969: Fast matrix manipulations1976: High Performance computing &

packages: LAPACK, LINPACK – Matlab1982: Wavelets1982: Fast Multipole method

13

TOP 10 ALGORITHMS

1946: Monte Carlo method 1947: Simplex method for linear programming 1950: Krylov subspace iterative methods 1951: Decompositional approach for matrix computation 1957: Fortran optimizing compiler 1959-61: QR algorithms 1962: Quicksort 1965: Fast Fourier Transform (FFT) 1977: Integer relation detection algorithm 1982: Fast multipole algorithm http://amath.colorado.edu/resources/archive/topten.pdf 14

DERIVATIVES Recall the limit definition of the first

derivative.

0

( ) ( )'( ) limh

dy f x h f xf xdx h

15

PARTIAL DERIVATIVES Same as derivatives, keep each other

dimension constant.

0

( , , ) ( , , )limxh

f f x h y z f x y zfx h

16

TANGENTS AND GRADIENTS Recall that the slope of a curve (defined as a

1D function) at any point x, is the first derivative of the function.

That is, the linear approximation to the curve in the neighborhood of t is l(x) = b + f’(t)x

f(x)

t

(1,f’(t))

x

y

17

TANGENTS AND GRADIENTS Since we also want this linear approximation

to intersect the curve at the point t. l(t) = f(t) = b + f’(t)t

Or, b = f(t) - f’(t)t We say that the line l(x) interpolates the

curve f(x) at the point t.

18

FUNCTIONS AS CURVES We can think of the curve shown in the

previous slide as the set of all points (x,f(x)). Then, the tangent vector at any point along

the curve is

, ( ) 1, '( )d x f x f xdx

19

SIDE NOTE ON CURVES There are other ways to represent

curves, rather than explicitly.Functions are a subset of curves

(x,y(x)).Parametric equations represent the

curve by the distance walked along the curve (x(t),y(t)).

Circle: (cos, sin) Implicit representations define a

contour or level-set of the function: f(x,y) = c.

20

TANGENT PLANES AND GRADIENTS In higher-dimensions, we have the same

thing: A surface is a 2D function in 3D:

Surface = (x, y, f(x,y) ) A volume or hyper-surface is a 3D function

in 4D:Volume = (x, y, z, f(x,y,z) )

21

TANGENT PLANES AND GRADIENTSThe linear approximation to the higher-

dimensional function at a point (s,t), has the form: ax+by+cz+d=0, or z(x,y) = …

What is this plane?

22

TANGENT PLANES AND GRADIENTSThe formula for the plane is rather

simple:

z(s,t) = f(s,t) - interpolatesz(s+dx,t) = f(s,t) + fx(s,t)dx = b + adx

Linear in dxOf course, the plane does not stay

close to the surface as you move away from the point (s,t).

( , ) ( , ) ( , )( ) ( , )( )x yz x y f s t f s t x s f s t y t

23

TANGENT PLANES AND GRADIENTS The normal to the plane is thus:

The 2D vector:

is called the gradient of the function. It represents the direction of maximal

change.

( , ), ( , ),1x yN f s t f s t

( , ), ( , )T

x yf s t f s t

24

GRADIENTS The gradient thus indicates the direction to

walk to get down the hill the fastest.

Also used in graphics to determine illumination.

( , ) ( , ), ( , )T

f ff x y x y x yx y

25

REVIEW OF FUNCTIONS Extrema of a function occur where f’(x)=0. The second derivative determines whether

the point is a minimum or maximum. The second derivative also gives us an

indication of the curvature of the curve. That is, how fast it is oscillating or turning.

26

THE CLASS OF POLYNOMIALS Specific functions of the form:

2 3 4 50 1 2 3 4 5

0

0 1 2 3 4 5

( )

( ( ( ( ( ) )))))

ii

i

p x a a x a x a x a x a x

a x

a x a x a x a x a x a

27

THE CLASS OF POLYNOMIALS For many polynomials, the latter coefficients

are zero. For example:p(x) = 3+x2+5x3

28

TAYLOR’S SERIES For a function, f(x), about a point c.

I.E. A polynomial

2 3

( )

0

( ) ( )( ) ( ) ( )( ) ( ) ( )2! 3!

( ) ( )!

kk

k

f c f cf x f c f c x c x c x c

f x x ck

29

TAYLOR’S THEOREM Taylor’s Theorem allows us to truncate this

infinite series:

( )

10

( 1)1

1

( )( ) ( )!

( ) ( )( 1)!

( ) ( , )

knk

nk

nn

n

f cf x x c Ek

fE x cn

where x c x

30

TAYLOR’S THEOREM

Some things to note:

1. (x-c)(n+1) quickly approaches zero if |x-c|<<12. (x-c)(n+1) increases quickly if |x-c|>>13. Higher-order derivatives may get smaller (for

smooth functions).

31

HIGHER DERIVATIVES What is the 100th derivative of sin(x)?

What is the 100th derivative of sin(3x)? Compare 3100 to 100!

What is the 100th derivative of sin(1000x)?

32

TAYLOR’S THEOREM Hence, for points near c we can just drop the

error term and we have a good polynomial approximation to the function (again, for points near c).

Consider the case where (x-c)=0.5

For n=4, this leads to an error term around 2.6*10-4 f()

Do this for other values of n. Do this for the case (x-c) = 0.1

( 1)

1 1

( ) 1( 1)! 2

n

n n

fEn

33

SOME COMMON DERIVATIVES

1( )

(sin ) cos

(cos ) sin

( )

1(ln )

n n

x x

d ax naxdxd x xdxd x xdxd e edxd xdx x

1( )

(sin ) cos

(cos ) sin

( )

1(ln )

n n

u u

d duau naudx dxd duu udx dxd duu u chain ruledx dxd due edx dxd duudx u dx

34

SOME RESULTING SERIES About c=0

2 3 4

1 2 2 3 3

1 2 2 3 3

2 3 4

12! 3! 4!

( 1) ( 1)( 2)( )2! 3!

1 2 31 1 1 1

1

x

n n n n n

n n n n

x x xe x

n n n n na x a na x a x a x

n n na a x a x a x

x x x x xx

35

SOME RESULTING SERIES About c=0

3 5 7

2 4 6

2 3 4 5

3 5 7

sin3! 5! 7!

cos 12! 4! 6!

ln(1 ) 1 12 3 4 5

1ln 2 1 11 3 5 7

x x xx x

x x xx

x x x xx x x

x x x xx xx

36

BOOK’S INTRODUCTION EXAMPLE Eight terms for first series not even yielding a

single significant digit. Only four for

second serieswith foursignificantdigits.

3 5 7

ln 2 0.6931471801 1 1 1 1 1 1ln(1 1) 12 3 4 5 6 7 8

0.63452

1 1 111 1 3 3 33ln 21 3 3 5 713

0.69313

37

MEAN-VALUE THEOREM Special case of Taylor’s Theorem, where n=0,

x=b.

Assumes f(x) is continuous and its first derivative exists everywhere within (a,b).

1( ) ( )( ) ( ) ( ) ( , )

f b f a Ef a b a f a b

38

MEAN-VALUE THEOREMSo what! What does this mean?Function can not jump away from

current value faster than the derivative will allow.

f(x)

a b secant 39

ROLLES THEOREM If a and b are roots (f(a)=f(b)=0) of a

continuous function f(x), which is not everywhere equal to zero, then f’(t)=0 for some point t in (a,b).

I.e., What goes up, must come down.

f(x)

a b

f’(t)=0

t40

CAVEAT For Taylor’s Series and Taylor’s Theorem to

hold, the function and its derivatives must exist within the range you are trying to use it.

That is, the function does not go to infinity, or have a discontinuity (implies f’(x) does not exist), …

41

REFERENCE Thanks….

42