numerical technique

7/24/2019 numerical technique

1/46

General linear methodsfor ordinary differential equations

John Butcher

The University of Auckland, New Zealand

Acknowledgements:

Will Wright, Zdzisaw Jackiewicz, Helmut Podhaisky

General linear methods for ordinary differential equations p. 1/46


2/46

Solving ordinary differential equations numerically is,even today, still a great challenge.

This applies especially to stiff differential equations andto closely related problems involving algebraicconstraints (DAEs).

Although the problem seems to be solved there arealready highly efficient codes based on RungeKuttamethods and linear multistep methods questions

concerning methods that lie between the traditionalfamilies are still open.

I will talk today about some aspects of these so-calledGeneral linear methods.General linear methods for ordinary differential equations p. 2/46


3/46

ContentsIntroduction to stiff problems

Why do we want a new numerical method?

What are General Linear Methods?

What we want from General Linear MethodsPropagation of errors

Algebraic analysis of order

High order and stage orderGeneral Linear Methods and RungeKutta stability

Doubly companion matrices

Inherent RungeKutta stability

Proof of stability result

Example methodsImplementation questionsGeneral linear methods for ordinary differential equations p. 3/46


4/46

Introduction to stiff problems

We will write a standard initial value problem in the formy(x) =f(x, y(x)), y(x0) =y0.

The variablex will be referred to as time even if itdoes not represent physical time.

Consider the problem

y1(x)

y2(x)

y3(x)=

y2(x)

y1(x)

Ly1(x) +y2(x) +Ly3(x) ,

with initial valuesy1(0) = 0, y2(0) = 1, y3(0) =.



5/46

If = 0, the solution is

y(x) = sin(x)

cos(x)sin(x)

.

Change the value ofand the third component changesto

sin(x) + exp(Lx).

We will see how this works for values ofL=25and= 2.

The solution is plotted for the interval[0, 0.85].

For the approximate solution computed by the Eulermethod, we choosen = 16steps and then decreasento

10.General linear methods for ordinary differential equations p. 5/46


6/46

0

1

2

0 0.5 x

y1

y2

y3Exact solution forL =25



7/46

0

1

2

0 0.5

x

y1

y2

y3Approximate solution forL=25

withn= 16steps



8/46

0

1

2

0 0.5 xy1

y2

y3

L=25,n = 10



9/46

Note that there seems to be no difficulty approximating

y1andy2, even for quite large stepsizes.

For stepsizes less thanh = 0.08, the approximation toy3converges to the same value as

y1, but not in precisely the

same way as for the exact solution.

For stepsizes greater thanh = 0.08the approximationstoy3are hopeless.

We can understand what happens better by studying the

differencey3y1.



10/46

The value ofy3y1satisfies the equationy(x) =Ly(x),

with solution constexp(Lx).

The exact solution toy =Lyis multiplied byexp(z)(wherez=hL) in each time step but what about the

computedsolution?

According to the formula for the Euler method,

yn=yn1+hf(tn1, yn1), the numerical solution ismultiplied instead by1 +z

in each time step.



11/46

The fact that1 +zis a poor approximation toexp(z)whenzis a very negative number, is at the heart of thedifficulty in solving stiff problems.

Sometimes the implicit Euler method is used for thesolution of stiff problems because in this case the

approximationexp(z)1 +zis replaced byexp(z)(1z)1.

To see what happens in the case of the contrived problemwe have considered, when Euler is replaced by implicitEuler, we present three further figures.



12/46

0

1

2

0 0.5 x

y1

y2

y3

Solution computed by implicit Euler forL =25

withn= 16steps



13/46

0

1

2

0 0.5 x

y1

y2

y3


withn= 10steps



14/46

0

1

2

0 0.5 x

y1

y2

y3


withn= 5steps



15/46

It looks as though, even forn as low as5, we getcompletely acceptable performance.

The key to the reason for this is that|(1z)

1

|is

bounded by1for allzin the left half-plane.

This means that a purely linear problem, with constant

coefficients, will have stable numerical behaviourwhenever the same is true for the exact solution.

Methods with this property are said to be A-stable.

Although by no means a guarantee that it will solve all

stiff problems adequately, A-stability is an importantattribute and a useful certification.General linear methods for ordinary differential equations p. 15/46


16/46

Why do we want a new numerical method?

We want methods that are efficient for stiff problemsHence we cannot use highly implicit Runge-Kuttamethods

Hence we want to avoid non- A-stable BDF methods

We want methods that have high stage order

Otherwise order reduction can occur for stiffproblems

Otherwise it is difficult to estimate accuracy

Otherwise it is difficult to interpolate for denseoutput



17/46

We want methods for which asymptotically correct errorestimates can be found

This means having accurate information available in

the step to make this possibleHigh stage order makes this easier

We want methods for which changes to adjacent orders

can be considered and evaluated

This means being able to evaluate, for example,

h

p+2

y

(p+2) as well ash

p+1

y

(p+1)

(pdenotes the order of the method)

Evaluating the error for lower order alternative

methods is usually easyGeneral linear methods for ordinary differential equations p. 17/46


18/46

What are General Linear Methods?

General linear methods are the natural generalization ofRungeKutta (multi-stage) methods and linear multistep(multivalue) methods.

This can be illustrated in a diagram.

General Linear Methods

Runge-Kutta Linear Multistep

EulerGeneral linear methods for ordinary differential equations p. 18/46


19/46

We will considerr-value,s-stage methods, wherer= 1for a RungeKutta methodand

s= 1for a linear multistep method.

Each step of the computation takes as input a certainnumber (r) of items of data and generates for output the

same number of items.The output items correspond to the input items butadvanced through one time step (h).

Within the step a certain number (s) of stages ofcomputation are performed.

Denote bypthe order of the method and byqthestage-order.



20/46

At the start of step numbern, denote the input items byy

[n1]i ,i= 1, 2, . . . , r.

Denote the stages computed in the step and the stagederivatives byYiandFirespectively,i= 1, 2, . . . , s.

For a compact notation, write

y[n1] =

y[n1]1

y[n1]2

...

y[n1]r

, y[n] =

y[n]1

y[n]2...

y[n]r

, Y=

Y1Y2...

Ys

, F=

F1F2

...

Fs



21/46

The stages are computed by the formulae

Yi=s

j=1

aijhFj+r

j=1

uijy[n1]j , i= 1, 2, . . . , s

and the output approximations by the formulae

y[n]i =

sj=1

bijhFj+

rj=1

vijy[n1]j , i= 1, 2, . . . , r

These can be written together using a partitioned matrix

Yy

[n] = A UB V

hFy

[n1] General linear methods for ordinary differential equations p. 21/46


22/46

What we want from General Linear Methods

Convergence (consistency and stability)

Local accuracy

Global accuracy (understand propagation of errors)A-stability for stiff problems

RK stability (behaves like Runge-Kutta method)



23/46

Propagation of errors

LetSdenote a starting method, that is a mapping fromR

N to RrN and a corresponding finishing method

F : RrN RN such thatFS=id

The order of accuracy of a multivalue method is definedin terms of the diagram

E

S S

MO(hp+1)

(h= stepsize)



24/46

By duplicating this diagram over many steps, globalerror estimates may be found

E E E

S S S S S

M MM

O(hp)

F

O(hp)


Al b i l i f d


25/46

Algebraic analysis of order

We recall the way order is defined for Runge-Kuttamethods.We consider the solution of an autonomous differentialequationy(x) =f(y(x)).LetTdenote the set of rooted trees

. . .

To each of these is associated an elementarydifferentialF(t)defined in terms of the functionf.

Also associated with each tree is its symmetry(t)andits density(t). In the following table,r(t)is theorder (number of vertices) oft.



26/46

r(t) t (t) (t) F(t) F(t)i1 1 1 f fi

2 1 2 ff fijfj

3 2 3 f(f, f) fijkfjfk

3 1 6 fff fijfjkf

k

4 6 4 f(f , f , f ) fijklfjfkfl

4 1 8 f(f, ff) fijkfjfkl f

l

4 2 12 ff(f, f) fijfjklfkfl

4 1 24 ffff fijfjkf

klf

l



27/46

We will look at a single example tree in more detail.

t= r(t) = 6

(t) = 4

(t) = 18

6

1 1 3

1 1

F(t) =f(f,f,f(f,f))

f

f f f

f f



28/46

The solution to the standard autonomous initial-valueproblem

y(x) =f(y(x)), y(x0) =y0

has the formal Taylor series

y(x0+h) =y0+tT

hr(t)

(t)(t)F(t)(y0)

A Runge-Kutta method has the same series but with

1/(t)replaced by a sequence of elementary weights

characteristic of the specific Runge-Kutta method.Expressions of this type are usually referred to asB-series.



29/46

LetGdenote the set of mappingsT

#

R

whereT# ={} T (and is the empty tree).

Also writeG0andG1as subsets for which 0or

1respectively.A mappingG1GGcan be defined whichrepresents compositions of various operations.

In particular if this is restricted toG1G1G1itbecomes a group operation.

LetEdenote the mappingt1/(t) andDthemapping which takes every tree to zero except the uniqueorder 1 tree, which maps to one.



30/46

A general linear method defined from the matrices[A,U,B,V]has orderpif there existsGr andGs1, such that

=AD+U ,E=BD+V , (*)

where pre-multiplication byEand post-multiplicationbyDare scalar times vector products and (*) is to holdonly up to trees of orderp.


High order and stage order


31/46

High order and stage order

If we consider only methods withq=p, then simplercriteria can be found.Letexp(cz)denote the component-by-componentexponential then orderp and stage-orderqp1isequivalent to

exp(cz) =zA exp(cz) +U w(z) +O(zq+1),

exp(z)w(z) =zB exp(cz) +V w(z) +O(zp+1),where the power series

w(z) =w0+w1z+ +wpzp

represents an input approximation

y[0] =

p

i=0

wihiy(i)(x0).


GL Methods and RK stability


32/46

GL Methods and RK stability

A General Linear Method is RungeKutta stable if itsstability matrix

M(z) =V +zB(IzA)1Uhas only one non-zero eigenvalue.

Armed with this property we should expect attraction to

the manifoldS(RN), and stable adherence to thismanifold.

This means that the method starts acting like a one-step

method.




33/46


Matrices like the following are companion matrices forthe polynomial

zn + 1zn1 + +nor

zn +1zn1 + +n,respectively:

123 n1n1 0 0 0 00 1 0 0 0

... ... ... ... ...

0 0 0 0 00 0 0

1 0

,

0 0 0 0 n1 0 0 0 n10 1 0 0 n2

... ... ... ... ...

0 0 0 0 20 0 0

1

1



34/46

Their characteristic polynomials can be found fromdet(IzA) =(z)or(z), respectively, where,(z) = 1+1z+ +nzn, (z) = 1+1z+ +nzn.A matrix with both

and

terms:

X=

1 2 3 n1 nn1 0 0 0 n10 1 0 0 n2... ... ... ... ...0 0 0 0 20 0 0 1

1

,

is known as a doubly companion matrix and hascharacteristic polynomial defined by

det(IzX) =(z)(z) +O(zn+1)



35/46

Matrices

1 and

transformingX

to Jordan canonicalform are known.

In the special case of a single Jordan block withn-fold

eigenvalue, we have

1 =

1 +1 2 +1+2

0 1 2+1 0 0 1 ...

... ...

. . .

,

where row numberi + 1is formed from row numberibydifferentiating with respect toand dividing byi.

We have a similar expression for:



36/46

=

. . . ... ...

...

1 2+1 2 +1 +2

0 1 +1 0 0 1

The Jordan form is1X = J+I, whereJij = i,j+1.That is

1X =

0 0 0

1 0 0... ...

... ...

0 0 0

0 0 1


Inherent RungeKutta stability


37/46

Inherent Runge Kutta stabilityUsing doubly companion matrices, it is possible to

construct GL methods possessing RK stability usingrational operations. The methods constructed in this wayare said to possess Inherent RungeKutta Stability.

Apart from exceptional cases, (in which certain matricesare singular), we characterize the method withr=s=p + 1 =q+ 1by the parameters

single eigenvalue ofAc1,c2,. . . ,csstage abscissae

Error constant

1,2,. . . ,pelements in last column ofssdoubly companion matrixX

Information on the structure ofVGeneral linear methods for ordinary differential equations p. 37/46


38/46

Consider only methods for which the stepnoutputsapproximate the Nordsieck vector:

y[n]

1y

[n]2

y[n]3

...

y[n]p+1

y(xn)

hy(xn)

h2y(xn)

...

hpy(p)(xn)

For such methods,Vhas the form

V = 1 vT

0 V General linear methods for ordinary differential equations p. 38/46


39/46

Such a method has the IRKS property if a doublycompanion matrixXexists so that for some vector,

BA=XB, BU =XV V X+e1T

, (V) = 0

It will be shown that, for such methods, the stabilitymatrix satisfies

M(z)V +ze1T(IzX)1

which has all except one of its eigenvalues zero. The

non-zero eigenvalue has the role of stability function

R(z) = N(z)

(1z)s


Proof of stability result


40/46

oo o stab ty esu t

(IzX)M(z)(IzX)1

= (V zXV +z(Bz

XB =BA

XB)(IzA)1U)(IzX)1

= (V zXV +zBU)(IzX)1

BU=XV V X+e1T= V +ze1

T(IzX)1

This has a single non-zero eigenvalue equal to

R(z) = 1 + zT(IzX)1e1=det(I+z(e1

T X))

det(IzX)General linear methods for ordinary differential equations p. 40/46

Example methods


41/46

p

The following third order method is explicit and suitablefor the solution of non-stiff problems

AUBV

=

0 0 0 0 1 141

321

384

1761885 0 0 0 1 2237

37702237

150802149

90480

335624311025

2955

0 0 1 16195911244100

260027904800

151780139811200

678436435 39533 5 0 1 294286435 527585 41819102960678436435

39533 5 0 1

294286435

527585

41819102960

0 0 0 1 0 0 0 08233

27411

1709

43

0 48299

0 161264

8 12 403

2 0 263

0 0



42/46

Implementation questions


43/46

p q

Many implementation questions are similar to those fortraditional methods but there are some new challenges.We want variable order and stepsize and it is even arealistic aim to change between stiff and non-stiff

methods automatically.Because of the variable order and stepsize aims, we wishto be able to do the following:

Estimate the local truncation error of the current step

Estimate the local truncation error of an alternativemethod of higher order

Change the stepsize with little cost and with littleimpact on stability



44/46

My final comment is that all these things are possible andmany of the details are already known and understood.

What isnotknown is how to choose between different

choices of the free parameters.

Identifying the best methods is the first step in the

construction of a competitive stiff and non-stiff solver.


References


45/46

The following recent publications contain references toearlier work:

Numerical Methods for Ordinary DifferentialEquations, J. C. Butcher,J. Wiley, Chichester,(2003)

General linear methods, J. C. Butcher,ActaNumerica 15(2006), 157256.



46/46

Gracias

Thank you

Obrigado


Documents

numerical technique