2 Numerical Methods for Initial Value Problemsernst/Lehre/Numerik_II/Folien/nade... · Numerical Analysis of Differential Equations 44 2 Numerical Methods for Initial Value Problems

Numerical Analysis of Differential Equations 44

2 Numerical Methods for Initial Value Problems

Contents

2.1 Some Simple Methods

2.2 One-Step Methods – Definition and Properties

2.3 Runge-Kutta-Methods

2.4 Linear Multi-Step Methods

2.5 Convergence of One-Step Methods

2.6 Zero-Stability of Linear Multistep Methods

2.7 Stepsize Control

2 Numerical Methods for Initial Value Problems TU Bergakademie Freiberg, SS 2012


2.1 Some Simple Methods

Consider an IVPy ′ = f (t,y), y(t0) = y0. (IVP)

Under the assumptions of Theorem 1.1, (IVP) possesses a unique solutiony = y(t), say, on the interval I.

We will approximate y(t) for t ∈ [t0, tend] ⊂ I using the Euler methoda

(sometimes called the explicit Euler method), which can be regarded as aprototype for all numerical schemes for solving IVPs.

As all methods we shall encounter in this course, Euler’s method is basedon discretization, i.e., approximating the solution y on only a discrete subset{tn : n = 0, 1, . . . , N} of the interval [t0, tend].

aLEONHARD EULER (1707–1783)

2.1 Some Simple Methods TU Bergakademie Freiberg, SS 2012


We fixN ∈ N, set h := (tend−t0)/N and define tn := t0+nh, n = 0, 1, . . . , N ,i.e. t0 < t1 · · · < tN−1 < tN = tend. The number h > 0 is called the stepsize,which we choose to be constant for convenience only.

One step of Euler’s method requires an approximation yn to y(tn), n =

0, 1, . . . , N − 1. Noting that, under the assumption y ∈ C2[t0, tend],

y(tn+1) = y(tn + h) = y(tn) + hy ′(tn) +1

2h2y ′′(ξ)

≈ y(tn) + hy ′(tn) = y(tn) + hf (tn,y(tn)) ≈ yn + hf (tn,yn),

we obtain the

(Explicit) Euler scheme:

y0 given,

yn+1 := yn + h f (tn,yn) n = 0, 1, . . . , N − 1.



A common way to visualize a first order ODE y′ = f(t, y) is providedby its associated direction field, obtained by drawing an arrow with slopey′ = f(t, y) at every point in the (t, y)-plane.

The graph of the solution to the IVP

y′ = f(t, y), y(t0) = y0,

must pass through the point (t0, y0) and its tangents at every point (t, y(t))

must be parallel to the direction field.

The following figure illustrates the Euler method approximating the solutionof the logistic equation y′ = y(1 − y) with IC y(0) = 1/10 using the stepsize h = 1. Rather than following its exact trajectory (which is, of course,impossible), the Euler scheme may be viewed at producing a piecewiselinear approximation. At the starting point t0 the Euler approximation usesthe correct slope f(t0, y0) = 9/100. However, from the next point t1 = h = 1

onwards the slope is “wrong”, and the approximation may deviate furtherand further from the exact solution.



0 0.5 1 1.5 2 2.5 3 3.5 40

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9Euler−Verfahren

y’=y(1−y), y(0)=1/10



−3 −2 −1 0 1 2 30

1

2

3

4Richtungsfeld von y’=x(y−2)

und Loesungen mit Anfangswerten y(0) = −1:0.5:1



Does Euler’s method converge, i.e., do the approximations converge tothe exact solution y(t) as h→ 0?

Note: each value of the step size h is associated with a (finite) collection ofapproximations

yn = yn(h), n = 0, 1, . . . , N(h) := b(tend − t0)/hc.

The method is said to converge (in [t0, tend]) if

limh→0+

max0≤n≤N(h)

‖yn(h)− y(tn)‖ = 0. (Conv)

Theorem 2.1 (Convergence of Euler’s method) Under the assumptionsof Theorem 1.1 the Euler method is convergent. More precisely,

max0≤n≤N(h)

‖yn(h)− y(tn)‖ = O(h) as h→ 0.



t_0 t

exakte Loesung Schrittweite h

0Schrittweite h

1Schrittweite h

2



Modified Euler method:

y0 given,

yn+1 := yn + h f(tn + 1

2h,yn + 12hf (tn,yn)

)n = 0, 1, . . . , N − 1.

Improved Euler method:

y0 gegeben,

yn+1 := yn + 12h[f (tn,yn) + f (tn + h,yn + hf (tn,yn))]

n = 0, 1, . . . , N − 1.

The construction of these variants of Euler’s method is also easily interpre-ted using the direction field. Both are convergent schemes – in fact (by atrivial modification of the proof of Theorem 2.1) one can show that for bothmethods

max0≤n≤N(h)

‖yn(h)− y(tn)‖ = O(h2), as h→ 0.



0 0.5 1 1.5 2 2.5 3 3.5 40

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9exakte Loesung Euler−Verfahren modifiziertes Euler−Verf.verbessertes Euler−Verf.



Implicit Euler method:

y0 given,

yn+1 := yn + h f (tn + h,yn+1) n = 0, 1, . . . , N − 1.

In contrast with the methods introduced so far, this is an implicit method,which means that determining yn+1 from yn requires the solution of asystem of algebraic equations, which, unless f is linear in y , is generallynonlinear .

As in Theorem 2.1 one can show that for the implicit Euler method

max0≤n≤N(h)

‖yn(h)− y(tn)‖ = O(h) as h→ 0.



The schemes we have considered up to now involved two approximationsof the solution at consecutive times. Such schemes are called one-stepmethods.

Multistep methods are based on difference formulas involving more thantwo solution values. Examples are

y(t+ h)− y(t− h)

2h= y′(t) +

h2

6y′′′(t) +O(h4),

3y(t)− 4y(t− h) + y(t− 2h)

2h= y′(t)− h2

3y′′′(t) +O(h3).

When these formulas are evaluated at t = tn+1, we obtain . . .



. . . the explicit

Midpoint-rule: (leapfrog method)

y0,y1 given,

yn+1 = yn−1 + 2h f (tn,yn), n = 1, 2, . . . , N − 1.(2.1)

as well as an implicit method belonging to the so-called BDF-family (Back-ward Differentiation Methods)

A BDF method:

y0,y1 given,

3yn+1 − 4yn + yn−1 = 2hf (tn + h,yn+1), n = 1, 2, . . . , N − 1.

We observe that these methods can only proceed once, in addition tothe initial vaue y0, an approximation y1 is available. All multistep methodsrequire such a startup calculation.



2.2 One-Step Methods – Definition and Properties

A one-step method (OSM) has the general form

yn+1 = yn + hΦf (yn+1,yn, tn;h). (OSM)

We will consider exclusively methods whose increment function Φf pos-sesses the following properties:

Φf≡0 (yn+1,yn, tn;h) ≡ 0 (V1)

and

‖Φf (yn+1,yn, tn;h)− Φf (y∗n+1,y∗n, tn;h)‖ ≤M

1∑j=0

‖yn+j − y∗n+j‖. (V2)

For “reasonable” OSM property (V2) follows from the Lipschitz continuity off (cf. Theorem 1.1), which we always assume to hold.

2.2 One-Step Methods – Definition and Properties TU Bergakademie Freiberg, SS 2012


Examples of more complicated OSMs:

yn+1 − yn = 14h(k1+3k3), where

k1 = f (tn,yn),

k2 = f (tn + 13h,yn + 1

3hk1),

k3 = f (tn + 23h,yn + 2

3hk2),

(Example 1)

an explicit OSM belonging to the class of Runge-Kutta methods a b (cf.§2.3) and the implicit Runge-Kutta method

yn+1 − yn = 12h(k1+k2), where

k1 = f (tn,yn),

k2 = f (tn + h,yn + 12hk1 + 1

2hk2).

(Example 2)

aCARLE DAVID TOLME RUNGE (1856–1927)bMARTIN WILHELM KUTTA (1867–1944)



We say the method (OSM) is convergent if

limh→0

t=t0+nh

yn = limh→0

t=t0+nh

yn(h) = y(t)

holds — and does so

• for all IVPs which satisfy the assumptions of Theorem 1.1 where y(t)

denotes the solution of such an IVP,• uniformly for all t ∈ [t0, tend],• for all solutions {yn(h)} = {yn} of (OSM) with initial values y0(h)

satisfying limh→0 y0(h) = y0.

Equivalent: The global discretisation error

en = en(h) := y(tn)− yn(h)

converges to 0 uniformly (as h→ 0):

limh→0

max0≤n≤N

‖en(h)‖ = limh→0

max0≤n≤N

‖y(tn)− yn(h)‖ = 0.



We define the local discretion error (local truncation error) Tn = Tn(h) of amethod at step n to be the difference of the left and right-hand sides of theexpression defining the method (scaled in such a way that the differentialequation is approximated as h → 0) when the exact solution is substitutedfor the approximation.

A method is said to be consistent of order p if

Tn(h) = O(hp) as h→ 0.

Example: Midpoint rule

Tn =y(tn+1)− y(tn−1)

2h− f(tn,y(tn))

=

[y ′(tn) +

h2

6y ′′′(tn) +O(h4)

]− y ′(tn) =

h2

6y ′′′(tn) +O(h4),

i.e., we have consistency of order p = 2.



Warning: In the literature one often finds a slightly different definition of thelocal discretization error based on the method’s defining equation in theform (OSM). This increases the order of consistency by one power of h.

Motivation: Description of the error incurred in a single step independentof that accumulated from previous steps using the localization assumptionyn = y(tn). In this case one step of the scheme (OSM) compared with theexact solution compares as

yn+1(h) = y(tn) + hΦf (yn+1,y(tn), tn;h),

y(tn+1) = y(tn) + hΦf (y(tn+1),y(tn), tn;h) + hTn+1(h).

Denoting the difference Sn+1(h) := y(tn+1)− yn+1 as step error, we obtainusing (V2),

‖Sn+1‖ ≤ h‖Φf (y(tn+1),y(tn), tn;h)− Φf (yn+1,y(tn), tn;h)‖+ h‖Tn+1‖≤ hM‖Sn+1‖+ h‖Tn+1‖,

implying (1− hM)‖Sn+1‖ ≤ hTn+1 so that Sn behaves like hTn as h→ 0.



Relevance of Sn: The integration of (IVP) from t0 to tend requires (tend −t0)/h steps. If step error Sn is committed at step n, this results, under thesimplifying assumption that individual error have no effect on each other, inan accumulated (global) error of

tend − t0h

S(h) ∼ T (h).

As we shall see shortly, the crucial property for the validity of this simplifyingassumption is the stability of the method.



2.3 Runge-Kutta Methods

2.3.1 Derivation via Quadrature

Fundamental theorem of calculus:

y(t+ h) = y(t) + [y(t+ h)− y(t)] = y(t) +

∫ t+h

t

y ′(s) ds

Change of variables: s = t+ τh, 0 ≤ τ ≤ 1

= y(t) + h

∫ 1

0

y ′(t+ τh) dτ.

Approximate integral with quadrature formula:∫ 1

0

g(τ) dτ ≈m∑

j=1

βjg(γj). (∗)

To integrate at least g ≡ 1 exactly, we require∑m

j=1 βj = 1.

2.3 Runge-Kutta Methods TU Bergakademie Freiberg, SS 2012


We obtain

y(t+ h) ≈ y(t) + hm∑j=1

βjy′(t+ γjh)

= y(t) + hm∑j=1

βjf (t+ γjh,y(t+ γjh))

(RK-1)

Problem now: don’t know y(t+ γjh) = y(t) + h∫ γj0

y ′(t+ τh) dτ .

Again approximate by quadrature, but using same nodes {γj}mj=1 as in (∗)to avoid introducing new unknowns y(t+ node · h):∫ γj

0

g(τ) dτ ≈m∑`=1

αj,`g(γ`), j = 1, . . . ,m. (∗∗)

To integrate at least g ≡ 1 exactly, require∑m`=1 αj,` = γj , j = 1, . . . ,m.



This yields

y(t+ γjh) ≈ y(t) + h

m∑`=1

αj,` y′(t+ γ`h)

= y(t) + h

m∑`=1

αj,` f (t+ γ`h,y(t+ γ`h))

(RK-2)

Introduce kj := f(t+ γjh,y(t+ γjh)

), j = 1, . . . ,m.

(RK-2): kj ≈ f

(t+ γjh,y(t) + h

m∑`=1

αj,`k`

), j = 1, . . . ,m.

(RK-1): y(t+ h) ≈ y(t) + h

m∑j=1

βj kj .



m-stage Runge-Kutta method (RKM):

yn+1 = yn + h

m∑j=1

βjkj where

kj = f

(tn + γjh,yn + h

m∑`=1

αj,`k`

), j = 1, . . . ,m.

(RKM)

Shorthand notation for RKMs:

Butcher Tableaua

aJOHN CHARLES BUTCHER (∗1933)

γ1 α1,1 · · · α1,m

......

...

γm αm,1 · · · αm,m

β1 · · · βm



Examples. The Butcher Tableau

0 0 0

1 1 0

1/2 1/2

represents a two-stage explicit RKM, the improvedEuler method:

yn+1 = yn + h2

[f(tn,yn) + f(tn + h,yn + hf(tn,yn))

]= yn + h

2 (k1 + k2),

k1 = f(tn,yn),

k2 = f(tn + h,yn + hk1)

Note: a RKM is explicit when

αj,` = 0 ∀j ≤ `,

i.e., if the coefficients αj,` form a strictly lower triangular matrix.



0 1/4 −1/4

2/3 1/4 5/12

1/4 3/4

represents a two-stage implicit RKM:

k1 = f(tn,yn + h( 1

4k1 −14k2)

),

k2 = f(tn + 2

3h,yn + h( 14k1 + 5

12k2)),

(“two” — generally nonlinear — equations for k1 and k2)

yn+1 = yn +h

4(k1 + 3k2).

(Example 2 in §2.2 is a further example of an implicit two-stage RKM.)



0 0 0 0

1/2 1/2 0 0

1 −1 2 0

1/6 4/6 1/6

represents an explicit three-stage RKM

Kutta’s third-order method:

k1 = f (tn,yn),

k2 = f(tn + 1

2h,yn + 12hk1

),

k3 = f(tn + h,yn + h(−k1 + 2k2)

),

yn+1 = yn + 16h(k1 + 4k2 + k3).



0 0 0 0

1/3 1/3 0 0

2/3 0 2/3 0

1/4 0 3/4

represents another explicit three-stage RKM

Heun’s third-order method:

k1 = f (tn,yn),

k2 = f(tn + 1

3h,yn + 13hk1

),

k3 = f(tn + 2

3h,yn + 23hk2

),

yn+1 = yn + h4 (k1 + 3k3)

(cf. Example 1 in §2.2).



0 0 0 0 0

1/2 1/2 0 0 0

1/2 0 1/2 0 0

1 0 0 1 0

1/6 2/6 2/6 1/6

represents an explicit four-stage RKM

Classical Runge-Kutta method:

k1 = f (tn,yn),

k2 = f (tn + 12h,yn + 1

2hk1),

k3 = f (tn + 12h,yn + 1

2hk2),

k4 = f (tn + h,yn + hk3),

yn+1 = yn + 16h(k1 + 2k2 + 2k3 + k4).



An (equivalent) alternative way of representing (RKM) is

yn+1 = yn + h

m∑j=1

βj f (tn + γjh, yj)

where yj = yn + hm∑`=1

αj,`f (tn + γ`h, y`), j = 1, . . . ,m

(RKM∗)

(simply set kj = f (tn + γjh, yj)).

Interpretation:

yj : approximate solution y at time tn + γjh

kj : approximate slopes y ′ at time tn + γjh



2.3.2 Consistency of Runge-Kutta Methods

All RKM can be written

yn+1 = yn+hΦf (yn+1,yn, tn;h)

where Φf (yn+1,yn, tn;h) =m∑j=1

βjkj .

A RKM is consistent (and therefore convergent) if, and only if,

m∑j=1

βj = 1.

Determining the order of consistency of RKMs (or constructing m-stageRKMs with maximal order of consistency) often leads to rather involvedcalculations.



As a tractable example, we shall analyze all explicit three-stage RKMs. Anysuch method has a Butcher Tableau of the form

0 0 0 0

γ2 γ2 0 0

γ3 γ3 − α3,2 α3,2 0

β1 β2 β3

.

We expand the local truncation error

Tn+1(h) =y(tn+1)− y(tn)

h− Φf (y(tn), tn;h)

=y(tn+1)− y(tn)

h−

3∑j=1

βjkj

in powers of h (assuming y and f sufficiently differentiable).



We consider scalar IVPs and, introducing the abbreviations

F := ft + fyf and G := ftt + 2ftyf + fyyf2,

—all derivatives of f evaluated at (tn, y(tn))— employ the chain rule toobtain

y(tn+1)− y(tn)

h= f +

1

2Fh+

1

6(G+ fyF )h2 +O(h3).

On the other hand, we also have

k1 = f(tn, y(tn)) = f,

k2 = f(tn + hγ2, y(tn) + hγ2k1) = f + hγ2F + 12h

2γ22G+O(h3),

k3 = f(tn + hγ3, y(tn) + h(γ3 − α3,2)k1 + hα3,2k2)

= f + hγ3F + h2(γ2α3,2Ffy + 12γ

23G) +O(h3).



This means:

Tn+1(h) =[1−

∑3j=1βj

]f +

[12 − β2γ2 − β3γ3

]Fh

+[( 13 − β2γ

22 − β3γ23) 1

2G+ ( 16 − β3γ2α3,2)Ffy

]h2 +O(h3)

Consequences:

(1) The Euler method is the only explicit one-stage RKM of order one(β1 = 1).

(2) All explicit two-stage RKM of order 2 are characterized by

β1 + β2 = 1 and β2γ2 = 12 .

Examples are the modified (β1 = 0, β2 = 1, γ2 = 12 ) and the improved

(β1 = β2 = 12 , γ2 = 1) Euler methods.

There are no explicit two-stage RKMs of order three or higher.



(3) Explicit three-stage RKM of order 3 are characterized by the fourequations

β1 + β2 + β3 = 1, β2γ22 + β3γ

23 = 1

3 ,

β2γ2 + β3γ3 = 12 , β3γ2α3,2 = 1

6 .

(One can show that none of these methods have order 4.) Examplesinclude Heun’s method (β1 = 1

4 , β2 = 0, β3 = 34 , γ2 = 1

3 , γ3 = α3,2 = 23 )

and Kutta’s method (β1 = 16 , β2 = 2

3 , β3 = 16 , γ2 = 1

2 , γ3 = 1, α3,2 = 2).

(4) Similar (complicated) calculations reveal the existence of a two-parameterfamily of explicit four-stage RKMs, none of which have order 5.An example is the classical RKM; further examples are . . .



0 0 0 0 0

1/3 1/3 0 0 0

2/3 −1/3 1 0 0

1 1 −1 1 0

1/8 3/8 3/8 1/8

(3/8-rule)

0 0 0 0 0

2/5 2/5 0 0 0

3/5 −3/20 3/4 0 0

1 19/44 −15/44 40/44 0

55/360 125/360 125/360 55/360

(Kuntzmann’s method).



This way of analyzing consistency of explicit RKMs becomes increasinglytedious for higher orders:

• order 3: 4 nonlinear equations for coefficients (see above)

• order 8: 200 nonlinear equations for coefficients.

Butcher Theorya uses graph theory (trees) for systematic book-keeping ofpartial derivatives of f for computing the order of a given RKM. It does not,however, provide a technique for constructing RKMs of a desired order.

In the following, we derive a sequence of necessary order conditions,obtained from the specific family of IVPs

y′ = y + t`−1, y(0) = 0, (` ∈ N).

aJ. C. Butcher, The Numerical Analysis of Ordinary Differential Equations. Runge-Kutta andGeneral Linear Methods. John Wiley & Sons, Chichester 1987



Theorem 2.2 (Necessary order conditions for RKM) In order for theRKM defined by the Butcher tableau

c A

b>

to possess consistency order p there must hold

b>AkC`−1e =(`− 1)!

(`+ k)!=

1

`(`+ 1) . . . (`+ k)

for ` = 1, 2, . . . , p and k = 0, 1, . . . , p− `.

Notation:

b := [β1, β2, . . . , βm]>, A := [αj,ν ]1≤j,ν≤m,

C := diag(γ1, γ2, . . . , γm) and e := [1, 1, . . . , 1]> ∈ Rm.



Special cases of the necessary conditions of Theorem 2.2 are (for k = 0)

b>C`−1e =m∑j=1

βjγ`−1j =

1

`for ` = 1, 2, . . . , p

as well as (for ` = 1 with k ← k + 1)

b>Ak−1e =1

k!for k = 1, 2, . . . , p.

Remark. An explicit m-stage RKM can have at most order m, since in thiscase Am = O (A ist a strictly lower trianglular matrix). In fact, for m ≥ 5 themaximal order p(m) attainable by an explicit m-stage RKM is bounded byp(m) ≤ m− 1.

The exact maximal orders for 1 ≤ m ≤ 12 are

m 1 2 3 4 5 6 7 8 9 10 11 12

p(m) 1 2 3 4 4 5 6 6 7 7 8 9



2.4 Linear Multistep Methods

Multistep methods are characterized by using solution approximations ear-lier than yn in the update formula for advancing the numerical approximationfrom yn to yn+1. This is in contrast with one-step methods, which use onlyyn, besides possibly additional intermediate (e.g. stage) quantities.

Of these, the most important class of linear multistep methods (LMM)possesses the general form

r∑j=0

αjyn+j = hr∑j=0

βjf (tn+j ,yn+j), n = 0, 1, 2, . . . . (LMM)

defined by the parameters {αj}rj=0 and {βj}rj=0. Since these are uniqueonly up to a (nonzero) scaling factor, they are typically normalized by thecondition αr = 1.

The method (LMM) is explicit if βr = 0, otherwise implicit.

We first study various subfamilies of LMM.

2.4 Linear Multistep Methods TU Bergakademie Freiberg, SS 2012


2.4.1 Adams-Methods

Adams-Methods have the specific form

yn+r − yn+r−1 = hr∑j=0

βjf (tn+j ,yn+j), (2.2)

i.e., they are LMMs characterized by

αr = 1, αr−1 = −1, in addition to α0 = α1 = · · · = αr−2 = 0.

The coefficients βj result from maximizing the order of consistency for agiven value of r, which is r in the explicit case and r+ 1 in the implicit case.

The former class of methods are known as Adams-Bashforth methods, thelatter Adams-Moulton methods.



Alternative derivation of Adams methods:

From the differential equation, we have

y(tn+r) = y(tn+r−1) +

∫ tn+r

tn+r−1

y ′(τ) dτ = y(tn+r−1) +

∫ tn+r

tn+r−1

f (τ,y(τ)) dτ.

Choosing {βj}rj=0 in such a way that∫ tn+r

tn+r−1

g(τ) dτ ≈ hr∑j=0

βjg(tn+j)

is an interpolatory quadrature formulaa yields the coefficiants of the corre-sponding (explicit or implicit) Adams method.

ai.e., replace g by the unique interpolation polynomial passing through thepoints {(tn, g(tn)), . . . , (tn+r−1, g(tn+r−1))} of degree r − 1 (explicit case) or{(tn, g(tn)), . . . , (tn+r, g(tn+r))} of degree r (implicit case)



For r = 1 to 4 the explicit Adams-Bashforth methods are as follows:

r = 1 : yn+1 = yn + hf (tn,yn),

r = 2 : yn+2 = yn+1 +h

2(−f (tn,yn) + 3f (tn+1,yn+1)) ,

r = 3 : yn+3 = yn+2 +h

12(5f (tn,yn)− 16f (tn+1,yn+1)

+23f (tn+2,yn+2)) ,

r = 4 : yn+4 = yn+3 +h

24(−9f (tn,yn) + 37f (tn+1,yn+1)

−59f (tn+2,yn+2) + 55f (tn+3,yn+3)) .

Forr r = 1 we recover the explicit Euler method.



The implicit Adams-Moulton methods for r = 1 to 4 read:

r = 1 : yn+1 = yn +h

2(f (tn,yn) + f (tn+1,yn+1)) ,

r = 2 : yn+2 = yn+1 +h

12(−f (tn,yn) + 8f (tn+1,yn+1)

+5f (tn+2,yn+2)) ,

r = 3 : yn+3 = yn+2 +h

24(f (tn,yn)− 5f (tn+1,yn+1)

+19f (tn+2,yn+2 + 9f (tn+3,yn+3))) ,

r = 4 : yn+4 = yn+3 +h

720

(−19f (tn,yn) + 106f (tn+1,yn+1)

− 264f (tn+2,yn+2) + 646f (tn+3,yn+3)

+ 251f (tn+4,yn+4)).

For r = 1 we recover the trapezoidal rule.



2.4.2 Nystrom Methods

Nystrom methods have the form

yn+r − yn+r−2 = hr∑j=0

βjf (tn+j ,yn+j). (2.3)

The explicit Nystrom method of order r = 2 is the midpoint rule (2.1).

The implicit Nystrom method of order r = 2 is Simpson’s rule

yn+2 − yn =h

3

(f (tnyn) + 4f (tn+1,yn+1) + f (tn+2,yn+2)

).



2.4.3 BDF Methods

In contrast with Runge-Kutta and Adams methods, which were derivedusing quadrature, the BDF methods (backward differentiation formulas) arebased on numerical differentiation.

Construction of linear r-step method:

(1) Assume approximations yn, yn−1, . . . , yn−r+1 available.

(2) To compute new approximation yn+1, denote by p ∈ Pr the uniqueinterpolating polynomial through the r + 1 points {(tj , yj)}n+1

j=n−r+1.

(3) The missing interpolation condition p(tn+1) = yn+1 (we have not yetcomputed yn+1 ) is replaced by the requirement

p′(tn+1) = f(tn+1, yn+1), (2.4)

i.e., that p satisfy the differential equation at tn+1.



The Newton form of the resulting interpolation polynomial is given by

p(t) = yn+1 + yn+1,n(t− tn+1) + yn+1,n,n−1(t− tn+1)(t− tn)

+ · · ·+ yn+1,n,...,n−r+2,n−r+1(t− tn+1) · · · (t− tn−r+2)

in terms of the r + 1 divided differences yn+1, . . . , yn+1,n,...,n+1−r.

Under the simplifying assumption of constant stepsize h, its derivative att = tn+1 is

p′(tn+1) =r∑j=1

yn+1,n,...,n+1−jhj−1(j − 1)! . (2.5)

Introducing the backward differences

∇0yn+1 := yn+1

∇yn+1 := yn+1 − yn,

∇jyn+1 := ∇(∇j−1yn+1) = ∇j−1yn+1 −∇j−1yn, j = 2, 3, . . . ,



p′(tn+1) may be expressed as

p′(tn+1) =1

h

r∑j=1

∇jyn+1

j,

which, combined with condition (2.4), yields the BDF method of order r as

r∑j=1

1

j∇jyn+1 = hf(tn+1, yn+1), r ∈ N, (2.6)

denoted BDFr for short.

The index translation n+1 7→ n+r and dividing both sides by the coefficientof yn+1 transforms the BDF formulas (2.6) to our standard notation (LMM)for linear multistep methods.



The BDF methods for r = 1 to 6 are:

r = 1 : yn+1 − yn = hf (tn+1,yn+1),

r = 2 : 32yn+2 − 2yn+1 + 1

2yn = hf (tn+2,yn+2)

r = 3 : 116 yn+3 − 3yn+2 + 3

2yn+1 − 13yn = hf (tn+3,yn+3),

r = 4 : 2512yn+4 − 4yn+3 + 3yn+2 − 4

3yn+1 + 14yn = hf (tn+4,yn+4),

r = 5 : 13760 yn+5 − 5yn+4 + 5yn+3 − 10

3 yn+2 + 54yn+1 − 1

5un

= hf (tn+5,yn+5),

r = 6 : 14760 yn+6 − 6yn+5 − 15

2 yn+4 − 203 yn+3 + 15

4 yn+2 − 65yn+1 + 1

6un

= hf (tn+6,yn+6).

For r = 1 we recover the implicit Euler method.



2.4.4 The Local Discretization Error of LMM

Inserting the exact solution into (LMM), using the fact that y(t) satisfiesy ′(tn) = f (tn,y(tn)), we obtain after dividing by h

Tn+r(h) =1

h

r∑j=0

αjy(tn+j)− hr∑j=0

βjy′(tn+j)

.

Expanding all evaluations of y and y ′ at t = tn

y(tn+j) = y(tn) + jhy ′(tn) +(jh)2

2y ′′(tn) +

(jh)3

6y ′′′(tn) + . . .

y ′(tn+j) = y ′(tn) + jhy ′′(tn) +(jh)2

2y ′′′(tn) +

(jh)3

6y ′′′′(tn) + . . . ,

and collecting all terms with the same power of h, we obtain . . .



Tn+r(h) =1

h

r∑j=0

αj

y(tn) +

r∑j=0

(jαj − βj)

y ′(tn)

+ h

r∑j=0

(j2

2αj − jβj

)y ′′(tn)

+ · · ·+ hq

r∑j=0

(jq+1

(q + 1)!αj −

jq

q!βj

)y (q+1)(tn) +O(hq+1).

The LMM is consistent, i.e., Tn(h)→ 0 as h→ 0, if and only if

r∑j=0

αj = 0 as well asr∑j=0

jαj =r∑j=0

βj . (2.7)

Its order of consistency is p ∈ N whenever the first p + 1 terms in bracketsvanish.



2.4.5 Characteristic Polynomials

The polynomials formed with the coefficients {αj}rj=0 and {βj}rj=0 of aLMM, known as its characteristic polynomials

ρ(ζ) :=

r∑j=0

αjζj and σ(ζ) :=

r∑j=0

βjζj , (2.8)

play an important role in its analysis.

For an r-step method we have ρ ∈ Pr and, for an implicit r-step method,also σ ∈ Pr. When an r step method is explicit the degree of σ ist strictlyless than r.

The two consistency conditions (2.7) can be formulated in terms of the twocharacteristic polynomials as

ρ(1) = 0 und ρ′(1) = σ(1). (2.9)



Example: For the Adams-Moulton method with r = 2

yn+2 = yn+1 +h

12

(−f (tn,yn) + 8f (tn+1,yn+1) + 5f (tn+2,yn+2)

),

we have

α0 = 0, α1 = −1, α2 = 1, β0 =−1

12, β1 =

8

12, β2 =

5

12

and the characteristic polynomials are

ρ(ζ) = ζ2 − ζ, σ(ζ) =1

12

(5ζ2 + 8ζ − 1

).



Because of

α0 + α1 + α2 = 0,

0 · α0 + 1 · α1 + 2 · α2 − (β0 + β1 + β2) = 0,

0 · α0 +1

2· α1 +

4

2α2 − (0 · β0 + 1 · β1 + 2 · β2) = 0,

1

6(0 · α0 + 1 · α1 + 8α2)− 1

2(0 · β0 + 1 · β1 + 4 · β2) = 0,

1

24(0 · α0 + 1 · α1 + 16α2)− 1

6(0 · β0 + 1 · β1 + 8 · β2) =

5

8− 2

36= 0

this method is consistent of order p = 3.



2.4.6 Starting Values

When an r-step method (r > 1) is started, it is necessary to calculate,besides y0, the additional solution approximations y1,y2, . . . ,yr−1.

To maintain the method’s order p, it is sufficient to calculate these using anyother method of order at least p− 1.

Explanation: Since only a fixed number of steps (r − 1) is performedwith this startup method, independently of h, no power of h is lost in theaccumulation of step errors as h→ 0; for this reason the order of accuracydepends only on that of the method used from step r onwards.

Example: To generate y1 to start the midpoint rule (2.1), which is consistentof order p = 2, one can use the explicit Euler method without loss of order.



2.4.7 Predictor-Corrector Methods

A popular technique for avoiding the expensive solution of algebraic equa-tion systems required by implicit LMM consists of

(a) first computing an initial approximation yn+r of y(tn+r) using an explicitmethod with the same step size (predictor) and

(b) then inserting yn+r in place of yn+r in an implicit method (corrector).

Example: Combination of r = 1 Adams-Bashforth method (explicit Euler)with Adams-Moulton method with same step number (trapezoidal rule):

yn+1 = yn + hf (tn,yn), (predictor step),

yn+1 = yn +h

2(f (tn,yn) + f (tn+1, yn+1)) , (corrector step).

It can be shown: this predictor/corrector combination has consistency orderp = 2.



2.4.8 One-Step vs. Multistep Methods

Advantages of one-step methods:

• One-step methods are self-starting.

• Changing the stepsize h possible witout extra effort.

• Integration across discontinuities of the solution possible without lossof order if these are grid points.

Advantage of multistep methods: one evaluation of right hand side per timestep (important if f expensive to evaluate).



2.5 Convergence of One-Step Methods

Theorem 2.3 (Relation between local and global discretization error)Under the given assumptions (cf. Theorem 1.1 as well as (V2)) there holds

‖en‖ ≤(

(‖e0‖+ (tn − t0) max1≤j≤n

‖Tj‖)

exp(M(tn − t0)).

In particular: a one-step method is convergent if, and only if, it is consistent.

2.5 Convergence of One-Step Methods TU Bergakademie Freiberg, SS 2012


Theorem 2.4 (Total error of explicit OSM) Let the explicit one-step me-thod (OSM) be realized in floating point arithmetic with machine precisionε as

yn+1 = yn + hΦf (yn, tn;h) + εn+1, y0 = y0 + ε0.

If ‖εn‖ ≤ ε and ‖Tn‖ ≤ T for all n = 0, 1, . . ., then

‖y(tn)− yn‖ ≤(‖ε0‖+ (tn − t0)(T + ε

h ))

exp(M(tn − t0)).

Dilemma: For h large, in general (local error) T large; for h small T be-comes small, however ε

h grows, i.e., total error dominated by roundingerror.

This fact favors high order methods, which achieve small T for moderatevalues of the step size h.

2.5 Convergence of One-Step Methods TU Bergakademie Freiberg, SS 2012


2.6 Zero-Stability of Linear Multistep Methods

Convergence of one-step methods: Contribution of n-th step error Snbounded by eM(tend−t0)‖Sn‖, which still goes to zero as h→ 0.

This property of the error is called zero-stability, and it always holds forOSM. This is not the case for LMM.

Example: For the LMM yn+2 − 3yn+1 + 2yn = −hf (tn,yn) the localdiscretization error is

Tn+2(h) =h

2y ′′(tn) +O(h2).

We apply it to IVP

y′(t) = 0, y(0) = 0 with exact solution y(t) ≡ 0. (2.10)

For starting values y0 = y1 = 0 it indeed yields the exact solution.

2.6 Zero-Stability of Linear Multistep Methods TU Bergakademie Freiberg, SS 2012


However, setting y1 = h to simulate an error of order h when approximatingy(t1), one obtains for the approximation of the solution at t = 1 the followingvalues (h = 1/N ):

N yN

5 6.2

10 102.3

20 5.2× 104

Since y1 → 0 as h→ 0 these initial data are admissible.Nonetheless, yN fails to converge to y(tN ) = y(1) = 0.

The approximations satisfy the difference equation

yn+2 − 3yn+1 + 2yn = 0, n = 0, 1, 2, . . . , (2.11)

with solution

yn = (2n − 1)y1 − (2n − 2)y0 = 2y0 − y1 + 2n(y1 − y0), n ≥ 2.



2.6.1 Solution of Linear Difference Equations

Solutions of a homogeneous linear difference equation of order r

αryn+r + αr−1yn+r−1 + · · ·+ α0yn =r∑j=0

αjyn+j = 0 (2.12)

can be obtained by hypothesizing solutions of the form yn = ζn (”Ansatz“).Inserting in (2.12) and dividing by ζn leads to the condition

r∑j=0

αjζj = 0,

i.e., yn = ζn solves (2.12) if ζ is a root of the first characteristic polynomialρ(ζ) (cf. (2.8)).



If the characteristic polynomial ρ has the factored form

ρ(ζ) = αr(ζ − ζ1)(ζ − ζ2) · · · (ζ − ζr)

with r distinct roots {ζj}rj=1, then {ζnj }rj=1 constitute r linearly independentsolutions of (2.12). Due to the linearity and homogeneity of (2.12), anylinear combination

yn = c1ζn1 + c2ζ

n2 + · · ·+ crζ

nr , c1, . . . , cr arbitrary, (2.13)

of these is also a solution.Coefficients {cj}rj=1 determined by fixing first r values of yn:

c1 + c2 + · · ·+ cr = y0,

c1ζ1 + c2ζ2 + · · ·+ crζr = y1,

......

c1ζr−11 + c2ζ

r−12 + · · ·+ crζ

r−1r = yr−1.



Example: The characteristic equation of the difference equation (2.11) is

ρ(ζ) = ζ2 − 3ζ + 2 = (ζ − 1)(ζ − 2).

Conclusion: If ρ(ζ) has roots of modulus > 1, the associated LMM cannotbe convergent.

If two (or more) roots coincide, i.e., ζi = ζj , then ζni and ζnj are linearlydependent. A further, linearly independent solution is yn = nζn−1i .

For a root ζi of multiplicity three, yn = n2ζn−2i is linearly independent etc.

Example: The (consistent) LMM

yn+2 − 2yn+1 + yn =h

2

[f (tn+2,yn+2)− f (tn,yn)

].

applied to y′(t) = 0.

Conclusion: If ρ(ζ) has multiple roots of modulus 1, the associated LMMis also not convergent.



Example: The consistent LMM

yn+3 − 2yn+2 +5

4yn+1 −

1

4yn =

h

4f(tn,yn)

applied to (2.10).

Conclusion: For multiple roots of modulus > 1, the associated LMM isconvergent (linear growth is outweighed by exponential decay).

A LMM is called zero-stable, if the roots {ζj}rj=1 of its first characteristicpolynomial ρ(ζ) satisfy

(a) |ζj | ≤ 1 for all roots.

(b) |ζj | < 1 for all multiple roots.

Examples:• All Adams-methods are zero-stable.• The BDF-methods are zero-stable for 1 ≤ r ≤ 6.



These examples indicate that zero-stability is a necessary condition forconvergence of LMMs. In fact, this condition is also sufficient.

Theorem 2.5 (Dahlquist, 1956) For LMM applied to (IVP),

consistency + zero-stability ⇔ convergence .

Remarks 2.6(a) The first characteristic polynomial of a consistent LMM always posses-

ses the root ζ = 1 (cf. (2.9)).

(b) We will see that, besides zero-stability, there are further importantstability properties for numerical methods for IVPs.

(c) All single-step methods are zero-stable.



2.7 Stepsize Control

In practice, numerical methods for solving IVPs for ODEs almost neveremploy a constant stepsize h. Rather, it is much more efficient to adapt hto the local behavior of the solution y in the sense that

• rapid change in y necessitates small h• slow variation of y allows large h.

Here we will introduce one (of many) stepsize control schemes aimedat maintaining the (easily estimated) local discretization error Tn near aprescribed tolerance tol

‖Tn‖ ∼ tol, n = 1, 2, . . . .

For systems of ODEs (particularly when solution components vary stronglyin magnitude) absolute error tolerances for each solution component aswell as a global error tolerance are typically supplied.

2.7 Stepsize Control TU Bergakademie Freiberg, SS 2012


Theorem 2.3 states that bounding the local discretization error also resultsin bounding the global discretization error (the actual quantity of interest).

The local discretization error may be estimated by using two methods ofdifferent consistency orders, say, p and q where p < q, for computing ynfrom yn−1:

yn = yn−1 + hΦf (yn−1, tn−1;h) and yn = yn−1 + hΦf (yn−1, tn−1;h)

with associated local discretization errors

Tn =y(tn)− y(tn−1)

h− Φf (y(tn−1), tn−1;h) = O(hp),

Tn =y(tn)− y(tn−1)

h− Φf (y(tn−1), tn−1;h) = O(hq).

This implies

Tn − Tn = Φf (yn−1, tn−1;h)− Φf (yn−1, tn−1;h) = 1h (yn − yn).



Because Tn − Tn = Tn(1 +O(hq−p)) ∼ Tn, the quantity

1h‖yn − yn‖ ∼ ‖Tn‖

provides a (rough) estimate for ‖Tn‖.

Whenever 1h‖yn − yn‖ > tol, we discard the stepsize h, replacing it with h

determined by (h

h

)p= α

h tol

‖yn − yn‖. (∗)

This choice of h is motivated as follows:

discarded stepsize h: 1h‖yn − yn‖ ∼ ‖Tn‖ = chp +O(hp+1) ∼ chp,

desired stepsize h: tol = ‖Tn‖ = chp +O(hp+1) ∼ chp.

(α is added as a safety factor, e.g. α = 0.9.)



Starting again from yn−1, approximations yn and yn — now of y(tn−1 + h)

— are then generated and this process repeated until 1h‖yn − yn‖ ≤ tol

holds. When this is achieved, (∗) is used to propose a (trial) stepsize for thenext step (n→ n+ 1).

An elegant way to limit the computing effort is to compute yn and yn usingtwo RKM (of different orders) whose Butcher tableaus differ only in thevector b (same A and c). As a result, the slopes kj of their stages are alsothe same. Such pairs of RKMs are called embedded RKMs and denoted

c A

b>

b>

e.g.

0 0 0

1 0 0

1 0

1/2 1/2

.

Here the explicit Euler method (p = 1) is embedded in the improved Eulermethod (p = 2).



Another popular embedded pair is the Fehlberg 4(5)-formula:

0 0 0 0 0 0 014

14 0 0 0 0 0

38

332

932 0 0 0 0

1213

19322197 − 7200

219772962197 0 0 0

1 439216 −8 3680

513 − 8454104 0 0

12 − 8

27 2 − 35442565

18594104 − 11

40 025216 0 1408

256521974104 − 1

5 016135 0 6656

128252856156430 − 9

50255

consisting of two six-stage RKMs of orders 4 and 5.



A further example is the Dormand-Prince 4(5)-formula:

0 0 0 0 0 0 0 015

15 0 0 0 0 0 0

310

340

940 0 0 0 0 0

45

4445 − 56

15329 0 0 0 0

89

193726561 − 25360

2187644486561 − 212

729 0 0 0

1 90173168 − 355

33467325247

49176 − 5103

18656 0 0

1 35384 0 500

1113125192 − 2187

67841184 0

35384 0 500

1113125192 − 2187

67841184 0

517957600 0 7571

16695393640 − 92097

3392001872100

140

Here a six-stage RKM of order 4 is embedded in a seven-stage method oforder 5.


Documents

2 Numerical Methods for Initial Value Problemsernst/Lehre/Numerik_II/Folien/nade... · Numerical Analysis of Differential Equations 44 2 Numerical Methods for Initial Value Problems