A course in differential equations Chapter 2. Numerical ... · Numerical methods of solution 7 y =1 y =0 After we have taken one step of Euler’s method, we can tell to what extent

A course in differential equations

Chapter 2. Numerical methods of solution

We shall discuss the simple numerical method suggested in the last chapter, and explain several more sophisticatedones. We shall also deal with the problem of estimating error in the calculations. We make no attempt to producethe best routines possible, but just to present techniques which are reasonably simple to understand as well asreasonably effective in most circumstances.

1. Euler’s method

Suppose we want to find approximate values for the solution of the differential equation

y0 = f(t; y)

with initial conditiony(t0) = y0 :

The differential equation tells us that the instantaneous rate of change of y at time t can be calculated in terms ofy and t alone. Now if �t is any small interval of time we know that as a first order approximation we can write

y(t+�t):= y(t) + �t � y0(t)

in the sense that the difference between the right and left sides, for a fixed value of t, is essentially of order (�t)2.If y is a solution of the differential equation, this tells us that

y(t+�t):= y(t) + �t � f(t; y(t)) :

If t1 = t0 +�t we know therefore that

y(t1):= y1 = y(t0) + �t � f(t0; y(t0)) = y0 +�t � f(t0; y0) :

We can apply this reasoning once again. If t2 = t1 +�t = t0 + 2�t we obtain a further estimate

y(t2):= y2 = y1 +�t � f(t1; y1) :

Etc. If tn+1 = tn +�t = t0 + (n+ 1)�t we get an estimate yn+1 for y(tn+1) from the estimate yn for y(tn):

yn+1 = yn +�t � f(tn; yn) :

This technique for finding approximate values for the solution of a first order differential equation is the simplestof several similar ones. Each of them proceeds in this stepwise fashion, obtaining an estimate for y(t+�t) fromthat for y(t). This one is called Euler’s method. It has some great virtues: � It is simple to carry out. Doing it byhand is not impossible (although tedious and hence error prone). � It can be easily implemented on a computer,for example with a spread sheet. � The step from one estimate to the next is intuitive.

2. Euler’s method and slope fields

Euler’s method has a simple interpretation in terms of slope fields. In the figure below we are looking at thedifferential equation & initial condition

y0 = y � t; y(0) = 0:5 :

Numerical methods of solution 2

This is a linear equation and has the explicit solution

y = (t+ 1)� 0:5 et :

The differential equation tells us how to draw the slope field and the initial condition tells us where the graphstarts.

y = 1

y = 0

At the starting point we know that the slope of the graph is equal to f(0; 0:5) = 0:5. This means that the tangentline to the graph at the starting point is the line y

�(t) = 0:5+0:5 t. This tangent line and the graph of y(t) will lie

very close to each other for small values of t, and we can then use y�

as an estimate for y(t) for those values of t.After a while, however, it will cease to be a useful approximation. How can we get a better one? We pick someinterval �t and at t = �t we modify the tangent line. We must modify it by using only information available tous in the calculation so far, and in view of this it seems reasonable to use as the new approximation the straightline through (�t; y

�(�t)) whose slope agrees with the slope field at that point.

y = 1

y = 0

We then make new breaks in the approximation after every interval of size �t. As we take smaller and smallervalues of �t we get better approximations to the graph of the solution.

3. The error in Euler’s method


We expect that the error in the approximation we get from Euler’s method decreases as we let the step size �t

get smaller. We can get a good idea of how the error depends on the choice of �t by plotting in one picture theapproximations we get for different values of �t. Here we let �t = 1, 1=2, 1=4, etc. again with the differentialequation

y0 = y � t; y(0) = 0:5 :

y = 1

y = 0

It looks very much as though we have the error as we halve the step size. We can verify this in more detail bycomparing estimates for y(1) with the true value which we know to be (t+ 1)� 0:5 et at t = 1 which is equal to2� 0:5 e = 0:640859.

N estimated y(1) true y(1) error

2 0:875000 0:640859 0:2341414 0:779297 : : : 0:1384388 0:717108 : : : 0:076249

16 0:681036 : : : 0:04017732 0:661505 : : : 0:02064664 0:651328 : : : 0:010468

128 0:646130 : : : 0:005271256 0:643504 : : : 0:002645512 0:642184 : : : 0:0013251024 0:641522 0:640859 0:000663

This example leads to a guess which turns out to be true:

� The error in Euler’s method for estimating y(t) with a given differential equation and initial condition of yat a fixed value of t is proportional to the step size chosen.

The effect of this is to make Euler’s method extremely inefficient for serious calculation. It requires an enormousamount of calculation to achieve reasonable accuracy. Very roughly speaking, the amount of work it takes toachieve a certain level of accuracy is proportional to the accuracy you want. Thus if it takes N steps to get 1 digitof accuracy (answer within 1=10), it will take 10N to get an extra valid digit (within 1=100), 100N to get 3 validdigits, : : : , and 100000N to get 6.

The reason for the way error and step size interact for Euler’s method is not hard to understand at least informally.Euler’s method uses the estimate

y(t+�t):= y(t) + �t � y0(t) + terms of order (�t)2 :


That means that the error in making each step is essentially of order (�t)2. But for a fixed interval across whichwe want to calculate, the number of steps necessary is proportional to the inverse of �t, which makes it plausiblethat the overall error is of order (1=�t)(�t)2 = �t.

4. Illustrating errors graphically

We can use this to estimate the error in Euler’s method even when we don’t know the exact answer! For example,suppose we look at the equation

y0 = xy + 1; y(0) = 1

This is a linear equation, but if we were to use the usual formula for the solution we would see it produces anintegral we cannot find explicitly. Here is the estimate we get for y(1) with various values of N :

N y(1)

4 2:655151378 2:83657060

16 2:9418932932 2:9989870064 3:02876204128 3:04397338256 3:05166223512 3:05552774

1024 3:05746579

And here is a graph of the pairs (�; y(1)).

y = 2:5

3:0

3:5

� �!

Figure 4.1. A plot of y(1) versus step size.

What we can see from this plot is that the points (�; y(1)) are to a good approximation on a straight line. Thisis in complete agreement with the principle I stated above, that the error is proportional to �. But we can alsosee that to some extent we can predict what the end of the line is. This is because we can estimate from eachsuccessive pair what the slope of the purported straight line is—it ought to be about

yN � yN�1

�N ��N�1

for each N . Of course this is only an approximation, so we don’t expect any of these estimates to be exact. Hereis what we get in the runs above:


N �N = 1=N yN (1) yN (1)� yN�1(1) estimated slope

4 0:25000000 2:65515137 � �8 0:12500000 2:83657060 0:18141923 �1:45135383

16 0:06250000 2:94189329 0:10532269 �1:6851630932 0:03125000 2:99898700 0:05709371 �1:8269986864 0:01562500 3:02876204 0:02977504 �1:90560247

128 0:00781250 3:04397338 0:01521134 �1:94705210256 0:00390625 3:05166223 0:00768885 �1:96834562512 0:00195312 3:05552774 0:00386551 �1:979138771024 0:00097656 3:05746579 0:00193806 �1:98457249

The rough proportionality certainly shows up here, and the last column tells us roughly what the constant ofproportion is.

� One run of Euler’s method gives you no idea of the error involved.

� Two runs with values y1 and y2 for step sizes �1 and �2 give you the estimate

�y1 � y2

�1 ��2

��

for the error with step size �.

5. How errors propagate

The exact way in which errors propagate as we carry out Euler’s method can be rather complicated. But a fairlysimple argument can lead to some idea of the worst things can go, at least under mild conditions on the equation.Suppose y(t) is the exact solution to

y0 = f(t; y); y(t0) = y0 :

and let yn be the estimate for y(tn) obtained by Euler’s’ method, where tn = t0 + n�t. Let "n = y(tn) � yn bethe error in the method, the difference between the true and the estimated methods. Then

"n+1 = y(tn+1)� yn+1

= y(tn+1)� [yn +�t � f(tn; yn)]= [y(tn)� yn] + [y(tn+1)� y(tn)��t � f(tn; y(tn))] + �t [f(tn; y(tn))� f(tn; yn)]

= "n + [y(tn+1)� y(tn)��t � f(tn; y(tn))] + �t [f(tn; y(tn))� f(tn; yn)] :

If we assume that jy00j is bounded in the range we are looking at, say by A, then we have

jy(tn+1)� y(tn)��t � f(tn; y(tn))j � A(�t)2 :

The quantityjf(tn; y(tn))� f(tn; yn)j

will be bounded byB jy(tn)� ynj = B "n

if ��@f@y�� B

in the range we are looking at and then we have

"n+1 � "n +B�t "n +A (�t)2

= "n(1 + B�t) + A(�t)2 :


this leads to"1 � A(�t)2

"2 � (1 +B�t)A(�t)2 + A(�t)2

"3 � (1 +B�t)2A(�t)2 + (1 +B�t)A(�t)2 +A(�t)2

: : :

"n � A(�t)2[(1 +B�t)n�1 + � � � + (1 +B�t)2 + (1 +B�t) + 1]

= A(�t)2�(1 + B�t)n � 1

(1 + B�t)� 1

�

=A

B�t [(1 +B�t)n � 1]

=A

B�t [eBn�t � 1]

=A

B�t [eB(tn�t0) � 1]

if tn = t0 + n�t, since(1 + x) � ex

for all x � 0.

In other words, we can see that

� Under very mild restrictions, in Euler’s method the error for a fixed t is at worst proportional to �t and theconstant of proportionality grows at worst exponentially with t.

This is really a worst case scenario under the restrictions necessary. Most of the time the constant will behavemuch better. But for equations whose solutions grow exponentially the estimate will be reasonably close to whathappens.

The restrictions are mild, but there are cases where they don’t hold. For the equation

y0 = y2

the conditions on @f=@y are not satisfied for the solutions, which all go off to infinity in bounded time.

6. A better method

Euler’s method is quite inefficient. Because the error is essentially proportional to the step size, if we want to halveour error we must double the number of steps taken across the same interval. So ifN steps give us one significantfigure, then 10N give us 2 figures, 100N give us 3, 100000N give us 6, and 10n�1N give us n figures. Not toogood, because even a fast computer will think a bit about doing several billion operations. The method can beimproved dramatically by a simple modification. Look again at what goes on in one step of Euler’s method. Saywe are at step n. We know tn and yn and want to calculate tn+1, yn+1. In Euler’s method we assuming that theslope across the interval [tn; tn +�t] = [tn; tn+1] is what it is at tn. Of course it is not true, but it is very roughlyOK.


y = 1

y = 0

After we have taken one step of Euler’s method, we can tell to what extent the assumption on the slope was false,by comparing the slope of the segment coming from the left at (tn+1; yn+1) with the value f(Tn+1; yn+1) thatit ought now to be. The new method will use this idea to get a better estimate for the slope across the interval[tn; tn+1].

One step of the new method goes like this: Start at tn with the estimate yn. Do one step of Euler’s method to getan approximate value

y�= yn +� � F (tn; yn)

for yn+1. Calculate what the slope should now be at the point (tn+1; y�). It should be

f(xn+1; y�)

We now make a modified guess as to what the slope across [tn; tn+1] should have been by taking the average ofthese values f(tn; yn) and f(tn+1; y�). In other words

� In one step of the new method we calculate the sequence

s0 = F (xn; yn)

y�= yn +� � s0

xn+1 = xn +�

s1 = F (xn+1; y�)

yn+1 = yn +� �hs0 + s1

2

i

This is in contrast with one step of Euler’s:

s0 = F (xn; yn)

yn+1 = yn +� � s0xn+1 = xn +�

In the new method, we are doing essentially twice as much work in one step, but it should be a somewhat moreaccurate. It is, and dramatically so.

Let’s see, for example, how the new method, which is called the improved Euler’s method, does with the simpleproblem

y0 = y � t; y(0) = 0:5

we looked at before. Here is how one run of the new method goes in covering the interval [0; 1] in 4 steps:


t y s0 y�

s1

0:00 0:500000 0:500000 0:625000 0:3750000:25 0:609375 0:359375 0:699219 0:1992190:50 0:679199 0:179199 0:723999 �0:0260010:75 0:698349 �0:051651 0:685436 �0:3145641:00 0:652572

And here is how the estimates for y(1) compare for different step sizes:


2 0:679688 0:640859 0:0388284 0:652572 : : : 0:0117138 0:644079 : : : 0:003220

16 0:641703 : : : 0:00084432 0:641075 : : : 0:00021664 0:640914 : : : 0:000055

128 0:640873 : : : 0:000014256 0:640863 : : : 0:000003512 0:640860 : : : 0:0000011024 0:640859 0:640859 0:000000

Halving the step size here cuts down the error by a factor of 4. This suggests:

� The error in the improved Euler’s method is proportional to �2.

The new method, which is called either the improved Euler’s method or the Runge-Kutta method of order 2, ismuch more efficient than Euler’s. If it takes N intervals to get 2 decimal accuracy, it will take 10N to get 4decimals, 100N to get 6 (as opposed to 100000N in Euler’s method). Very roughly, to get accuracy of " it takes1=p" steps.

7. Relations with numerical calculation of integrals

If y is a solution ofy0(t) = f(t; y(t))

then for any interval [tn; tn+1] with tn+1 = tn +�t we can integrate to get

y(tn+1)� y(tn) =

Z tn+1

tn

f(t; y(t)) dt :

This does not amount to a formula for y(tn+1) in terms of y(tn) because the right hand side involves the unknownfunction y(t) throughout the interval [tn; tn+1]. In order to get from it an estimate for y(tn+1) we must use somesort of approximation for the integral. In Euler’s method we set

Z tn+1

tn

f(t; y(t)) dt:= f(tn; y(tn))�t :

We then replace y(tn) by its approximate value yn to see how one step of Euler’s method goes.

In the special case where f(t; y) = f(t) does not depend on y, the solution of the differential equation & initialcondition

y0 = f(t); y(t0) = y0

is given by a definite integral

y(tn) = y0 +

Z tn

t0

f(t) dt


and Euler’s method is equivalent to the rectangle rule for estimating the integral numerically. Recall that therectangle rule approximates the graph of f(t) by a sequence of horizontal lines, each one agreeing with the valueof f(t) at its left end.

In this special case it is simple to see geometrically why the overall error is roughly proportional to the step size.

In the improved Euler’s method we approximate the integral

Z tn+1

tn

f(t; y(t)) dt

by the trapezoid rule, which uses the estimate

�t

2[f(tn; y(tn)) + f(tn+1; y(tn+1))]

for it. We don’t know the exact values of y(tn) or y(tn+1), but we approximate them by the values yn andyn +�t f(tn; yn).

In the special case f(t; y) = f(t) the improved Euler’s method reduces exactly to the trapezoid rule. Again, it iseasy to visualize why the error over a fixed interval is proportional to the square of the step size.

8. Runge-Kutta

Neither of the two methods discussed so far is used much in practice. The simplest method which is in factpractical is one called Runge-Kutta method of order 4. It involves a more complicated sequence of calculations ineach step. Let � be the step size. We calculate in succession:


y�= yn +

h�2

is�

x�= x+

h�2

i

s��

= f(x�; y

�)

y��

= yn +h�2

is��

s��

= f(x�; y

��)

xn+1 = x+�

y��

= yn +� � s��

s��

= f(xn+1; y��)

yn+1 = yn +� �hs

�+ 2s

��+ 2s

��+ s

��

6

i

The purpose of using � here is that the numbers like y�

etc. are not of interest beyond the immediate calculationthey are involved in.

You wouldn’t want to do this by hand, but again it is not too bad for a spread sheet. Here is a table of thecalculations for

y0 = y � t; y(0) = 0:5

with 4 steps.

t y s�

y�

s��

y��

s��

y��

s��

0:00 0:500000 0:500000 0:562500 0:437500 0:554688 0:429688 0:607422 0:3574220:25 0:607992 0:357992 0:652740 0:277740 0:642709 0:267709 0:674919 0:1749190:50 0:675650 0:175650 0:697607 0:072607 0:684726 0:059726 0:690582�0:0594180:75 0:691521�0:058479 0:684211�0:190789 0:667672�0:207328 0:639689�0:3603111:00 0:640895

And here is how the estimates for y(1) compare for different step sizes. The method is so accurate that we mustuse nearly all the digits of accuracy possible:


2 0:64132690429688 0:64085908577048 0:000467818526404 0:64089503039934 : : : 0:000035944628868 0:64086157779163 : : : 0:00000249202116

16 0:64085924982971 : : : 0:0000001640592332 0:64085909629440 : : : 0:0000000105239364 0:64085908643684 : : : 0:00000000066636

128 0:64085908581240 : : : 0:00000000004192256 0:64085908577311 : : : 0:00000000000263512 0:64085908577064 : : : 0:000000000000161024 0:64085908577049 0:64085908577048 0:00000000000001

Here halving the step size reduces the error by a factor of 16. This suggests:

� The error in RK4 is proportional to �4.


Euler’s method and the improved Euler’s method arise from using the rectangle rule and the trapezoid rule forestimating the integral Z t

n+1

tn

f(t; y(t)) dt :

RK4 uses a variant of Simpson’s Rule for calculating integrals and reduces to it in the special case when f(t; y)doesn’t depend on y.

9. Remarks on how to do the calculations

When implementing these by hand, which you might have to do every now and then, you should set thecalculations out neatly in tables. Here is what it might look like for the improved Euler’s method applied to thedifferential equation

y0 = 2xy � 1

with step size � = 0:1.

x y s�= f(x; y) y

�= y +� � s

�s��

= f(x+�; y�)

0:000000 1:000000 �1:000000 0:900000 �0:8200000:100000 0:909000 �0:818200 0:827180 �0:6691280:200000 0:834634 �0:666147 0:768019 �0:539189

etc.

Of course this is redundant and tedious work, and ideally suited to a computer. You can transfer the idea of tablesto a spread sheet without much trouble. And here is a simple Pascal program to implement Euler’s method forthe same differential equation.

program euler;

var x, y, xf, h: real;i, N : integer;

function f(x, y:real): real;

beginf := 2*x*y - 1;

end;

function next_y(x, y, h: real):real;

beginnext_y := y + h*f(x, y);

end;

beginx := 0.0; xf := 1.0; y := 1.0; N := 10;h := (xf - x)/N;for i := 1 to N do beginwriteln(x:8:6, ’ ’, y:8:6);y := next_y(x, y, h);x := x + h;

end;writeln(x:8:6, ’ ’, y:8:6);


end.

This is very convenient, because to change the differential equation involves very little change in the program(but it does involve recompiling).

10. How to estimate the error

When you set out to approximate the solution of a differential equation

y0 = f(x; y)

by a numerical method, you are given initial conditions y(x0) = y0 and a range of values [x0; xf ] (f for ‘final’)over which you want to approximate the solution. The particular numerical method you are going to use isusually specified in advance, and your major choice is then to choose the step size—the increment by which x

changes in each step of the method.

The basic idea is that if you choose a small step size your approximation will be closer to the true solution, butat the cost of a larger number of steps. What is the exact relationship between step size and accuracy? We havediscussed this already to some extent, but now we shall look at the question again. Some explicit data can giveyou a feel for how things go. Here is the outcome of several runs of Euler’s method for solving the equation

y0 = 2xy � 1; y(0) = 1

over the interval [0; 1].

Number of steps Step size Estimated value of y(1)

4 0.250000 0.4267588 0.125000 0.540508

16 0.062500 0.60867232 0.031250 0.64676364 0.015625 0.667026

128 0.007812 0.677495256 0.003906 0.682819512 0.001953 0.685503

1024 0.000977 0.686851

This table suggests that if we keep making the step size smaller, then the approximations to y(1) will converge tosomething, but does not give a good idea, for example, of how many steps we would need to take if we wantedan answer correct to 4 decimals. To get this we need some idea of how rapidly the estimates are converging. Theway to do this most easily is to add a column to record the difference between an estimate and the previous one.

Number of steps Step size Estimated value of y(1) Difference from previous estimate

4 0.250000 0.426758 0.4267588 0.125000 0.540508 0.113751

16 0.062500 0.608672 0.06816432 0.031250 0.646763 0.03809164 0.015625 0.667026 0.020263

128 0.007812 0.677495 0.010469256 0.003906 0.682819 0.005323512 0.001953 0.685503 0.002685

1024 0.000977 0.686851 0.001348


The pattern in the last column is simple. Eventually, the difference gets cut in half at each line. This means thatat least when h is small, the difference is roughly proportional to the step size. Thus we can extrapolate entries inthe last column to be

0:000674; 0:000337; 0:000169; 0:000084; : : : :

But the total error should be the sum of all these differences, which can be written as

0:001348 (1=2+ 1=4 + 1=8 + 1=16 + : : :) = 0:001348!

It is not in fact necessary to look at a whole sequence of runs. We can summarize the discussion more simply: ifwe make two runs of Euler’s method, the error in the second of the two estimates for y(xf) will be (roughly) thesize of the difference between the two estimates.

The reason the error can be estimated in this way is because the error in Euler’s method is proportional to thestep size. Suppose we have made two runs of the method, one with step size h and another with step size h=2.Then because of the proportionality, and the meaning of error, we know that

true answer := estimate h +Ch

true answer := estimate h=2 +C(h=2)

for some constantC . Here estimateh means the estimate corresponding to step size h. If we treat the approximateequalities as equalities, we get two equations in the two unknowns true answer and C . We can solve them to get

true answer := 2 estimate h=2 � estimate h

= estimate h=2 + (estimate h=2 � estimate h)

error in estimate h=2:= estimate h=2 � estimate h

In other words, one run of Euler’s method will tell us virtually nothing about how accurate the run is, but tworuns will allow us to get an idea of the error and an improved estimate for the value of y(xf) as well.

The same is true for any of the other methods, since we know that in each case the error is roughly proportionalto some power of h—for improved Euler’s it is h2 and for RK4 it is h4. In general, if the error is proportional tohk and we make two runs of step size h and h=2 we get

true answer := estimate h + Chk

true answer := estimate h=2 +C(h=2)k

2k true answer := 2k estimate h=2 + Chk

and subtract getting

(2k � 1) true answer := 2k estimate h=2 � estimate h

true answer :=

2k estimate h=2 � estimate h

2k � 1

= estimate h=2 +estimate h=2 � estimate h

2k � 1

and finally:

� If you do one run with step size h and another of size h=2 with a method of order k then

error in estimate h=2:=

estimate h=2 � estimate h

2k � 1

true answer := estimate h=2 + error in estimate h=2


The earlier formula is the special case of this with k = 1.

11. Step size choice

When we use one of the numerical methods to approximate the solution of the differential equation, we usuallyhave ahead of time some idea of how accurate we want the approximation to be—to within 6 or 7 decimals ofaccuracy, say. The question is, how can we choose the step size in order to obtain this accuracy? The discussionabove suggests the following procedure:

� Make two runs of the method over the given range, with step size h and then with step size h=2.

We get two estimates for y(xf), estimate h and estimate h=2. If the method has error proportional to hk (i.e. if theerror is of order k) then we know that the error in estimate h is approximately

error in estimate h:=

estimate h � estimate h=2

1� 2�k:

If we multiply the step size by � we multiply the error by �k , so if we want an error of about �:

� Solve this equation to get the new step size h�:

�h�

h

�k

=�

error in estimate h

h� = h

��

error in estimate h

�1=k

The procedure looks tedious, but plausible: we choose some initial step size h which we judge (arbitrarily) to bereasonable. We do two runs at step sizes h and h=2 to see how accuracy depends on step size (to get the constantof proportionality). We then calculate the step size h needed to get the accuracy we want, and do a third run atthe new size.

Example. Here is a typical question:

In solving the differential equationy0 = xy + 1; y(0) = 1

by RK4, with step sizes 0:0625 and 0:03125, the estimates 3:05940 72706 92 and 3:05940 73971 09 for y(1) werecalculated. (a) Make an estimate of the error involved in these calculated values. (b) What step size should bechosen to obtain 16-decimal accuracy?

One thing to notice is that we don’t need to know at all what the differential equation is, or even the exact methodused. The only important thing to know is the order of the method used, which is k = 4.

We sety1 = 3:05940 72706 92

y2 = 3:05940 73971 09

The formula for the error in the second value, say, is simple. We estimate it to be

y2 � y1

2k � 1=

0:00000 01264 17

15= 0:00000 00084 28

To figure out what step size to use to get a given accuracy, we must know how the error will depend on step size.It is

c�4


for some constant c, where � is the step size. In the second run the step size is 0:03125, so we estimate

c =0:00000 00084 28

(0:03125)4= 0:008829

To get the step size we want we solve

c�4 = 0:008829�4 = 10�16; � =

�10�16

0:008829

�1=4

=10�4

0:3017= 0:00033

12. Summary of the numerical methods introduced so far

We have seen three different methods of solving differential equations by numerical approximation. In choosingamong them there is a trade-off between simplicity and efficiency. Euler’s method is relatively simple to under-stand and to program, for example, but almost hopelessly inefficient. The third and final method, the order 4method of Runge-Kutta, is very efficient but rather difficult to understand and even to program. The improvedEuler’s method lies somewhere in between these two on both grounds.

Each one produces a sequence of approximations to the solution over a range of x-values, and proceeds from anapproximation of y at one value of x to an approximation at the next in one more or less complicated step. Ineach method one step goes from xn to xn+1 = xn +�. The methods differ in how the step from yn to yn+1 isperformed. More efficient single steps come about at the cost of higher complexity for one step. But for a givendesired accuracy, the overall savings in time is good.

Method The step from (xn; yn) to (xn+1; yn+1) Overall error

Euler’s s�

= f(xn; yn) proportional to �

yn+1 = yn +� � s�

xn+1 = xn +�

improved Euler’s s�

= f(xn; yn) proportional to �2

(RK of order 2) y�

= yn +� � s�

xn+1 = xn +�

s��

= f(xn+1; y�)

yn+1 = yn +� �hs�+ s

��

2

i

Runge-Kutta of order 4 s�

= f(xn; yn) proportional to �4

y�

= yn +h�2

is�

x�

= x+h�2

i

s��

= f(x�; y

�)

y��

= yn +h�2

is��

s��

= f(x�; y

��)

xn+1 = x+�

y��

= yn +� � s��

s��

= f(xn+1; y��)

yn+1 = yn +� �hs�+ 2s

��+ 2s

��+ s

��

6

i


In practice, of course, you cannot expect to do several steps of any of these methods except by computer. Youshould, however, understand � how to one step of each of them; � how to estimate errors in using them; � howto use error estimates to choose efficient step sizes.

Numerical methods are often in practice the only way to solve a differential equation. They can be of amazingaccuracy, and in fact the methods we have described here are not too different from those used to send spacevehicles on incredibly long voyages. But often, too, in practice you want some overall idea about how solutionsof a differential equation behave, in addition to precise numerical values. There is no automatic way of obtainingthis.

In all these methods, one of the main problems is to choose the right step size. The basic principle in doing thisis that:

� Two runs of any method will let you estimate the error in the calculation, as well as decide what step size touse in order to get the accuracy you require.

13. The problem of stability

Although we shall not show how to solve it, we want to point out here a problem which, although somewhattechnical, does cause some trouble in practice. First consider the equation & initial conditions

y0 = c� y=�; y(0) = 0 :

This is linear with constant coefficients and has the exact solution

y = c� � c�e�t=� = c�(1� e�t=� ) :

Assume that � > 0. Then this is the sum of two terms. One is the constant c� and the other is exponentiallydecaying. The rate of decay—the time it takes to reduce any fixed factor, for example—is proportional to � . Thequantity � is called the relaxation time of the equation, and when it is short the function y reaches equilibrium—orrelaxes—quickly.

Solve this by Euler’s method with step size �. We get

t0 = 0

y0 = 0

tn+1 = tn + c�

yn+1 = yn +�(c� yn=�)

= c�+ yn(1��=�)

The problem is that we can get some rather strange behaviour if � is not chosen carefully. Here is the graph weget with � not quite small enough.


Figure 13.1. The behaviour of Euler’s method for a stiff equation with threshold step size.

You can see what happens—all the solutions move very rapidly towards the equlibrium, and this means thattheir slopes are very steep not too far from the equilibrium solution. You have to take very small steps in Euler’smethod to take this into account. Here is what happens when � is just a bit smaller. The process does converge,but very slowly.

Figure 13.2. The behaviour of Euler’s method for a stiff equation with step size barely below threshold.

We can explain more precisely what we see. Let � = 1��=� for convenience. Then

y0 = 0

y1 = c�

y2 = c�(1 + �)

y3 = c�(1 + � + �2)

yn = c�(1 + � + �2 + � � � �n�1)

Now as n!1 this is supposed to converge to c� , just like the exact solution. This can’t possibly happen unlessj�j < 1 or

j1��=� j < 1

Since � and � are both positive, j1��=� j < 1 means that

1��=� > �1; � < 2�

If this condition does not hold then 1 � �=� < �1 and the numbers yn will just oscillate back and forth withlarger and larger swings. In particular, when the relaxation time � is relatively small we must choose the stepsize small � also if Euler’s method is to make any sense.

This is a paradoxical situation. When � is small then the true solution will be just a constant for all large values oft, the simplest possible sort of behaviour. But we must still make very small steps in order to have Euler’s methodproduce reasonable numbers. This means that we will be working very hard to produce a lot of apparently simpledata.

Unfortunately, this sort of thing is fairly common in real-life situations—whenever there are components of aphysical system whose change takes place on scales widely different. Here the two components undergo nochange and rapid decay. Such systems are said to be stiff. The idea is that in trying to apply Euler’s method to thesystem we have to make small steps in order to keep the rapidly changing part from causing our calculation tovary wildly, even though it is only the slowest changing parts we can actually measure. How can we do better?


This is a complicated topic. There is a simple way to take care of the difficulty in the case of Euler’s methodwhich does suggest how to proceed in general. We perform something called the backward Euler’s method. Theforward Euler’s method (the one we are familiar with) assumes the slope across a time segment to be what it isat the beginning:

yn+1 = yn +� f(tn; yn)

The backward one setsyn+1 = yn +� f(tn+1; yn+1)

Of course this doesn’t give an immediate expression for yn+1 since it occurs on both sides. The equation has tobe solved for yn+1. In the example above we get

yn+1 = yn +�(1� yn+1=�)

yn+1 =yn +�

1 +�=�

which will always converge as long since � and � are positive. Note that when �=� is small

1

1 + �=�

:= 1��=�

so the new rule and the old one are consistent.

Each of the other methods we have seen has backwards versions too.

14. Rounding errors

Computers can only deal with numbers of limited accuracy, normally about 16 digits of precision. They cannotcalculate even the product and sums of numbers exactly, much less calculate the exact values of functions likesin. Errors in computations due to this limited accuracy are called rounding errors.

Rounding errors are not usually a problem in the methods we have seen, except in the circumstance where Euler’smethod is used to obtain extremely high accuracy, in which case it will require enough steps for serious error tobuild up. But then Euler’s method should never be used for high accuracy anyway.

There is one circumstance, however, in which there will be a problem. Consider this differential equation:

y0 = cos t � y; y(0) = y0

The general solution isy = (sin t� cos t)=2 + (y0 + 1=2)ect

Suppose c > 0. Then whenever y0 6= �1=2 the solution will grow exponentially. In the exceptional case y0 = 1=2it will simply oscillate. Here, however, is the graph of a good numerical aproximation matching initial conditiony(0) = �1=2:


Figure 14.1. How rounding errors affect the calculations for a stiff equation.

Rounding errors are inevitably made at some point in almost any computer calculation, and as soon as thathappens the estimated y(x)will make an almost certainly irreversible move from the graph of the unique boundedsolution to one of the nearby ones with exponential growth. From that point on it must grow unbounded, nomatter how well it has looked so far. This is not a serious problem in the following sense: in the real world, wecan never measure initial conditions exactly, so that we can never be sure to hit the one bounded solution exactly.Nor can nature. Since the bounded solution is unstable in the sense that nearby solution eventually divergeconsiderably from it, we should not expect to meet with it in physical problems.

� Unstable solutions do not realistically occur in nature. Rounding errors simulate the approximate nature ofour measurements.

Another way to say this:

� Because of rounding errors, the best we can hope to do with machine computations is to find the exact answerto a problem near to the one that is originally posed.

Appendix: Euler’s method in C 20

Appendix. Euler’s method in C

Below is a complete program in C to implement Euler’s method for the equation y0 = y. It includes routines toscan the command line to read values of x0, xf , y0, N . It also prints out the run data as the run proceeds.

In order to change this program to handle another differential equation, only the function f(x; y) has to bechanged, and the program recompiled.

#include <stdio.h>#include <math.h>

/* global variables ------------------------------------------------ */

double xinit, xf, yinit;int N;

/* function f(x, y) ------------------------------------------------ */

double f(x, y)double x, y;

{return(y);

}

/* options --------------------------------------------------------- */

void usage()

{fprintf(stderr, "usage: euler -xi <xinit> -xf <xf> -yi <yinit> -N <N>\n");exit(1);

}

#define OPTNO 4enum {Optxinit, Optxf, Optyinit, OptN};

char opt_string[][16] = {"-xi", "-xf", "-yi", "-N"};

int get_opt(s)char *s;

{int i;for (i=0;i<OPTNO;i++) {

if (strcmp(s, opt_string[i]) == 0) return(i);}fprintf(stderr, "Invalid option [%s]\n", s);usage();

}

double atof();


int read_command_line(argc, argv)int argc;char **argv;

{int opt;char *a;

argv++;argc--;

while (argc > 0) {opt = get_opt(a = *argv++);argc--;switch(opt) {

case Optxinit:a = *argv++;argc--;xinit = atof(a);break;

case Optxf:a = *argv++;argc--;xf = atof(a);break;

case Optyinit:a = *argv++;argc--;yinit = atof(a);break;

case OptN:a = *argv++;argc--;N = atof(a);break;

default: usage();}

}}

/* ----------------------------------------------------------------- */

main(argc, argv)int argc;char **argv;

{int i;double x, y, h;

if (argc > 1) {


read_command_line(argc, argv);}else {

usage();}x = xinit; y = yinit;h = (xf - xinit)/((double) N);for (i=0;i<N;i++) {

printf("%8.6lf %8.6lf\n", x, y);y = y + h*f(x, y);x = x + h;

}printf("%8.6lf %8.6lf\n", x, y);exit(0);

}

Documents

A course in differential equations Chapter 2. Numerical ... · Numerical methods of solution 7 y =1 y =0 After we have taken one step of Euler’s method, we can tell to what extent