IVP for ODEsbased on

Scientific Computing: An Introductory Survey by Michael T. Heathhttp://www.cse.uiuc.edu/heath/scicomp/

andApplied Numerical Methods for Engineers and Scientists by Singiresu S. Rao

http://cwx.prenhall.com/bookbind/pubbooks/rao/

Małgorzata Stojek,PhD Dr. Es-sciences techniques EPFL

Cracow University of Technology

CUT January 2011

SS Rao (Prentice Hall 2002) IVP for ODEs CUT’2011 1 / 14

Initial Value ProblemProblem Definition

Find the solution of the first order differential equation

y ′ = y + 2x − 1

over the interval0 ≤ x ≤ 1

with the initial condition

y (x)|x=x0=0 = 1

SS Rao (Prentice Hall 2002) IVP for ODEs CUT’2011 2 / 14

Exact Solutiony’=y+2x-1, y(x=0)=1

y(x) = −1− 2x + 2ex

0.0 0.2 0.4 0.6 0.8 1.01.0

1.5

2.0

x

y

y(0.0) = 1.0000y(0.1) = 1.0103y(0.2) = 1.0428y(0.3) = 1. 0997y(0.4) = 1.1836y(0.5) = 1. 2974y(0.6) = 1. 4442y(0.7) = 1. 6275y(0.8) = 1. 8511y(0.9) = 2.1192y(1.0) = 2.4366

SS Rao (Prentice Hall 2002) IVP for ODEs CUT’2011 3 / 14

Euler’s Method

ODE:

y ′ = f (x , y)

y ′ = y + 2x − 1

Euler’s method (equivalent to Taylor’s series method of order 1):

yi+1 = yi + h f (xi , yi ) ↔ yi+1 (xi + h) = yi (xi ) + hy ′(xi )

where h is the uniform step sizeforward (explicit) iteration:

f (x , y) = y + 2x − 1⇓

yi+1 = yi + h (yi + 2xi − 1)Recall: backward (implicit) iteration

yi+1 = yi + h (yi+1 + 2xi+1 − 1)

SS Rao (Prentice Hall 2002) IVP for ODEs CUT’2011 4 / 14

Euler’s Method

ODE:

y ′ = f (x , y)

y ′ = y + 2x − 1Euler’s method (equivalent to Taylor’s series method of order 1):

yi+1 = yi + h f (xi , yi ) ↔ yi+1 (xi + h) = yi (xi ) + hy ′(xi )

where h is the uniform step size

forward (explicit) iteration:

f (x , y) = y + 2x − 1⇓

yi+1 = yi + h (yi + 2xi − 1)Recall: backward (implicit) iteration

yi+1 = yi + h (yi+1 + 2xi+1 − 1)

SS Rao (Prentice Hall 2002) IVP for ODEs CUT’2011 4 / 14

Euler’s Method

ODE:

y ′ = f (x , y)

y ′ = y + 2x − 1Euler’s method (equivalent to Taylor’s series method of order 1):

yi+1 = yi + h f (xi , yi ) ↔ yi+1 (xi + h) = yi (xi ) + hy ′(xi )

where h is the uniform step sizeforward (explicit) iteration:

f (x , y) = y + 2x − 1⇓

yi+1 = yi + h (yi + 2xi − 1)

Recall: backward (implicit) iteration

yi+1 = yi + h (yi+1 + 2xi+1 − 1)

SS Rao (Prentice Hall 2002) IVP for ODEs CUT’2011 4 / 14

Euler’s Method

ODE:

y ′ = f (x , y)

y ′ = y + 2x − 1Euler’s method (equivalent to Taylor’s series method of order 1):

yi+1 = yi + h f (xi , yi ) ↔ yi+1 (xi + h) = yi (xi ) + hy ′(xi )

where h is the uniform step sizeforward (explicit) iteration:

f (x , y) = y + 2x − 1⇓

yi+1 = yi + h (yi + 2xi − 1)Recall: backward (implicit) iteration

yi+1 = yi + h (yi+1 + 2xi+1 − 1)

SS Rao (Prentice Hall 2002) IVP for ODEs CUT’2011 4 / 14

Euler’s Methody’=y+2x-1, y(x=0)=1

assuming h = 0.1 we have 10 iterations for 0 ≤ x ≤ 1

yi+1| xi+0.1 = yi + 0.1 (yi + 2xi − 1)(xi , yi ) i → (xi+1, yi+1) i+1

initial value (0, 1) 0 → y1 = 1+ 0.1 (1+ 2 · 0− 1) = 1→ (0.1, 1) 1(0.1, 1) 1 → y2 = 1+ 0.1 (1+ 2 · 0.1− 1) = 1.02→ (0.2, 1.02) 2(0.2, 1. 02) 2 → y3 = 1.02+ 0.1 (1.02+ 2 · 0.2− 1) = 1.062→ (0.3, 1.062) 3. . .(0.9, 1.915 8) 9 →y10 = 1.915 8+ 0.1 (1.915 8+ 2 · 0.9− 1) = 2.187 4→ (1.0, 2.187 4) 10 → STOP

SS Rao (Prentice Hall 2002) IVP for ODEs CUT’2011 5 / 14

Euler’s Methody’=y+2x-1, y(x=0)=1

assuming h = 0.1 we have 10 iterations for 0 ≤ x ≤ 1

yi+1| xi+0.1 = yi + 0.1 (yi + 2xi − 1)(xi , yi ) i → (xi+1, yi+1) i+1

initial value (0, 1) 0 → y1 = 1+ 0.1 (1+ 2 · 0− 1) = 1→ (0.1, 1) 1

(0.1, 1) 1 → y2 = 1+ 0.1 (1+ 2 · 0.1− 1) = 1.02→ (0.2, 1.02) 2(0.2, 1. 02) 2 → y3 = 1.02+ 0.1 (1.02+ 2 · 0.2− 1) = 1.062→ (0.3, 1.062) 3. . .(0.9, 1.915 8) 9 →y10 = 1.915 8+ 0.1 (1.915 8+ 2 · 0.9− 1) = 2.187 4→ (1.0, 2.187 4) 10 → STOP

SS Rao (Prentice Hall 2002) IVP for ODEs CUT’2011 5 / 14

Euler’s Methody’=y+2x-1, y(x=0)=1

assuming h = 0.1 we have 10 iterations for 0 ≤ x ≤ 1

yi+1| xi+0.1 = yi + 0.1 (yi + 2xi − 1)(xi , yi ) i → (xi+1, yi+1) i+1

initial value (0, 1) 0 → y1 = 1+ 0.1 (1+ 2 · 0− 1) = 1→ (0.1, 1) 1(0.1, 1) 1 → y2 = 1+ 0.1 (1+ 2 · 0.1− 1) = 1.02→ (0.2, 1.02) 2

(0.2, 1. 02) 2 → y3 = 1.02+ 0.1 (1.02+ 2 · 0.2− 1) = 1.062→ (0.3, 1.062) 3. . .(0.9, 1.915 8) 9 →y10 = 1.915 8+ 0.1 (1.915 8+ 2 · 0.9− 1) = 2.187 4→ (1.0, 2.187 4) 10 → STOP

SS Rao (Prentice Hall 2002) IVP for ODEs CUT’2011 5 / 14

Euler’s Methody’=y+2x-1, y(x=0)=1

assuming h = 0.1 we have 10 iterations for 0 ≤ x ≤ 1

yi+1| xi+0.1 = yi + 0.1 (yi + 2xi − 1)(xi , yi ) i → (xi+1, yi+1) i+1

initial value (0, 1) 0 → y1 = 1+ 0.1 (1+ 2 · 0− 1) = 1→ (0.1, 1) 1(0.1, 1) 1 → y2 = 1+ 0.1 (1+ 2 · 0.1− 1) = 1.02→ (0.2, 1.02) 2(0.2, 1. 02) 2 → y3 = 1.02+ 0.1 (1.02+ 2 · 0.2− 1) = 1.062→ (0.3, 1.062) 3

. . .(0.9, 1.915 8) 9 →y10 = 1.915 8+ 0.1 (1.915 8+ 2 · 0.9− 1) = 2.187 4→ (1.0, 2.187 4) 10 → STOP

SS Rao (Prentice Hall 2002) IVP for ODEs CUT’2011 5 / 14

Euler’s Methody’=y+2x-1, y(x=0)=1

assuming h = 0.1 we have 10 iterations for 0 ≤ x ≤ 1

yi+1| xi+0.1 = yi + 0.1 (yi + 2xi − 1)(xi , yi ) i → (xi+1, yi+1) i+1

initial value (0, 1) 0 → y1 = 1+ 0.1 (1+ 2 · 0− 1) = 1→ (0.1, 1) 1(0.1, 1) 1 → y2 = 1+ 0.1 (1+ 2 · 0.1− 1) = 1.02→ (0.2, 1.02) 2(0.2, 1. 02) 2 → y3 = 1.02+ 0.1 (1.02+ 2 · 0.2− 1) = 1.062→ (0.3, 1.062) 3. . .

(0.9, 1.915 8) 9 →y10 = 1.915 8+ 0.1 (1.915 8+ 2 · 0.9− 1) = 2.187 4→ (1.0, 2.187 4) 10 → STOP

SS Rao (Prentice Hall 2002) IVP for ODEs CUT’2011 5 / 14

Euler’s Methody’=y+2x-1, y(x=0)=1

assuming h = 0.1 we have 10 iterations for 0 ≤ x ≤ 1

yi+1| xi+0.1 = yi + 0.1 (yi + 2xi − 1)(xi , yi ) i → (xi+1, yi+1) i+1

initial value (0, 1) 0 → y1 = 1+ 0.1 (1+ 2 · 0− 1) = 1→ (0.1, 1) 1(0.1, 1) 1 → y2 = 1+ 0.1 (1+ 2 · 0.1− 1) = 1.02→ (0.2, 1.02) 2(0.2, 1. 02) 2 → y3 = 1.02+ 0.1 (1.02+ 2 · 0.2− 1) = 1.062→ (0.3, 1.062) 3. . .(0.9, 1.915 8) 9 →y10 = 1.915 8+ 0.1 (1.915 8+ 2 · 0.9− 1) = 2.187 4→ (1.0, 2.187 4) 10 → STOP

SS Rao (Prentice Hall 2002) IVP for ODEs CUT’2011 5 / 14

Euler’s Method, Errorsy’=y+2x-1, y(x=0)=1

xi yi Euler y (xi ) exact absolute error % relative error

0.0 1.00000.1 1.0 1.0103 0.0103 1.01950.2 1.02 1.0428 0.0228 2.186 40.3 1.062 1. 0997 0.0377 3.61530.4 1.128 2 1.1836 0.0554 4.68060.5 1.221 1. 2974 0.0764 5.88870.6 1.343 1 1. 4442 0.1011 7.00040.7 1.497 4 1. 6275 0.1301 7.99390.8 1.687 1 1. 8511 0.164 8.85960.9 1.915 8 2.1192 0.2034 9.59801.0 2.187 4 2.4366 0.2492 10.227

SS Rao (Prentice Hall 2002) IVP for ODEs CUT’2011 6 / 14

Runge-Kutta Methods

do not require the derivatives of f (x , y)

yi+1 = yi + hα (xi , yi , h)

where α (xi , yi , h) is the increment function

α (xi , yi , h) - the average slope over interval xi ≤ x ≤ xi+1α (xi , yi , h) = c1k1 + c2k2 + . . .+ cnkn

where: n - the order of Runge-Kutta method; ci - constants;ki - recurrence relations given by

k1 = f (xi , yi )

k2 = f (xi + p2h, yi + a21hk1)

k3 = f (xi + p3h, yi + a31hk1 + a32hk2)...

kn = f (xi + pnh, yi + an1hk1 + an2hk2 + . . .+ an,n−1hkn−1)

SS Rao (Prentice Hall 2002) IVP for ODEs CUT’2011 7 / 14

Runge-Kutta Methods

do not require the derivatives of f (x , y)

yi+1 = yi + hα (xi , yi , h)

where α (xi , yi , h) is the increment functionα (xi , yi , h) - the average slope over interval xi ≤ x ≤ xi+1

α (xi , yi , h) = c1k1 + c2k2 + . . .+ cnkn

where: n - the order of Runge-Kutta method; ci - constants;

ki - recurrence relations given by

k1 = f (xi , yi )

k2 = f (xi + p2h, yi + a21hk1)

k3 = f (xi + p3h, yi + a31hk1 + a32hk2)...

kn = f (xi + pnh, yi + an1hk1 + an2hk2 + . . .+ an,n−1hkn−1)

SS Rao (Prentice Hall 2002) IVP for ODEs CUT’2011 7 / 14

Runge-Kutta Methods

do not require the derivatives of f (x , y)

yi+1 = yi + hα (xi , yi , h)

where α (xi , yi , h) is the increment functionα (xi , yi , h) - the average slope over interval xi ≤ x ≤ xi+1

α (xi , yi , h) = c1k1 + c2k2 + . . .+ cnkn

where: n - the order of Runge-Kutta method; ci - constants;ki - recurrence relations given by

k1 = f (xi , yi )

k2 = f (xi + p2h, yi + a21hk1)

k3 = f (xi + p3h, yi + a31hk1 + a32hk2)...

kn = f (xi + pnh, yi + an1hk1 + an2hk2 + . . .+ an,n−1hkn−1)

SS Rao (Prentice Hall 2002) IVP for ODEs CUT’2011 7 / 14

Second-Order Runge-Kutta Methods

yi+1 = yi + h (c1k1 + c2k2)

Modified Euler’s Method

yi+1 = yi + hk2

k1 = f (xi , yi ) , k2 = f(xi +

h2, yi +

12hk1

)

Ralston’s Method

yi+1 = yi +13h (k1 + 2k2)

k1 = f (xi , yi ) , k2 = f(xi +

34h, yi +

34hk1

)Heun’s Method- the simplest one

yi+1 = yi +12h (k1 + k2)

k1 = f (xi , yi ) , k2 = f (xi + h, yi + hk1)

SS Rao (Prentice Hall 2002) IVP for ODEs CUT’2011 8 / 14

Second-Order Runge-Kutta Methods

yi+1 = yi + h (c1k1 + c2k2)

Modified Euler’s Method

yi+1 = yi + hk2

k1 = f (xi , yi ) , k2 = f(xi +

h2, yi +

12hk1

)Ralston’s Method

yi+1 = yi +13h (k1 + 2k2)

k1 = f (xi , yi ) , k2 = f(xi +

34h, yi +

34hk1

)

Heun’s Method- the simplest one

yi+1 = yi +12h (k1 + k2)

k1 = f (xi , yi ) , k2 = f (xi + h, yi + hk1)

SS Rao (Prentice Hall 2002) IVP for ODEs CUT’2011 8 / 14

Second-Order Runge-Kutta Methods

yi+1 = yi + h (c1k1 + c2k2)

Modified Euler’s Method

yi+1 = yi + hk2

k1 = f (xi , yi ) , k2 = f(xi +

h2, yi +

12hk1

)Ralston’s Method

yi+1 = yi +13h (k1 + 2k2)

k1 = f (xi , yi ) , k2 = f(xi +

34h, yi +

34hk1

)Heun’s Method- the simplest one

yi+1 = yi +12h (k1 + k2)

k1 = f (xi , yi ) , k2 = f (xi + h, yi + hk1)SS Rao (Prentice Hall 2002) IVP for ODEs CUT’2011 8 / 14

Second-Order Runge-Kutta Method, Heun’s Methody’=y+2x-1, y(x=0)=1

for f (x , y) = y + 2x − 1 we have:

k1 = f (xi , yi ) = yi + 2xi − 1k2 = f (xi + h, yi + hk1) = (yi + hk1) + 2 (xi + h)− 1

= (yi + h (yi + 2xi − 1)) + 2 (xi + h)− 1

yi+1 = yi + 12h (k1 + k2)

yi+1 = yi + h(xi (2+ h) + yi

(1+

h2

)+

(h2− 1

))

for h = 0.1yi+1 = 0.21 · xi + 1.105 · yi − 0.095

SS Rao (Prentice Hall 2002) IVP for ODEs CUT’2011 9 / 14

Second-Order Runge-Kutta Method, Heun’s Methody’=y+2x-1, y(x=0)=1

for f (x , y) = y + 2x − 1 we have:

k1 = f (xi , yi ) = yi + 2xi − 1k2 = f (xi + h, yi + hk1) = (yi + hk1) + 2 (xi + h)− 1

= (yi + h (yi + 2xi − 1)) + 2 (xi + h)− 1

yi+1 = yi + 12h (k1 + k2)

yi+1 = yi + h(xi (2+ h) + yi

(1+

h2

)+

(h2− 1

))

for h = 0.1yi+1 = 0.21 · xi + 1.105 · yi − 0.095

SS Rao (Prentice Hall 2002) IVP for ODEs CUT’2011 9 / 14

Second-Order Runge-Kutta Method, Heun’s Methody’=y+2x-1, y(x=0)=1

for f (x , y) = y + 2x − 1 we have:

k1 = f (xi , yi ) = yi + 2xi − 1k2 = f (xi + h, yi + hk1) = (yi + hk1) + 2 (xi + h)− 1

= (yi + h (yi + 2xi − 1)) + 2 (xi + h)− 1

yi+1 = yi + 12h (k1 + k2)

yi+1 = yi + h(xi (2+ h) + yi

(1+

h2

)+

(h2− 1

))

for h = 0.1yi+1 = 0.21 · xi + 1.105 · yi − 0.095

SS Rao (Prentice Hall 2002) IVP for ODEs CUT’2011 9 / 14

Heun’s Method, continuedy’=y+2x-1, y(x=0)=1

yi+1 = 0.21 · xi + 1.105 · yi − 0.095

initial value (0, 1) 0 → (0.1, 1. 01) 1

y1 = 0.21 · 0+ 1.105 · 1− 0.095 = 1.01

(0.1, 1.01) 1 → (0.1, 1.0421) 2

y2 = 0.21 · 0.1+ 1.105 · 1. 01− 0.095 = 1.0421

. . .(0.9, ) 9 → (1.0, 2.4282) 10 → STOP

y10 = 0.21 · 0.9+ 1.105 · 2.1124− 0.095 = 2.4282

SS Rao (Prentice Hall 2002) IVP for ODEs CUT’2011 10 / 14

Heun’s Method, continuedy’=y+2x-1, y(x=0)=1

yi+1 = 0.21 · xi + 1.105 · yi − 0.095

initial value (0, 1) 0 → (0.1, 1. 01) 1

y1 = 0.21 · 0+ 1.105 · 1− 0.095 = 1.01

(0.1, 1.01) 1 → (0.1, 1.0421) 2

y2 = 0.21 · 0.1+ 1.105 · 1. 01− 0.095 = 1.0421

. . .(0.9, ) 9 → (1.0, 2.4282) 10 → STOP

y10 = 0.21 · 0.9+ 1.105 · 2.1124− 0.095 = 2.4282

SS Rao (Prentice Hall 2002) IVP for ODEs CUT’2011 10 / 14

Heun’s Method, continuedy’=y+2x-1, y(x=0)=1

yi+1 = 0.21 · xi + 1.105 · yi − 0.095

initial value (0, 1) 0 → (0.1, 1. 01) 1

y1 = 0.21 · 0+ 1.105 · 1− 0.095 = 1.01

(0.1, 1.01) 1 → (0.1, 1.0421) 2

y2 = 0.21 · 0.1+ 1.105 · 1. 01− 0.095 = 1.0421

. . .

(0.9, ) 9 → (1.0, 2.4282) 10 → STOP

y10 = 0.21 · 0.9+ 1.105 · 2.1124− 0.095 = 2.4282

SS Rao (Prentice Hall 2002) IVP for ODEs CUT’2011 10 / 14

Heun’s Method, continuedy’=y+2x-1, y(x=0)=1

yi+1 = 0.21 · xi + 1.105 · yi − 0.095

initial value (0, 1) 0 → (0.1, 1. 01) 1

y1 = 0.21 · 0+ 1.105 · 1− 0.095 = 1.01

(0.1, 1.01) 1 → (0.1, 1.0421) 2

y2 = 0.21 · 0.1+ 1.105 · 1. 01− 0.095 = 1.0421

. . .(0.9, ) 9 → (1.0, 2.4282) 10 → STOP

y10 = 0.21 · 0.9+ 1.105 · 2.1124− 0.095 = 2.4282

SS Rao (Prentice Hall 2002) IVP for ODEs CUT’2011 10 / 14

Heun’s Method, Errorsy’=y+2x-1, y(x=0)=1

xi yi Heun y (xi ) exact absolute error % relative error

0.0 1.00000.1 1.01 1.0103 0.0003 2. 969 4× 10−20.2 1.0421 1.0428 0.0007 6. 712 7× 10−20.3 1.0985 1. 0997 0.0012 0.109120.4 1.1818 1.1836 0.0018 0.152080.5 1.2949 1. 2974 0.0025 0.192690.6 1. 4409 1. 4442 0.0033 0.22850.7 1. 6232 1. 6275 0.0043 0.264210.8 1. 8456 1. 8511 0.0055 0.297120.9 2. 1124 2.1192 0.0068 0.320881.0 2. 4282 2.4366 0.0084 0.34474

SS Rao (Prentice Hall 2002) IVP for ODEs CUT’2011 11 / 14

Forth-Order Runge-Kutta Method

yi+1 = yi +h6(k1 + 2k2 + 2k3 + k4)

where

k1 = f (xi , yi )

k2 = f(xi +

12h, yi +

12hk1

)k3 = f

(xi +

12h, yi +

12hk2

)k4 = f (xi + h, yi + hk3)

SS Rao (Prentice Hall 2002) IVP for ODEs CUT’2011 12 / 14

Forth-Order Runge-Kutta Method, Errorsy’=y+2x-1, y(x=0)=1

xi yi 4th R-K y (xi ) exact absolute error % relative error

0.0 1.00000.1 1.0103 1.0103 0 00.2 1.0428 1.0428 0 00.3 1.0997 1. 0997 0 00.4 1.1836 1.1836 0 00.5 1.2974 1. 2974 0 00.6 1.4442 1. 4442 0 00.7 1. 6275 1. 6275 0 00.8 1. 8511 1. 8511 0 00.9 2.1192 2.1192 0 01.0 2.4366 2.4366 0 0

Note: for five digits precisionSS Rao (Prentice Hall 2002) IVP for ODEs CUT’2011 13 / 14

% Relative Errorsy’=y+2x-1, y(x=0)=1

xi Euler’s Method Heun’s Method 4th R-K

0.00.1 1.0195 2. 969 4× 10−2 00.2 2.186 4 6. 712 7× 10−2 00.3 3.6153 0.10912 00.4 4.6806 0.15208 00.5 5.8887 0.19269 00.6 7.0004 0.2285 00.7 7.9939 0.26421 00.8 8.8596 0.29712 00.9 9.5980 0.32088 01.0 10.227 0.34474 0

SS Rao (Prentice Hall 2002) IVP for ODEs CUT’2011 14 / 14