AN EXTRAPOLATED EULER METHOD OF SECOND-ORDER …

AN EXTRAPOLATED EULER METHOD OF SECOND-ORDER ACCURACY

FOR STOCHASTIC DIFFERENTIAL EQUATIONS

by

SALLY THERESA GOODLETT, B.S.

A THESIS

IN

MATHEMATICS

Submitted to the Graduate Faculty of Texas Tech University in

Partial Fulfillment of the Requirements for

the Degree of

MASTER OF SCIENCE

Approved

Accepted

May, 1992

ACKNOWLEDGMENTS

I would like to thank the committee members of my thesis for their

assistance. And I would like to thank especially my chairman, Prof. Edward

Allen, for placing his knowledge and expertise at my disposal. His guidance

has been invaluable.

I would also like to thank my family for their assistance and

understanding throughout my years of study at Texas Tech University. Most of

all, I would like to thank my husband, Sean, for his encouragement and support.

n

TABLE OF CONTENTS

ACKNOWLEDGMENTS ................................................................................................. ii

ABSTRACT ..................................................................................................................... .iv

LIST OF TABLES ............................................................................................................. v

CHAPTER

I. INTRODUCTION ...................................................................................... 1

II. THEORETICAL ANALYSIS ................................................................... 3

Ill. VARIANCE REDUCTION METHODS ................................................ 15

IV. NUMERICAL RESULTS ...................................................................... 19

v. CONCLUSION ........................................................................................ 26

BIBLIOGRAPHY ............................................................................................................. 27

lll

ABSTRACT

An extrapolated Euler method is developed for numerical solutions of

stochastic differential equations. It is proven that expectations of functions of the

stochastic process and expectations of solutions of systems of stochastic

differential equations are approximated to second-order accuracy using the

extrapolated Euler method. Numerical results support the theoretical analysis.

In addition, a new variance reduction procedure is easily implemented with

Euler's method and is described and tested.

IV

LIST OF TABLES

4.1 Euler's Method for a Scalar Linear Equation ............................................... 19

4.2 Richardson Extrapolation for a Scalar Linear Equation ............................ 20

4.3 Euler's Method for a Scalar Nonlinear Equation ........................................ 20

4.4 Richardson Extrapolation for a Scalar Nonlinear Equation ...................... 20

4.5 Euler's Method for a Function of a Scalar Linear Equation ...................... 21

4.6 Richardson Extrapolation for a Function of a Scalar Linear Equation .... 21

4. 7 Euler's Method for a Function of a Scalar Nonlinear Equation ................ 22

4.8 Richardson Extrapolation for a Function of a Scalar Nonlinear Equation ............................................................................................................... 22

4.9 Euler's Method for a System of Linear Equations ....................................... 23

4.10 Richardson Extrapolation for a System of Linear Equations .................... 23

4.11 Euler's Method for a System of Nonlinear Equations - Solution of y1 ..................................................................................................................... 24

4.12 Richardson Extrapolation for a System of Nonlinear Equations -Solution of Y1 ..................................................................................................... 24

4.13 Euler's Method for a System of Nonlinear Equations - Solution of Y2 ..................................................................................................................... 25

4.14 Richardson Extrapolation for a System of Nonlinear Equations -Solution of Y2 ..................................................................................................... 25

v

CHAPTER I

INTRODUCTION

Considered in this thesis are stochastic differential equations of the form

{dy(t) = f(t,y(t))dt + g(t,y(t))dW(t)

y(to) = YO (1·1)

where y(t) is a random variable, f(t,y(t)) and g(t,y(t)) are functions of timet and

the random process y(t), and W(t) is the Wiener process. It is assumed that

functions f and g satisfy the following conditions given by Gard [2] such that (1.1)

has a unique solution: the functions f and g are measurable with respect tot

and x, forte [O,T] and x e 1R; the functions f and g are also be Lipschitz

continuous and exhibit linear growth in x.

It is often the case that one must find numerical solutions of expectations

of functions of y(t). Most numerical methods for solving (1.1) yield

approximations to expectations of functions of only first order. However,

Milshtein [4,5], Klauder-Petersen [3] and Talay [6] have developed methods of

O(h2) where h is the time interval in the numerical scheme. Unfortunately, these

methods are complicated and may involve first and second derivatives of the

functions f and g. Therefore, for some practical problems, these methods may

be very difficult to implement. It should be noted that Wagner's scheme [7] for

avoiding systematic error due to time discretization based on unbiased

estimation of the transition density of the solution process was not considered in

this investigation.

In the present investigation, an extrapolated Euler method is used to

approximate expectations of functions. This method is very simple to

implement, because it involves only the application of Richardson extrapolation

1

to two approximations (with different step sizes) of Euler's method. In the scalar

case, it is proven that expectations of functions of the random process are

approximated with second-order accuracy using this method. It is also proven

that the extrapolated Euler method approximates expectations of solutions to

systems of stochastic differential equations with second-order accuracy.

Several numerical examples are given which support the theoretical analysis.

Theoretical work concerning the expectations of functions of systems of

stochastic differential equations remains for future work. However, numerical

examples indicate that the accuracy may be O(h2) for expectations of functions

for systems and no counter-examples have been found in this investigation to

indicate order less than two.

To be useful in practical problems, a variance reduction procedure is

implemented to reduce statistical errors. A variance reduction method

developed by Chang [1] appeared to be particularly suited to the incorporation

into the extrapolated Euler method. Chang's method was extended in the

present investigation to reduce further statistical errors. Since this new method

is simple to implement, it is suitable for many problems. Numerical tests

indicate that the statistical variation can be dramatically reduced using the new

variance reduction procedure.

2

CHAPTER II

THEORETICAL ANALYSIS

In this chapter, theoretical analysis prove that the extrapolated Euler

method gives second-order accurate approximations concerning both scalar

stochastic differential equations and systems of stochastic differential equations.

In the scalar case, this method is shown to provide approximations of

expectations of functions of the random variable y(t) which are accurate to

O(h2). The extrapolated Euler method is also proven to be accurate to O(h2) in

approximating the expectations of solutions to systems. In the following

lemmas and theorems, expectations of stochastic differential equations will be

taken. Thus, it is useful to first evaluate the expectation of the following

stochastic differential equation:

dy = f(t,y(t))clt + g(t,y(t))dW(t). (2.1)

Taking the integral of both sides of (2.1) over the interval [O,h], we obtain

h h h

Jdy = Jt(t,y(t))clt + Jg(t,y(t))dW(t). (2.2) 0 0 0

h

From Gard [2], we know the expectation of the integral Jg(t,y(t))dW(t) is zero. 0

Thus,

E{jdy} = ~;(t,y(t))~} and E{y(h)- y(O)} = E{F(h,y(h))- F(O,y(O)} (2.3)

where F'(t,y(t)) = t(t,y(t)). Dividing (2.3) by h and taking the limit as h approaches

zero, we obtain

lim Ey(h) - Ey(O) lim EF(h,y(h)) - EF(O,y(O)) h~O h = h~O h

which by definition is dEy= Ef(t,y(t))dt.

3

Now we consider the scalar case. We require the following lemma on local

error.

Lemma 1

Let y and y be one-step and two-step Euler approximations, respectively,

to the exact value y(h). Then, assuming functions f(y(t)) and g(y(t)) are

sufficiently smooth,

a. E[y(h)- (2Y1 -y1)] = O(h3)

b.

c.

d.

Proof:

E[y2(h) - (2~- y~)] = O(h3)

E[y3(h) - (2~- y~)] = O(h3)

E[y4(h) - (2y~- y~)] = O(h3).

First, consider the following scalar equations which are derived using

Ito's formula.

d[y(t)- YO] = f(y(t))dt + g(y(t))dW

d[(y(t)- YO)"]= [n(y(t)- YO)n-1f(y(t)) +~ n(n-1)(y(t)- YO)n-2g2(y(t))]dt +

n(y(t)- YO)n-1g(y(t))dW

Hence,

dE[y(t) -YO] = Ef(y(t))dt and (2.4)

dE[(y(t)- YO)"]= E[n(y(t)- YO)n-1f(y(t)) + ~ n(n- 1 )(y(t)- YO)n-2g2(y(t))]dt. (2.5)

By expanding equations (2.4) and (2.5) into Taylor series about yo, we obtain

dE[y(t) - YO] = E[f(YO) + (y(t) - YO)f'(YO) + ~ (y(t) - Y0)2f"(YO) + ... ]dt ( 2. 6)

dE[(y(t) -YO)"] = E[n(y(t)- YO)n-1 f(YO) + n(y(t)- YO)n f'(YO) +

~ (y(t) - YO)n+ 1 f"(YO) + ~ (n- 1 )(y(t) - YO)n-2 g2(yo) + (2. 7)

~ (n- 1 )(y(t)- yo)"-1 (g2(yo))' + ~ (n- 1 )((y(t)- yo)"(g2(yo))" + ... ]dt.

4

Let F 1 = E(y(t) - YO), F2 = E(y(t) - Y0)2, F3 = E(y(t) - Y0)3, and F 4 = E(y(t) - Y0)4.

Substituting F1 , F2 , F3 , and F4 into equations (2.6) and (2.7), we obtain

.... dF .... .... .... .... dt (t) = AF(t) + b + O(tt3) with F(O) = 0

where b = [ f 92 o o ]T, F(t) = [ F1 (t) F2(t) F3(t) F4(t) ]T, and A is the matrix

f' 1 f" 2

1 f"' 6

_1 f"" 24

2f + (g2)' 2f' + 1.. (g2)" 2

f" + 1.. (g2)'" 6

1 f"' + _1 (g2)"" 3 24

3g2 3f + 3(g2)' 3f' + ~ (g2)" 2

~ f" + 1.. (g2)"' 2 2

0 6g2 4f + 6(g2)' 4f' + 3(g2)" .... ....

Note that in b and A, all functions are evaluated at t = 0. Now, expanding F(h)

in a Taylor's series about t = 0, we obtain

.... .... dF h2d2'F F(h) = F(O) + hdt (0) +2 dt2 (0) + O(h3).

.... 2 .... Note that ~~ (O) = b, and ~t: (0) =A b. Thus, clearly

1f f' + !...f"g2 2 4

. 1 1 f2 + fgg' + f'g2 + 2 g3g" + 2 g2(g')2 (2.8)

3fg2 + 3g3g• 3g4

By equation (2.8), the moments, F1 , F2 , F3 , and F4, are given to an accuracy of

O(h3). Now, consider the one-step and two-step Euler methods where, for

convenience, Yo = 0.

y 1 = h f(YO) + ...Jh Q(Y0)1lO 1\ h - Til_ Y1 /2 = 2 f(YO) + 'J 2 Q(Y0)1l1

1\ 1\ hA _Th_A Y1 = Y1/2 +2f(Y1/2) + 'J 29(Y1/2)1l2·

(2.9)

(2.1Qa)

(2.10b)

Note that 110. 111, 112 are independent normal random variables with unit

variance and E[11oJ = E[111J = E[112J = o. Now, by expanding functions f(Y1/2) and

5

Q(Y1/2) in Taylor series about yo, we obtain from (2.1 0)

Ey1 = h f(YO) +f h2 f(YO) f'(YO) +i h2 f"(YO) g2(yo) + O(h3).

By applying Richardson extrapolation, we obtain

E(2Y1- Y1) = h f +i h2f f' +f h2 f"g2 + 0(h3). (2.11)

Similarly, we have

E(2~- y~) = hg2 + h2t2 + h2fgg' +i h2g2(g')2 + ~ h2g3g" + O(h3) (2.12)

E(2¢;- y~) = 3h2fg + ~2g3g• + O(h3) (2.13)

1\4 -4 2 4 3 E(2y1-y1)=3h g +O(h ). (2.14)

In equations (2.11 ), (2.12), (2.13) and (2.14), all the functions are evaluated at

t = 0. From equations (2.8) and (2.11) where t = h and Yo = 0, we see

E[y(h) - (2Y1 - y 1)] = O(h3).

Similarly, we have

E[y2(h) - (2~- y~)] = O(h3)

3 1'3 -3 3 E[y (h)- (2y1- y 1)] = O(h )

.A 1\4 -4 3 E[y ·(h)- (2y1- y 1)] = O(h ).

This completes the proof. Notice that it is only necessary to consider the first

four moments as E[y(t)- yo]n is inherently O(h3) for n > 4. Also, it is

straightforward to show using the same procedure that E[y"(h)- y~] = O(h2)

for n = 1 ,2,3, 4 where y is the one-step Euler method. We are now ready to

show that the local error in approximating EF(y(h)) is of O(h3).

Theorem 1

Let y(t) and y(t) be one-step and two-step Euler approximations,

respectively, to y(t). Then, for a function F(y(t)) where t = h,

E[F(y(h)) - (2 F(y(h))- F(y(h))] = O(h3).

6

Proof:

First, by expanding function F(y(h)), F(y(h)) and F(y(h)) about yo, we obtain

F(y(h)) = F(yo) + (y(h) - YO) P(YO) + i (y(h) - Y0)2F"(YO) + ...

A A 1 A 2 F(y(h)) = F(yo) + (y(h)- YO) F'(YO) +2 (Y(h)- YO) F"(YO) + ···

F(y(h)) = F(YO) + (y(h)- YO) F'(YO) +i (y(h)- Y0)2P'(YO) + ....

Thus, E[F(y(h))- (2 F(y(h))- F(y(h)) 1 = E[y(h)- (2Y1 - Y1)] F'(YO) +

~ E[(y(h)- Y0)2 - (2(y(h)- Y0)2 - (y(h)- Y0)2)] P'(YO) +

i E[(y(h) - Y0)3 - (2(y(h)- Y0)3 - (y(h) - Y0)3)] P"(YO) +

214 E[(y(h)- Y0)4 - (2(y(h)- Y0)4 - (y(h)- Y0)4)] F"'(YO) + O(h3).

Clearly, by Lemma 1 we know that

E[y(h) - (2y(h)- y(h))] = O(h3)

and E[(y(h) - Y0)2 - (2(y(h)- Y0)2- (y(h) - Y0)2)] = E[(y2(h)- (2~- y~))- 2yo(y(h)- (2y(h)- y(h))] = O(h3).

Similarly,

E[(y(h) - Y0)3 - (2(y(h) - Y0)3 - (y(h) - Y0)3)] = O(h3)

and E[(y(h)- Yo)4 - (2(y(h)- Y0)4 - (y(h)- Y0)4)] = O(h3).

Thus, E[F(y(h)) - (2 F(y(h))- F(y(h))] = O(h3).

This completes the proof. The following lemma is required to show global error.

Lemma 2

Let y andy be one-step and two-step Euler approximations, respectively,

to the exact value y(t) for any t = kh. Then, assuming functions f(y(t)) and g(y(t))

are sufficiently smooth,

a. E[y(t)- (2y(t)- y(t))] = O(h2)

b. E[y2(t)- (2y2(t)- y2(t))] = 0(~)

c. E[y3(t) - (2y3(t)- y3(t))] = O(h2)

d. E[y4(t) - (294(t)- y4(t))] = O(h2).

7

Proof:

As in Lemma 1, let F (t) be a vector of length four with components ~

Fn(t) = E[y(t)- Yo]" for n = 1 ,2,3,4. ~

Let Gi be the vector with components

~ A -(Gi)n = E[2(Yi- Yo)"- (Yi- Yo)"] for n = 1 ,2,3,4.

Let II • II be the maximum norm, i.e., II vII= 1 ~~ 4 I vi I.

It is known from Lemma 1 that the local error satisfies II F(h) -G1 II = O(h3). The

global error is now considered. It is straightforward from (2.4)- (2.7) to find a ~

function <I> such that

F(ti+1) = F(tj) + h<i>(h.F(tj)) + o(~).

Similarly,

~+ 1 = Gj + h'i'(h,Gj) + O(h3).

Furthermore, by the local error result Lemma 1

II <i>(h,z) - i¥(h,z) II = O(~.

Defining, the local error ei =II F(ti)-Gi II. we have

ei+1 = ei + hll ~h.F(ti))- 'i'(h,Gi) II+ O(h3)

~ ej +hi I ~h.F(tj)) _ljl(h.F(tj)) II+ hllljl(h.F(tj)) _ljl(h,Gj) II+ O(~). ~

Thus, ei+1 ~ ei + hLei + O(h3), assuming the'¥ is Lipschitz continuous in the second

variable with Lipschitz constant L. (This is guaranteed provided that f and g are

sufficiently smooth.) Thus, the above inequality implies that

ei+ 1 ~ O(h2) for any i. We are now ready to prove the extrapolated Euler method

approximates expectations of functions of the random variable y(t) with an

accuracy of O(h2).

8

Theorem 2

Given the local error, E[F{y{h))- {2F{y{h))- F{y{h))], has an accuracy of O(h3)

where y(h) and y(h) are the one-step and two-step Euler's methods,

respectively, then the global error satisfies

E[F(y(t))- {2F(y{t))- F(y(t))] = O(h2)

where t = hm is fixed time.

Proof:

Define the e(t) as

" -e{t) = E[F(y{t))- (2F(y(t))- F(y(t))].

By expanding the function F(y(t)) in a Taylor series about yo, we obtain

" -e(t) = E[F'(Yo)[y(t) - (2y(t)- y(t))) +

~ F"(Yo)[(y(t) - Yo)2 - (2(y(t) - Yo)2- (y(t) - Yo)2))] + ...

1 1 1 2 = P(yo)e1 (t) +2 F"(yo)e2(t) + 6 F"'(yo)e3(t) + 24 F'"'(Yo)e4(t) + O(h )

where en(t) = E[(y(t)- yo)n- (2(y(t)- y0)n- (y(t)- y0)n)] and the O(h2) term arises due to

errors in moments greater than four which have inherent local errors of O(h3).

The proof is now completed by applying Lemma 2. Note that Theorem 2

applies only to scalar equations. We now prove that the extrapolated Euler

method approximates expectations of solutions of systems of stochastic

differential equations with an accuracy of O(h2). First the following lemma on

local error is required.

Lemma 3 1\ -Consider a system of stochastic differential equations. Let ya(t) and ya(t)

be one-step and two-step Euler approximations, respectively, to ya(t) where

9

1 ::;; a and J3::;; K, subject to some initial condition . Then, assuming functions

fa(y(t)) and gp(Y(t)) are sufficiently smooth,

a. E[ya(h} - (2ya(h} - ya(h))) = O(h3)

b. E[ya(h)y~(h)- (2Rh)y~(h)- ya(h)y~(h))] = 0(~).

Proof:

The integral of the stochastic differential equation

dya(t) = fa(y(t))dt + gp(y(t))dW~(t)

has the formal solution, fort = h, h h

ya(h) = yg + Jta(y'Y(s))ds + J 9p(Y'Y(s))T)~(s)ds. 0 0

Klauder-Petersen [3] show this solution has deterministic part, D(ya(h)), and a

stochastic part, S(ya(h)) where D(ya(h)) and ES(ya(h)) are

D(y~h)) = yg + h fa+~ h2t ~.~ + 0(~) (2.15)

where t.~ = ~ta(y'Y)I~y~ I y 'Y = yJ

d (l 1 2 Jl v (l 0 3 an ES(y (h))= 4" h gtgtf,Jlv + (h ). (2.16)

Klauder-Petersen [3] also show that the covariance of the stochastic part of the

formal solution has the form

E[S(ya(h)S(y~(h)] = hgpg~ +~ h2~~f~ +~ h2g~~g~~ +

(2.17)

It is now shown that the Euler approximations have the identical deterministic

and stochastic parts (to O(h3)) hence implying (a) and (b). Expanding ya(h) and

10

ya(h) in Taylor series about yg we obtain

/\-.. a 1 2 R.a ~a R ~ a R h R a II 't yu(h) = y + h fa +- h f 1-'T r:t + - g 11'"' + - a.; 11'"' +- g'"' g R 11,..11 + 0 4 .... 2 p 0 2 -p 1 2 ll 't,p 0 1

_1_ h3/2gPf ~n! + _1_ h3/2fP~ R'Jl 't + _1_ ~/2gP gil Q; r:t n!Tl all 't + 2...J2 'Y •1-' 'U 2...[2 •1-' 1 4...[2 y a •I-'ll \J 0 1

1 2 pva 'Y.A 1 2 pllAa "(.acj>'t 8 h g'YgAf,pyllollo + 24 h 9y9a9cp9't,PilA 11Q1lo1lo1l1 +

~ h2 g~Ag~pATlcill~ + 0(~)

and ya(h) = yg + h fa+ ...ft. Q~Tl~ + 0(~)

where E[TlO] = E[111J = E[1121 = o.

Thus,

(2.18)

(2.19)

(2.20)

and E[2ya(h)yA(h)- ya(h)yA(h)] = D*(y(h),y(h),a,A) + S*(y(h),y(h),a,A) + O(h3) (2.21)

where .. ,.. - a A A a + a A 1 2 A r:t.a 1 2 A p v a

D (y(h),y(h),a,A)= Yo y0 + h y0 f hy0 f + 2 h y0f 1-'l,p +4 h y0g'Yg'Yf ,pv +

1 2 a r:t.A 1 2 a p v a ._~ 2 h Yo f 1-'l,p +4 h Yo 9y9yf ,pv + O(n-)

d - * ,.. - a A 1 2 A p a 1 2, . .A.p a an ES (y(h),y(h),a,A) = hQp9p +2 h 9y9yf ,p +2 h ~ 9y,p +

1 2 paa A 1 2 Aalla 1 2 apA 2 h 9iJ y9v,p9v,a + 4 h 9y9 p9p9y,CJJJ. + 2 h 9y 9/ ,p +

1 2apA 1 2aallA 3 2 h 9y f 9y,p + 4 h 9y 9p9p9y,CJJJ. + O(h ).

Thus from (2.20) and letting ya(h) = 2ya(h)- ya(h), we obtain

D(ya(h)) = yg + h fa+~ h2f ~.~ + 0(~)

and

1 1

(2.22)

(2.23)

From equations (2.15), (2.16), (2.17), (2.20), (2.21 ),(2.22) and (2.23) we obtain

D(y~h))- D(ya(h)) = O(h3),

ES(ya(h)) - ES(ya(h)) = O(h3),

and E[S(ya(h)S(y~(h)) - ES*(y(h),y(h),a,A.) = O(h3).

Hence, for ea(t) = ya(t)- ya(t),

Eea(t+h) = ea(t) + hE(D(ya(t))- D(ya(t))) + 0(h3).

This completes the proof. We are now ready to show the global error in

approximating expectations of solutions of systems of stochastic differential

equations is O(h2).

Theorem 3 A ...

Consider a system of stochastic differential equations. Let y<X(t) and ya(t)

be one-step and two-step Euler approximations, respectively, to ya(t) where

1 ~a and~~ K, subject to some initial condition. Then, assuming functions

fa(y(t)) and g~(y(t)) are sufficiently smooth, the global error satisfies

E[ya(t) - (2ya(t)- ya(t))) = 0(h2)

where t = km is a fixed time.

Proof:

First, we will define the notation for the cumulants. Let

« ya ,, = E[ya]

« yay A. , = E[yayA.].

Furthermore, let

za(ti+ 1) = 2ya(ti+ 1) - ya(ti+ 1)

vaA.(ti+ 1) = 2ya(ti+ 1 )yA(t;+ 1) - ya(ti+ 1 )yA(ti+ 1).

12

For the analytic solution, we can find functions .11 and .12 such that

a a « Yi+ 1 ,, = cc Yj ,, + hA1 (h,yi)

(2.24)

Similarly, for the approximate solution

a a 11 -« zi+ 1 ,, = « ~ ,, + hel>1 (h,yi.Yi)

aA. a A. 11 -« vi+ 1 ,, = « ~ zi ,, + hel»2(h,yj,yj)

(2.25)

where by Lemma 3 I Cl>1 (h,yj,yj)- .11 (h,yj) I ~ C1 h2,

(2.26)

Let the errors be denoted by e1 and e2. where e1 = cc Zj~1 ,, - '' y~1 ,, and

aA. a A. e2 = « vi+1 ,, - cc Yi+1Yi+1 ». Thus, by subtracting (2.24) from (2.25), we have

(2.27)

e2(ti+ 1) = e2(ti) + h{ Cl>2(h,yi.Yi)- C1>2(h,yj,yi) + Cl>2(h,yj,yi)- .12(h,yi) }.

Analogous to Klauder-Petersen [3], e1>1 and C1>2 are expanded to the second

order in the cumulants and thus obtaining

"" 11 - "" a a aA. a A. "'1 (h,yi.Yi) - "'1 (h,yj,yi) = a1 { cc ~+ 1 , - cc Yi+ 1 , } + a2{ cc '1+ 1, - cc Yi+ 1 Yi+ 1 , } (2.28)

where a1, a2. b1 and b2 are the expansion coefficients. Thus, to this order in the

cumulants from (2.26), (2.27) and (2.28), we find that

le1(ti+1)l ~ le1(ti)l+hla1lle1(ti)l+hla2lle2(ti)l + c1h3,

I e2(ti+1) I ~ I e2(ti) I+ hI b1 II e1 (ti) I+ hI b2ll e2(ti) I + c2h3.

Let II • II be the norm defined as II e II= I e1 I+ I e2l. Letting L be the constant

L = max{ I a1 I+ I b1 I. I a2l +I b2l } and c = c1 + c2. we find

13

II e(ti+ 1 > II = II e(ti) II + hLII e(ti) II + ctr3.

Equation (2.29) is easily solved for a fixed time t = kh, giving k-1

II e(tk) II~ (1 + hL)II e(O) II+ L(1 + hL)jch3. j=O

Because the initial error, II e(O) 11. is zero, we have

II e(tk) II ~ ( 1 + ~t>k - 1 ch3 ~ eLh~ - 1 ch2.

Thus, the proof is complete.

14

(2.29)

CHAPTER Ill

VARIANCE REDUCTION METHODS

Through the application of Richardson Extrapolation in conjunction with

Euler's method, we have reduced the deterministic error involved in

approximating stochastic differential equations. The stochastic error can also

be reduced by implementing a variance reduction technique. In this chapter,

we consider different variance reduction techniques for both scalar stochastic

differential equations and systems of stochastic differential equations. We

consider two variance reduction techniques, namely Chang's method [1] and a

new method, applied in unison with Euler's method in order to evaluate the

expectations of functions of scalar stochastic differential equations. We

also show that both methods are easily modified to evaluate expectations of

solutions of systems of stochastic differential equations. Although not

considered here, the methods can be modified to reduce the stochastic error in

evaluating functions of solutions of systems of stochastic differential equations.

First, we describe Chang's variance reduction method.

Chang's Variance Reduction Method

Consider the scalar stochastic differential equation

dy = f(t,y)dt + g(t,y)dW(t).

Recall that Euler's method for solving (3.1) is

Yk+ 1 = Yk + hfk + ..JhgkZk

where E[zk] = o and Var{zk] = 1. Let k = 0. Thus Euler's method for Y1 is

Y1 = YO + hfo + ..Jhgozo.

(3.1)

Define F(y1) to be a function of the stochastic variable Y1· By expanding F(y1) in

15

a Taylor series about yo, we obtain

F(Y1) = F(yo) + F'(YO)(hfO + ..Jhgozo) + ....

Define F* (Y1) as

F•(Y1) = F(YO) + F'(YO)[hfQ) + ... = F(Y1)- F'(yo)[..Jhgozo).

Notice that E[F.(Y1)] = E[F(y1)] and it is likely that Var[F.(y1)] s Var[F(y1)].

Thus,

F• (Y1) = F(YO) + [F(Y1) - F(yQ) - ..JhF'(YO)QQZQ].

Now consider k = 1. By Euler's method, we obtain

Y2 = Y1 + hf1 + ..Jhg1 Z1

and by expanding F(y2) in a Taylor series about Y1. we obtain

F(Y2) = F(Y1) + F'(Y1)[hf1 + ..Jhg1z1] + ....

Thus,

F•(Y2) = F•(Y1) + [F(y2)- F(Y1)- -ftlF'(Y1)Q1Z1]·

Again, we see that E[F. (Y2H = E[F(y2)] and it is likely that Var[F. (Y2)] s Var[F(Y2H·

Hence, F.(Yk) can be defined recursively as

F• (y~~ 1) = F• (y~)) + [F(y~~ 1)- F(y~))- ..JhF'(y~))QkZk]

fork= 0,1,2, ... and i = 1, ... , N where N is the number of time intervals. Note that n

EF(y(t)) =~ :2l·(y~ where Mh = t.

i=1

Chang's method can easily be extended to solutions of systems of stochastic

differential equations. Consider the system .. ...llrro.... ..a.. ...

dy = f (Y (t))clt + g(y(t))dW(t)

where y(t) = [Y1 (t), Y2(t), ... , Yn(t)]T, f (y) = [f1 (y),f2(y), ... ,fn(Y)]T and g is an n x n

matrix of functions of y (t). Thus, Chang's variance reduction technique for

systems is

16

I

where y k = [1, 1, .... , 1 ]T fork= 0,1 ,2, ... and i = 1 , ... , N where N is number of time

intervals.

Now, by modifying Chang's variance reduction method, a new variance

reduction technique is developed which further reduces the variance.

New Variance Reduction Method

Define F(yk) to be a function of the stochastic variable Yk· Let k = 0. Thus

Euler's method for Y1 is

Y1 =YO+ hfo + -Yhgozo.

Expanding F(y1) in a Taylor series about yo, we obtain

F(Y1) = F(yo) + F'(yo)[hfo + -Yhgozo] + i F"(yo)[h2t~ + 2-Yhhtogozo + hg~z~] + ....

Define u(yk) and v(yk) fork= 0 as

1 2 2 2 2 u(yo) = F'(yo)[hfo] + 2 F"(YO)[h t0 + hg0z0]

and v(yo) = F'(yo)[hto] + i F"(YO)[h2t~ + hg~.

Note that v is just a nonstochastic function of YO·

Thus,

F(y1) = F(yo) + F'(yo)[-Yhgozo] + F"(yo)[..Jhhtogozo] + u(yo) + ....

* Now, define F (Y1) as

F* (Y1) = F(Y1) - \l'hF'(YO)QOZO- \l'hhF"(YO)fOQOZO- U(YO) + v(yo).

Note that E[F. (Y1 )] = E[F(Y1 )]. We define v*(Y1) using Chang's method and thus

obtaining

v*(Y1) = v(yo) + [v(y1)- v(yo)- -Yhv'(yo)gozo].

17

Thus, in general we can define the new method as

v*(ii)) = v*(ii) ) + [v(y(i))- v(y(i) ) - -.Thv '(y(i) )g(i) z(i) ] k k-1 k k-1 k-1 k-1 k-1

F* (y~~ 1) = F* (y~)) + [F(y~~ 1)- F(y~)- -ftiF'(y~))QkZk- -ftihF"(y~))fkQkZk

- u(y~))] + v*(y~))

fork= 0,1 ,2, ... and i = 1 ,2, ... ,N where N is the number of time intervals.

Note that

v(yk) = hF'(yk)fk + ~ F"(Yk)[h2~ + h~

d 1 22 22 an u(yk) = hF'(yk)fk +2 F"(Yk)[h fk + hgkzk].

Thus, v'(yk) = hF'(yk)f'k + F''(Yk)[h2fkf'k + hQkQ'k + hfk] + F"'(Yk)~ hg~ +~ h2f~].

The new variance reduction method is easily extended to solutions of systems

of stochastic differential equations. Consider the system

dy = 1 &(t))dt + g(y(t))dW(t)

where y(t) = [Y1(t), Y2(t), ... , Yn(t)]T, f(y) = [f1(Y).f2(Y) •... ,fn(Y)]T and g is ann x n matrix of

functions of y(t). The new variance reduction method for the solution of systems

of stochastic differential equations is defined as

v*(y .... (i)) = v*(y .... (Q ) + rv ,~(i) ) - -ftlv·(y .... (Q )g(i) z-(i) 1 k k-1 \1 k-1 k-1 k-1 k-1

!Y~+1 J(il = IY~J(Q +{!Yk+1J(il- IYkJ(il- ,,hly~(Qg~Z~ - ii~1l} +V·&~1l

fork= O, 1 ,2, ... and i = 1 ,2, ... ,N where N is the number of time intervals.

Note that

v(yk) = hf k and u(yk) = hf k·

.... .... {51 k 5 f k 5 f k} Thus, v'(yk)=h ~ + ~ + ... + ~. uY1 uY2 oYn

In a similar manner, the variance reduction method can be extended to

functions of solutions of systems of stochastic differential equations.

18

CHAPTER IV

NUMERICAL RESULTS

We present several numerical examples which indicate O(h2) accuracy

acquired by applying the extrapolated Euler method. Both variance reduction

methods are used to show the decrease of stochastic error when the new

variance reduction is applied in comparison to Chang's variance reduction

method. We first provide numerical examples for the scalar stochastic

differential equations. Linear and nonlinear examples are given for both

approximations of expectations of solutions of stochastic differential equations

and functions of stochastic differential equations. In every example 1 0,000

simulations were performed.

Example 1 : Numerical results for y(t)

{ dy = ~dt + ydW(t)

y(O) = 1

Fort = 1.0, the exact value of Ey(t) is 1.6487213 (see tables 4.1 and 4.2).

Table 4.1: Euler's Method for a Scalar Linear Equation

Chang's Method New Method

#of time Expected Standard Expected Standard

intervals Value Deviation Value Deviation

2 1.55914 0.169478 1.56249 0.006836

4 1.60353 0.285398 1.60204 0.026704

8 1.62238 0.359972 1.62394 0.047025

16 1.63535 0.404022 1.63625 0.060508

19

Table 4.2: Richardson Extrapolation for a Scalar Linear Equation

Chang's Method New Method #of time Expected Richardson Expected Richardson intervals Value Extrapolation Value Extrapolation

2 1.55914 1.56249 4 1.60353 1.64792 1.60204 1.64159 8 1.62238 1.64123 1.62394 1.64584 16 1.63535 1.64832 1.63625 1.64856

Example 2: Numerical results for y(t)

{ dy = (3y213 + 3y113)dt + 3y213dW(t)

y(O) = 1

Fort= 1.0, the exact value of Ey(t) is 14 (see tables 4.3 and 4.4).

Table 4.3: Euler's Method for a Scalar Nonlinear Equation




2 9.9892 1.8163 9.9915 1.5143

4 12.278 3.7204 12.233 1.4030

8 13.297 5.3767 13.326 1.2796

16 13.672 6.4234 13.676 1.3080

Table 4.4: Richardson Extrapolation for a Scalar Nonlinear Equation


#of time Expected Richardson Expected Richardson

intervals Value Extrapolation Value Extrapolation

2 9.9892 9.9915

4 12.278 14.664 12.233 14.475

8 13.297 14.316 13.326 14.419

16 13.672 14.047 13.676 14.026

20

Example 3: Numerical results for F(y(t)) = y3(t)

{ dy = ~dt + ydW(t)

y(O) = 1

Fort= 1.0, the exact value of EF(y(t)) = Ey3(t) is

90.017131 (see tables 4.5 and 4.6).

Table 4.5: Euler's Method for a Function of a Scalar Unear Equation




2 14.3271 28.2689 14.4303 15.2022

4 25.1840 121.2213 25.5036 56.6832

8 40.2521 177.2859 40.8489 110.7710

16 51.0455 302.9312 54.6746 192.0087

32 70.3139 666.4928 72.4737 422.3107

Table 4.6: Richardson Extrapolation for a Function of a Scalar Unear Equation




2 14.3271 14.4303

4 25.1840 36.0409 25.5036 36.5769

8 40.2521 55.3202 40.8489 56.1942

16 51.0455 61.8389 54.6746 68.5003

32 70.3139 89.5823 72.4737 90.2728

21

Example 4: Numerical results for F(y(t)) = y2(t) where

{ dy = (3y213 + 3y113)dt + 3y213dW(t)

y(O) = 1

Fort= 1.0, the exact value of EF(y(t)) = Ey2(t) is

499 (see tables 4. 7 and 4.8).

Table 4.7: Euler's Method for a Function of a Scalar Nonlinear Equation

Chanq's Method New Method



2 146.1652 178.1462 146.0280 74.0173

4 264.2405 466.5475 266.4706 293.3166

8 356.7456 759.6386 360.3555 544.7186

16 419.3520 1021.878 424.4428 777.2277

32 462.9625 1452.823 461.8799 1160.892

Table 4.8: Richardson Extrapolation for a Function of a Scalar Nonlinear

Equation




2 146.1652 146.0280

4 264.2405 382.3158 266.4706 386.9132

8 356.7456 449.2507 360.3555 454.2404

16 419.3520 481.9584 424.4428 488.5301

32 462.9625 506.5730 461.8799 499.3170

We now present numerical examples for systems of stochastic differential

equations. A linear and a nonlinear example are given for the approximation of

expectations of the solutions of systems.

22

Example 5: Numerical results for Y1 and Y2

#of time

intervals

2

4

8

16

{dY1 = f1dl + Y2dW1 (I)

dy2 = 2)'2dt + Y1 dW2(t)

Y1(0) = 1

Y2(0) = 1

Fort = 1.0, the exact value of Ey1 (t) = Ey2(t) is

1.6487213 (see tables 4.9 and 4.1 0).

Table 4.9: Euler's Method for a System of Linear Equations

Chang's Method for Y1 and Y2 New Method for Y1 and Y2

Expected Standard Expected Standard Value Deviation Value Deviation

1.5626 0.17768 1.5625 0.012010

1.5986 0.28276 1.6014 0.026085

1.6245 0.35612 1.6240 0.046799

1.6374 0.40269 1.6362 0.060350

Table 4.10: Richardson Extrapolation for a System of Linear Equations

Chang's Method for Y1 and Y2 New Method for Y1 and Y2



2 1.5626 1.5625

4 1.5986 1.6346 1.6014 1.6403

8 1.6245 1.6504 1.6240 1.6466

16 1.6374 1.6503 1.6362 1.6484

23

Example 6: Numerical results for Y1 and Y2

{

2 1~~ dy1 = y2dt + (y2e-Y1 - 1 )dW1 (t) + 4V y2 + 1 dW2(t)

dy2 = -~ Y2dt + ..J 2y1 + 6 dW1(t)

Y1 (0) = 1

Y2(0) = 1

Fort= 1.0, the exact value of Ey1 (t) is 1.8504061 (see tables 4.11 and 4.12)

and of Ey2(t) = 0.6065307 (see tables 4.13 and 4.14).

Table 4.11: Euler's Method for a System of Nonlinear EquationsSolution of Y1

Chang's Method for Y1 New Method for Y1



2 1.9244 0.20994 1.9236 0.12129

4 1.8906 0.24314 1.8911 0.22051

8 1.8780 0.26035 1.8763 0.25978

16 1.8684 0.28427 1.8663 0.26479

Table 4.12: Richardson Extrapolation for a System of Nonlinear Equations -Solution of Y1




2 1.9244 1.9236

4 1.8906 1.8568 1.8911 1.8586

8 1.8780 1.8654 1.8763 1.8615

16 1.8684 1.8588 1.8663 1.8563

24

Table 4.13: Euler's Method for a System of Nonlinear EquationsSolution of Y2


#of time Expected Standard Expected Standard intervals Value Deviation Value Deviation

2 0.56189 0.17764 0.56247 0.00622 4 0.58691 0.21029 0.58610 0.02367 8 0.59738 0.22604 0.59676 0.03583 16 0.60197 0.22993 0.60147 0.04169

Table 4.14: Richardson Extrapolation for a System of Nonlinear EquationsSolution of Y2




2 0.56189 0.56247

4 0.58691 0.61193 0.58610 0.60973

8 0.59738 0.60785 0.59676 0.60742

16 0.60197 0.60656 0.60147 0.60618

The numerical examples given show that Euler's method is a first-order

method. When Richardson extrapolation is applied, the accuracy of the method

increases to second-order. This increase in accuracy is consistent regardless

to the variance reduction technique used.

25

CHAPTERV

CONCLUSION

The extrapolated Euler method has been proven to approximate the

expectations of functions of scalar stochastic differential equations and of

solutions of systems of stochastic differential equations with second-order

accuracy. Second-order accurate results were found regardless to which

variance reduction technique, Chang's method or the new method, was

implemented. It should be noted, however, that because the new variance

reduction technique reduces more stochastic error, better results were found

when the new variance reduction technique was used.

Theoretic work concerning the expectations of functions of systems of

stochastic differential equations remains for future work. Numerical examples

indicate that accuracy may be O(h2) for expectations of functions for systems

and no counter-examples were found to indicate order less than two.

26

BIBLIOGRAPHY

[1] Chien-Cheng Chang, Numerical Solution of Stochastic Differential Equations with Constant Diffusion Coefficients, Mathematics of Computation, 49, 523-542, 1987.

[2] Thomas C. Gard, Introduction to Stochastic Differential Equations, Marcel Dekker, Inc., New York, 1988.

[3] John R. Klauder and Wesley P. Petersen, Numerical integration of multiplicative-noise stochastic differential equations, SIAM Journal on Numerical Analysis, 22, 1153-1166, 1985.

[4] G. N. Milshtein, Approximate integration of stochastic differential equations, Theory of Probability and its Applications, 19, 557-562, 1974.

[5] G. N. Milshtein, A method of second-order accuracy integration of stochastic differential equations, Theory of Probability and its Applications, 23, 396-401' 1978.

[6] D. Talay, Efficient numerical schemes for the approximation of expectations of functionals of the solution of a stochastic differential equation, and applications, Lecture Notes in Control and Information Science, 61, 294-313, 1984.

[7] Wolfgang Wagner, Monte Carlo evaluation of functionals of solutions of stochastic differential equations. Variance reduction and numerical examples, Stochastic Analysis and Applications, 6, 447-468, 1988.

27

Documents

AN EXTRAPOLATED EULER METHOD OF SECOND-ORDER …